With the consumer world waking up to and jumping on the bandwagon of Generative AI, new research by the Data Protection Excellence Centre, the research arm of Straits Interactive, has unveiled significant privacy concerns in Generative AI desktop applications, particularly among startups and individual developers.

A study of 113 apps harnessing Generative AI shows that most of them fall short of data privacy standards, raising concerns about this fast-growing tech’s acceptance by the public. 

According to research from the Data Protection Excellence Centre, the research arm of Straits Interactive, the Singapore-based risk and compliance firm, it has unveiled “significant privacy concerns in Generative AI desktop applications, particularly among startups and individual developers.” (Generative AI is a broad label that’s used to describe any type of AI that can be used to create new text, images, video, audio, code or synthetic data.)

"This study highlights the pressing need for clarity and regulatory compliance in the Generative AI app sphere,” Kevin Shepherdson, CEO of Straits Interactive, said. “As organisations and users increasingly embrace AI, their corporate and personal data could be jeopardised by apps, many originating from startups or developers unfamiliar with privacy mandates.”

The findings of the study come at a time when sectors, including private banking and wealth management, are being shaken up by AI tech in all its forms. (See editorial thoughts here.)

Conducted from May to July this year, the study focused on apps primarily from North America (48 per cent) and the European Union (20 per cent). Selection criteria included recommendations, reviews, and advertisements. 

The apps were categorised as: Core apps – industry leaders in the Generative AI sector; clone apps – typically startups or individual developers/developer teams; and combination apps – existing applications that have incorporated generative AI functionalities.

Some 12 per cent of the apps, predominantly startups and individual developers, lacked a published privacy policy. Of those with published privacy policies, 69 per cent identified a legal basis (such as consent and contract performance) for processing personally identifiable information. Only half of the apps meant for children considered age restrictions and aligned with child privacy standards such as the Children’s Online Privacy Protection Act (COPPA) in the US and/or the General Data Protection Regulation in the European Union.

Though 63 per cent cited the GDPR, only 32 per cent were apparently within the GDPR’s purview. The majority, which are globally accessible, alluded to the GDPR without understanding when it applies outside the EU. Of those cases where GDPR seemed to be relevant, 48 per cent were compliant, with some overlooking the GDPR’s international data transfer requirements.

In terms of data retention, where users often share proprietary or personal data, 35 per cent of the apps did not specify retention durations in their privacy policies, as required by the GDPR or other laws.

Transparency regarding the use of AI in these apps was limited, the report said. Fewer than 10 per cent transparently disclosed AI use or model sources. Out of the 113 apps, 64 per cent remained ambiguous about their AI models, and only one clarified if AI influences user data decisions.

Apart from high-profile players like OpenAI, Stability AI, and Hugging Face, which disclose the existence of their AI models, the remainder primarily relied on established AI APIs, such as those from OpenAI, or integrated multiple models, it said.

The study shows a tendency among apps to collect excessive user personally identifiable information, often exceeding their primary utility. 

Lyn Boxall, a legal privacy specialist at Lyn Boxall LLC and a member of the research team, added: "It's significant that 63 per cent of the apps reference the GDPR without understanding its extraterritorial implications. Many developers seem to lean on automated privacy notice generators rather than actually understanding their app's regulatory alignment.”

“With the EU AI Act on the horizon, the urgency for developers to prioritise AI transparency and conform to both current and emerging data protection norms cannot be overstated." (For a related set of features about EU legislation on AI, see here, here and here.)