Google is taking aim at potentially problematic generative AI apps with a recent policy, to be enforced starting early next yr, that may require developers of Android applications published on its Play Store to supply the flexibility to report or flag offensive AI-generated content. The brand new policy will insist that flagging and reporting may be done in-app and developers should use the report back to inform their very own approaches to filtering and moderation, the corporate says.
The change to the policy follows an explosion of AI-generated apps, a few of which where users tricked the apps into creating NSFW imagery, as with Lensa last yr. Others, meanwhile, have more subtle issues. For example, an app that went viral this summer for AI headshots, Remini, was found to be greatly enhancing the dimensions of some women’s breasts or cleavage, and thinning them. Then there have been the more moderen issues with Microsoft’s and Meta’s AI tools, where people found ways to bypass the guardrails to make images like Sonic the Hedgehog pregnant or fictional characters doing 9/11.
In fact, there are much more serious concerns around the usage of AI image generators, as pedophiles were discovered using open source AI tools to create child sexual abuse material (CSAM) at scale. And with the approaching elections, there are also concerns around using AI to create fake images, aka deepfakes, to mislead or misinform the voting public.
Image Credits: Google
The text of the brand new policy indicates that examples of AI-generated content includes “text–to-text conversational generative AI chatbots, wherein interacting with the chatbot is a central feature of the app,” which encompass apps like ChatGPT, in addition to apps where images are “generated by AI based on text, image, or voice prompts.”
Google, in its announcement, reminded developers that each one apps, including AI content generators, must comply with its existing developer policies, which prohibit restricted content like CSAM and others that enable deceptive behavior.
Beyond changing its policy to crack down on AI content apps, Google says some app permissions will even receive an extra review by the Google Play team, including those apps that request broad photo and video permissions. Under its recent policy, apps will only have the ability to access photos and videos if it’s directly related to their functionality. In the event that they have a one-time or infrequent need — like AI apps that ask users to upload a set of selfies, perhaps — the apps need to make use of a system picker, just like the recent Android photo picker.
Image Credits: Google
The brand new policy will even limit disruptive, full-screen notifications to only those times when there’s a high-priority need. The power to pop up full-screen notifications has been abused by many apps in an try to upsell users into paid subscriptions or other offers, when really the functionality ought to be limited to real-world priority use cases, like receiving a phone call or video call. Google says it is going to now change the restrictions and requires a special app access permission. This “Full Screen Intent permission” will only be granted to apps targeting Android 14 and above that really require the complete screen functionality.
It’s surprising to see that Google is first out of the gate with a policy on AI apps and chatbots, as historically, it’s been Apple that issues recent rules to crack down on unwanted behavior from apps, which Google then mimics. But Apple doesn’t have a proper AI or chatbot policy in its App Store Guidelines as of yet, though it has tightened up in other areas, like apps’ requesting data for the aim of identifying the user or device, a way often called “fingerprinting,” in addition to on apps that try to copy others.
Google Play’s policy updates are being rolled out today though AI app developers have until early 2024 to implement the flagging and report changes to their apps.