Apple tightens App Store rules on nonconsensual intimate imagery and AI nudify apps

Apple has revised its App Review Guidelines to explicitly address nonconsensual intimate imagery, including pictures produced with AI tools, according to a post on its developer news site. The company presented the changes as a strengthening and extension of existing App Store protections. Apple noted that pornographic material has long been barred from the store, and said this already covers apps that produce nudity or sexually explicit pictures, such as "nudification" apps that depict real people in explicit images without their consent.

Under guideline 1.2, Apple said its content moderation requirements now name nonconsensual intimate imagery, material generated with AI included, as content that apps must proactively prevent and take down. According to the guidelines text, the rule applies to apps that let people create, generate, upload or share content, a group that explicitly includes social media services, user-generated content platforms and AI content creation apps. Such apps must offer a way to filter objectionable material before it is posted, a system for reporting offensive content with timely responses, a way to block abusive users, and published contact details.

Apple said developers that lack adequate safeguards, or that do not fix violations after being notified, could be found in breach of the Developer Code of Conduct and face removal from the App Store. The guidelines describe the process: if Apple finds violating content, it will ask the developer to remove it and to provide a plan for improving compliance. Depending on the response, the app may be pulled from the store until the developer can show improvements, and egregious or repeated behavior can lead to immediate removal of the app and expulsion from the Apple Developer Program.

A second change concerns guideline 2.5.19, which Apple said clarifies a longstanding ban on apps that misrepresent their function during App Review and then change once approved, for example by adding features absent from the submitted version. Apple's post calls this tactic bait-and-switch and cites nudification capabilities revealed after approval as an example. The guidelines text says apps must not contain hidden, dormant or undocumented features, lists code obfuscation as one method of concealment, and warns of removal from the store and termination of the developer account. Engadget framed the rule mainly around apps that conceal nudification features.

Apple also said it updated Section 5 of the guidelines to state that apps that enable human trafficking or child exploitation, including child sexual abuse material, will be taken off the App Store and also barred from alternative distribution; the company said such apps were always prohibited. On enforcement, Apple said that last year it kept more than 150K apps with safety problems away from users and closed more than 193K developer accounts over fraud and abuse. It attributed these violations to malicious actors, in some cases coordinated fraud operations, that hide harmful content until after an app is approved.