Image: grafy.ai
The Story
The apps looked harmless to reviewers because their abusive features appeared only after installation and under selected conditions.
A pet-portrait or fitness app could pass a snapshot review while its real abuse tools stayed behind a conditional web layer. The ordinary explanation is a coordinated bait-and-switch that exploited fragmented checks across app stores, ad networks, and remote servers.
- Platform failureStore and advertising policies were bypassed by checking the visible product rather than every capability delivered after download.
- Detection gapRemote interfaces and targeted behavior make one-time app review a poor match for software that changes what different users see.
- The apps used benign identities such as pet portraits, language tuition, and workout coaching while conditionally loading interfaces for explicit image and video generation.
- Server-side checks involving factors such as language, location, or the advertisement clicked could determine whether users saw the harmless interface or the hidden layer.
- The apps were available through Apple and Google stores and were promoted through Facebook and Instagram advertising despite platform rules against abusive sexual content.
Deeper Insight
Viefur sold pet portraits. The real product hid behind the screen, letting abuse wear a harmless face and outrun accountability.
A pet-portrait app could pass review while a hidden web interface generated sexual and violent deepfakes, including child sexual abuse material. More than 400 apps and sites used the same trick, promoted through major ad platforms. The product was scattered across code, ads, servers, and payments. A harmless storefront became a doorway.
- What it could changeApp reviews would have to test what software can become, not just what its first screen promises.
- How it touches todayNudifying tools already move across app stores, social platforms, search engines, and messaging services.
- Our futureAbuse could become harder to remove when its public face keeps changing faster than accountability can follow.
- A 2025 study identified a malicious technical ecosystem of open-source face-swapping models and nearly 200 nudifying programs, finding major weaknesses in current governance approaches.
- A systematic audit of 155 eligible face-swap apps found that 70% of apps with face-swap functionality had no technical safeguards against generating nude images.
- Earlier investigations found nudification services using app stores, social platforms, search engines, and messaging services together, showing how abuse can move across platform boundaries.
Sources · 8
Featured · Primary source · arxiv.org
Dual-Use AI Face Swap Apps Are Mostly Unsafe: A Systematic Safety Audit
Dual-Use AI Face Swap Apps Are Mostly Unsafe: A Systematic Safety Audit # Dual-Use AI Face Swap Apps Are Mostly Unsafe: A Systematic Safety Audit Alaa Daffalla Affiliation: Cornell University Sarah Chao Affiliation: Georgetown University Eric Zeng Affiliation: Georgetown University ###### Abstract A
Read sourceAnalysis · doi.org
The Malicious Technical Ecosystem: Exposing Limitations in Technical Governance of AI-Generated Non-Consensual Intimate Images of Adults
# The Malicious Technical Ecosystem: Exposing Limitations in Technical Governance of AI-Generated Non-Consensual Intimate Images of Adults arXiv (Cornell University). Published: 2025-04-24. Preprint. 1 citation. ## Authors - Ding, Michelle L. (Brown University) - Harini Suresh (Brown University): h-
Read sourceAnalysis · fakedup.substack.com
🤥 Faked Up #1 - by Alexios - Faked Up
🤥 Faked Up #1 - by Alexios - Faked Up ### Undoing undressers, detecting deepfakes, and Katy Perry's mom Alexios May 08, 2024 ∙ Paid Hi, folks. Thank you for signing up before the first issue of Faked Up was even out. I will work hard to be worthy of your confidence. For complaints, story ideas or t
Read source