Wesley Hilliard
Apple seems unable to stop influx of so-called "dual use" apps that look innocent on the surface but help users create deepfake porn — at a steep price.
Apple takes pride in regulating the App Store, and part of that control is preventing pornographic apps altogether. However, there are limits to this control given that some apps can offer features that users can easily abuse — seemingly without Apple being aware.
According to a report from 404 Media, Apple struggles with a "dual use" problem found in apps that offer features like face swapping. While the feature is innocent enough at first glance, users are swapping faces onto pornography, sometimes using minor's faces.
The issue became apparent when a reporter came across a paid ad on Reddit for a face swap app. Face swapping tends to be easily found and often free, so such an app would need a business model that allows paid ad placement.
What they found was an app offering users the ability to swap any face onto video from their "favorite website," with an image suggesting Porn Hub as an option. Apple doesn't allow porn-related apps on the App Store, but some apps relating to user content often feature such images and videos as a kind of loophole.
When Apple was alerted to the dual-use case of the advertised app, it was pulled. However, it seemed Apple wasn't aware of the issue at all, and the app link had to be shared.
This isn't the first time innocent-looking apps get through app review and offer a service that violates Apple's guidelines. While it isn't as blatant a violation as changing a children's app into a casino, the ability to generate nonconsensual intimate imagery (NCII) was obviously not something on Apple's radar.
Artificial intelligence features in apps can create incredibly realistic deep fakes, and it is important for companies like Apple to get ahead of these problems. While Apple won't be able to stop such use cases from existing, it can at least implement a policy that can be enforced in app review — clear guidelines and rules around pornographic image generation. It already stopped deepfake AI websites from using sign-in with Apple.
For example, no app should be able to source video from Porn Hub. Apple can also have specific rules in place for potential dual-use apps, like zero-tolerance bans for apps discovered trying to create such content.
Apple has taken great care to ensure Apple Intelligence won't make nude images, but that shouldn't be the end of its oversight. Given that Apple argues it is the best arbiter of the App Store, it needs to take charge of such things as NCII generation being promoted in ads.
Face-swapping apps aren't the only apps with a problem. Even apps that blatantly promote infidelity, intimate video chat, adult chat, or other euphemisms get through app review.
Reports have long suggested that app review is broken, and regulators are tired of platitudes. Apple needs to get a handle on the App Store or risk losing control.
William Gallagher
Christine McKee
AppleInsider Staff
Chip Loder
Malcolm Owen
9 Comments
At the same time keep in mind that there are a lot of legislators around the world that want Apple to open up the iPhone and iPad so they can load any store from anyone. Forget about trying to block porn apps, deepfake apps, and things that link to Pornhub. Anything and everything will be available. Of course I suspect a lot of the same people will then try to blame Apple for allowing them on.
A lot of pressure could be taken off Apple's back if Apple allowed users to choose to install either Android or iOS on their phones. This way, users can get third party app stores from Android OS that Apple officially supports (but probably pays Google to do the legwork.) But Apple could keep iOS fully controlled, and I suspect that the vast majority of users would install iOS. If the EU asked Apple for third party app stores, for example, Apple could say, "Sure, we let users install Android on iPhones, and we fully support that. Just not on iOS." This would put the EU in a quandary.
There’s a way of getting rid of them cut the software back to Apples core software. And then have the next 50,000 pieces of vital software to run the hardware and that’s it, But no one‘s going to be happy with that, can’t have it both ways but humans will keep trying…. unsolvable quagmire.