Two weeks ago, deepfake pornographic images of Taylor Swift spread like fire through X. It took the platform 19 hours to suspend the account that posted the content after they amassed over 27 million views and more than 260,000 likes.

That gave me pause. 260,000 people watched the content, knew it was fake, and felt no shame in sharing their delight publicly. Wow…

I’ve written before about our misconceptions regarding deepfake technology. For example, we’re told that most deepfakes target politicians but the reality is that 96% of deepfakes are of non-consensual sexual nature and 99% of them are from women. I’ve also talked about the legal vacuum regulating the use of this technology.

However, until now I hadn’t delved into the ecosystem underpinning the porn deepfakes: the industry and the viewers themselves. 

Let’s rectify this gap and get to know the key players.

Why is so easy to access porn deepfakes?

We may be led to believe that porn deepfakes are hard to create or find.

False and false.

• It takes less than 25 minutes and costs $0 to create a 60-second deepfake pornographic video. You only need one clear face image.
• I can confirm that when searching on Google “deepfakes porn,” the first hit was MrDeepFake’s website — one of the most famous websites in the world of deepfake porn.

Moreover, the risk of hosting the content is minimal.

Section 230, which passed in 1996, is a part of the US Communications Decency Act. It was meant to serve as protection for private blocking and screening of offensive material. 

However, it has become an ally of porn deepfakes as it provides immunity to online platforms from civil liability on third-party content — they are not responsible for the content they host and they can remove it in certain circumstances, e.g. material that the provider or user considers being obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable.

So whilst Section 230 does not protect platforms that create illegal or harmful content, it exempts them from any responsibility for third-party content.

Who’s making money from porn deepfakes?

Many are profiting from this nascent industry: Creators, deepfake porn websites, software manufacturers, infrastructure providers, marketplaces, and payment processors.

Creators

They get revenue from two main sources:

• Creating porn deepfakes on demand.
• Selling access to “libraries with thousands of videos for subscription fees as low as $5 a month.”

Deepfake porn websites

Let’s have a look at three deepfake porn websites, each with a different business model.

MrDeepFakes

Some highlights of how this platform operates 

• Videos are a few minutes long.
• Generates revenue through advertisement.
• Relies on the large audience that has been boosted by its positioning in Google search results.
• Its forums act as a marketplace for creators and clients can make requests.

Fan-Topia

Their business model 

• It bills itself on Instagram as “the highest paying adult content creator platform.”
• Paywalled.
• Clients may be redirected from sites such as MrDeepFakes afters clicking on the deepfake creators’ profiles. Once in Fan-Topia, they can pay for access to libraries of deepfake videos with their credit cards.

Pornhub

In 2018, the internet pornography giant Pornhub banned deepfake porn from their site. However, that’s not the whole truth

• When Pornhub removes deepfake porn videos from their site, they leave the inactive links as breadcrumbs that act as clickbait to drive traffic to the site.
• Users can advertise the creation and monetisation of porn deepfakes on the site.
• They advertise deepfakes through TrafficJunky, the advertising portal through which Pornhub makes all their ad revenue.
• Pornhub provides a database of abusive content that facilitates the creation of porn deepfakes.

Software manufacturers

A couple of examples

• Stability AI has made their model Stable Diffusion — a deep learning, text-to-image model— open-source, so any developer can modify it for purposes such as creating porn deepfakes. And there are plenty of tips about how to use the models in forums where deepfake porn creators swarm.
• Taylor Swift’s porn deepfake was created using Microsoft Designer, Microsoft’s graphic design app that leverages DALLE-3 — another text-to-image model— to generate realistic images. Users found loopholes in the guardrails that prevented inappropriate prompts that explicitly mentioned nudity or public figures. 

Infraestructure providers

Repositories

GitHub is a Microsoft-owned developer platform that allows developers to create, store, manage, and share their code. It’s also

• One of the top 10 referral sites for Mr.DeepFakes.
• A host of guides and hyperlinks to (a) sexual deepfake community forums dedicated to the creation, collaboration, and commodification of synthetic media technologies, and (b) AI-leveraged ‘nudifiying’ websites and applications that take women’s images and “strip them” of clothing.
• A repository of the source code of the software used to create 95% of deepfakes, DeepFaceLab, as well as other similar codes such as DeepNude and Unstable Diffusion. 
• A gateway for minors to deepfake source codes and related content, given Github’s worldwide partnership program with schools and universities and its terms of service stating that users can be as young as 13. 

Web hosting

According to a Bloomberg review, 13 of the top 20 deepfake websites are currently using web hosting services from Cloudflare Inc. Amazon.com Inc. provides web hosting services for three popular deepfaking tools listed on several websites, including Deepswap.ai.

Marketplaces

Etsy

As of December 2023, AI-generated pornographic images of at least 55 well-known celebrities were available for purchase on Etsy, an American e-commerce company focused on handmade or vintage items and craft supplies.

Moreover, a search for “deepfake porn” on the website returned about 1,500 results. Some of these results were porn and others offers non-explicit services to “make your own deepfake video.”  

Apps stores

Apple’s App Store and Google Play host apps that can be used to create deepfake porn. Some of them are available to anyone over 12.

Payment processors

• On the Fan-Topia payment page, the logos for Visa and Mastercard appear alongside the fields where users can enter credit card information. The purchases are made through an internet payment service provider called Verotel, which is based in the Netherlands and advertises to what it calls “high-risk” webmasters running adult services.
• The MakeNude.ai web app — which lets users “view any girl without clothing” in “just a single click” — has partnered with Ukraine-based Monobank and Dublin’s Beta Transfer Kassa which operates in “high-risk markets”.
• Deepfake creators also use PayPal and crypto wallets to accept payments. Until Bloomberg reached out to Patreon last August, they supported payment for one of the largest nudifying tools, which accepted over $12,500 per month.

Other enablers

Search engines

Between 50 to 80 percent of people searching for porn deepfakes find their way to the websites and tools to create the videos or images via search. For example, in July 2023, around 44% of visits to Mrdeepfakes.com were via Google.

NBC News searched the combination of a name and the word “deepfakes” with 36 popular female celebrities on Google and Bing. A review of the results found nonconsensual deepfake images and links to deepfake videos in the top Google results for 34 of those searches and the top Bing results for 35 of them. 

As for the victims, both Google and Microsoft services require in their content removal requests that people manually submit the URLs.

Social media

More than 230 sexual deepfake ads using Emma Watson and Scarlett Johansson’s faces ran on Facebook and Instagram in March 2023. It took 2 days for Meta to remove the ads, once they were contacted by NBC.

Users of X, formerly known as Twitter, regularly circulate deepfaked content. Whilst the platform has policies that prohibit manipulated media, between the first and second quarter of 2023, the number of tweets from eight hashtags associated with this content increased by 25% to 31,400 tweets.

Who’s watching porn deepfakes?

In their report “2023 State of Deepfakes”, Home Security Heroes state

• There were a total of 95,820 deepfake videos online in 2023.
• The ten-leading dedicated deepfake porn sites had monthly traffic of 35 million in 2023.

What about the deepfake porn consumers?

They surveyed 1522 American males who had viewed pornography at least once in the past six months. Some highlights:

• 48% of respondents reported having viewed deepfake pornography at least once.
• 74% of deepfake pornography users didn’t feel guilty about it. Top reasons they didn’t feel remorse? 36% didn’t know the person, 30% didn’t think it hurt anybody, 29% thought of it as a realistic version of imagination, and 28% thought that it’s not much different than regular porn.

That may lead us to believe that indeed those “watchers” felt porn deepfakes were innocuous. That’s until we learn that 

• 73% of survey participants would want to report to the authorities if someone close to them became a victim of deepfake porn.
• 68% indicated that they would feel shocked and outraged by the violation of someone’s privacy and consent in the creation of deepfake pornographic content.

In summary, non-consensual deepfakes are harmless until your mother and daughter are starring on them. 

if they don’t portray your loved ones.

What’s next?

As with other forms of misogynistic behaviour — rape, gender violence, sexual discrimination — when we talk about deepfake pornography, we focus on the aftermath: the victims and the punishment.

What if we instead focused on the bottom of the pyramid —  the consumers?

• Can we imagine a society where the deepfake porn videos from Taylor Swift would have had 0 views and no likes?
• What will take to raise boys that feel outrage — rather than unhealthy curiosity, lust, and desire for revenge  — at the opportunity to watch and purchase deepfake porn?
• How about believing that porn deepfakes are harmful even if they don’t portray your sister, mum, or wife?

As with physical goods, consumers have the power to transform the offer. Can we collectively lead the way towards a responsible digital future?

PS. You and AI

• ​Are you worried about ​the impact of A​I impact ​on your job, your organisation​, and the future of the planet but you feel it’d take you years to ramp up your AI literacy?
• Do you want to explore how to responsibly leverage AI in your organisation to boost innovation, productivity, and revenue but feel overwhelmed by the quantity and breadth of information available?
• Are you concerned because your clients are prioritising AI but you keep procrastinating on ​learning about it because you think you’re not “smart enough”?

I’ve got you covered.