xAI’s Grok chatbot has produced sexual deepfakes at a scale that dwarfs dedicated deepfake websites. Regulators on three continents are now trying to stop it.
Between December 2025 and January 2026, Grok generated and posted more than 4.4 million images to X, according to analysis cited in a class action lawsuit against xAI, with an estimated 41% containing sexual imagery of women. A separate 24-hour analysis on January 5-6, reported by Bloomberg, estimated that users had Grok create 6,700 sexually suggestive or nudified images per hour - 84 times more than the top five deepfake websites combined.
Some of those images were of children.
The Scale of the Problem
Grok Imagine, xAI’s image generator, launched in 2025 with a so-called “spicy mode” that can generate adult content. The problem snowballed in late December, when Grok began granting a large number of user requests to modify images posted by others, including requests to “undress” women without consent.
An analysis by the nonprofit AI Forensics of 20,000 images generated between December 25, 2025 and January 1, 2026 found that 2% depicted a person who appeared to be 18 or younger, including 30 of young or very young women or girls in bikinis or transparent clothes.
The UK’s Internet Watch Foundation said that dark web forum users were citing Grok as a tool for making “criminal imagery” of children, after finding topless images of girls that those users attributed to Grok Imagine.
Ashley St. Clair, mother of one of Elon Musk’s children, reported that Grok users were creating fake sexualized images from her photos - including a photo of her as a child. She filed a lawsuit against xAI in the New York Supreme Court on January 15.
The Regulatory Response
Investigations have now been opened on three continents:
United Kingdom: The Information Commissioner’s Office announced formal investigations into X Internet Unlimited Company and X.AI LLC on February 3, 2026. The probe examines whether personal data was processed lawfully, fairly and transparently, and whether appropriate safeguards were built into Grok’s design and deployment to prevent the generation of harmful manipulated images. Ofcom opened its own investigation into X over Grok on January 12.
European Union: The European Commission opened a probe on January 26 under the Digital Services Act. The investigation will assess whether X properly assessed and mitigated risks associated with deploying Grok’s functionalities into X in the EU. X has filed an appeal against a separate €120 million DSA fine imposed in December 2025.
California: Attorney General Rob Bonta launched an investigation on January 14, calling the “avalanche of reports” about xAI’s material “shocking.” On January 16, California sent xAI a cease and desist letter demanding the company immediately halt the creation and distribution of fake sexualized images of children.
US Congress: Democratic senators Ron Wyden, Ben Ray Luján, and Ed Markey wrote to the CEOs of Apple and Google asking them to remove the Grok and X apps from their stores. By January 23, 35 state attorneys general had called on xAI to cease allowing sexual deepfakes.
Southeast Asia: Indonesia and Malaysia became the first countries to block Grok, temporarily cutting access on January 10 and January 11. The Philippines blocked the chatbot on January 16, then lifted the ban on January 21 after xAI committed to removing specific tools to address child safety concerns.
xAI’s Response
Grok acknowledged “lapses in safeguards” and said in a post on X that it was “urgently fixing” them. On January 9, Grok’s X account stopped posting image generation responses to users who were not paid subscribers.
The change closed little. According to reporting by The Verge summarized on Wikipedia, all users could still generate Grok-altered images through X’s “Edit image” feature and the standalone Grok website and app.
A class action lawsuit, Jane Doe v. xAI Corp., filed on January 23 in the US District Court for the Northern District of California, alleges xAI knew Grok’s image generation capabilities were being exploited but failed to implement industry-standard safeguards, and instead monetized the feature by restricting it to paying subscribers.
The Legal Exposure
xAI faces potential liability on multiple fronts:
The EU’s Digital Services Act allows fines of up to 6% of global revenue for systemic failures. X is already contesting one DSA fine while facing a new investigation.
In the UK, Prime Minister Keir Starmer said on February 19 that a new law will require tech companies to remove intimate images shared without consent within two days, or face large fines and potentially have their services blocked.
California’s investigation could trigger enforcement under state consumer protection and privacy laws.
The proposed class in the US lawsuit covers all individuals in the United States who were depicted in sexualized or revealing deepfakes created and disseminated by Grok without their consent.
The Broader Problem
Grok’s failures highlight a gap in AI safety practices. Musk pitches Grok as an edgier alternative to rivals with more safeguards, and the class action complaint alleges that, unlike competitors such as Google and OpenAI, xAI did not use standard data filtration methods to remove sexual and abusive content from Grok’s training data.
On February 23, 61 data protection authorities issued a joint statement, coordinated through the Global Privacy Assembly, on AI systems that generate realistic images of identifiable people without their consent. But enforcement remains fragmented across jurisdictions with different laws and varying political will.
For users whose images were exploited, regulatory action offers uncertain remedy. Even if Grok is eventually restricted or fined, the images it generated are already distributed across the internet. No investigation can undo that.
The Bottom Line
xAI built an image generator marketed as edgier than its competitors, is accused in court of monetizing its worst feature, and is now facing investigations on three continents. Grok said it was fixing the problem. The early evidence suggests otherwise.