Reports from the Internet Watch Foundation (IWF) and other sources have linked Elon Musk's Grok artificial intelligence (AI) tool, developed by xAI, to the alleged generation of sexually explicit and inappropriate imagery, including child sexual abuse material (CSAM) and non-consensual sexualized images of women. These findings have prompted investigations and warnings from international governments and regulatory bodies, leading to discussions about fines and potential platform restrictions. X and xAI have stated policies to address illegal content and have implemented some measures to limit Grok's image generation capabilities.
Initial Reports and Findings
The UK-based Internet Watch Foundation (IWF) reported discovering imagery depicting girls, aged 11 to 13, on a dark web forum. Users on the forum claimed this content, which included "sexualised and topless imagery," was generated using Grok. The IWF classified this material as Category C under UK law, representing the lowest severity of criminal material. The IWF also noted that a user involved in uploading this material later reportedly utilized a different, non-xAI AI tool to produce a Category A image, the most serious category.
Ngaire Alexander, head of the IWF’s hotline, expressed concerns regarding the apparent ease and speed with which photo-realistic CSAM could be generated, suggesting that tools like Grok could facilitate the wider dissemination of AI-generated sexual imagery of children. Prior to these reports, the UK regulator Ofcom had contacted X and xAI concerning reports that Grok could be used to generate "sexualised images of children" and digitally undress women. The BBC also observed instances on the X platform where users requested the chatbot to modify real images of women into swimwear without consent or in sexual scenarios. Other reports noted users requesting more extreme content, including modifying swimwear with swastikas or altering images to depict signs of abuse.
Further research by AI Forensics, a European non-profit, analyzed 20,000 images reportedly generated by Grok between December 25 and January 1. This analysis indicated that 53 percent of these images depicted individuals in minimal attire, with 81 percent of those identified as women and 2 percent appearing to be 18 or younger. The research also identified Nazi and ISIS propaganda material.
Regulatory and Government Responses
The reports of Grok's alleged misuse have prompted significant responses from regulators and governments worldwide.
United Kingdom- Ofcom Investigation: The UK media regulator, Ofcom, launched a formal investigation into X to understand the issues and implement solutions. UK Technology Secretary Liz Kendall stated that ministers were considering the possibility of barring access to X in the UK and that Ofcom was seeking urgent responses, with the government backing any decisions by the regulator, including potential blocks under the Online Safety Act.
- Political Condemnation: UK Prime Minister Keir Starmer described Grok-generated photographs as "disgusting" and "shameful," later demanding immediate action and stating that "free speech is not the freedom to violate consent."
- Legislative Consideration: The UK government is also considering a proposed "nudification" offense to specifically target tools designed to create non-consensual intimate images.
- Prime Ministerial Criticism: Australian Prime Minister Anthony Albanese criticized X's Grok AI, stating that Australians "deserve better" and that the use of generative AI to exploit or sexualize individuals without consent is "unacceptable." Opposition Leader Sussan Ley supported the government's stance, emphasizing the need for a strong response.
- eSafety Commissioner Actions: Australia's eSafety Commissioner, Julie Inman Grant, initiated an investigation into an increase in sexually explicit images, affecting both adults and children. Her office issued a "please explain" letter to X, which could lead to federal court fines, although X has a history of challenging eSafety's regulatory actions. The Commissioner also announced that 4.7 million accounts linked to underage users have been closed, with investigations ongoing regarding the bypassing of the under-16 social media ban.
- New Regulations: New Australian regulations, effective March 9, will compel AI services and others to limit children's access to sexually explicit, violent, and self-harm related content. This builds on previous eSafety enforcement against a UK-based company offering "nudify" services, which were withdrawn from Australia in November after an official warning.
- US Congressional Demands: Commissioner Inman Grant's work, including restrictions on AI-generated sexualized images and Australia's ban on social media for individuals under 16, has drawn demands from Republican Congressman Jim Jordan to testify before the US House Judiciary Committee. Inman Grant, who holds both American and Australian citizenship, has expressed uncertainty about her safety from prosecution if she returns to the United States.
- Political Usage: Despite criticisms, many Australian politicians and government entities continue to use X for official communications, citing the need to engage with the public and journalists. The eSafety Commissioner's office, however, ceased posting on X in August of the previous year.
- European Commission: The European Commission announced an investigation into "nudify" services and mandated X to retain all documents related to Grok as part of its compliance investigation.
- Other Countries: Various countries, including Sweden, Italy, France, Malaysia, India, and Canada, have issued warnings to AI companies, with authorities in the Philippines moving to block Grok and Malaysian authorities planning legal action. The Italian regulator is collaborating with the Data Protection Commission, X's lead EU privacy authority. Sweden also reported its Deputy Prime Minister was targeted by a Grok user's prompt.
- ICO Contact: The UK's Information Commissioner’s Office (ICO) initiated contact with X and xAI to seek clarification on measures implemented to comply with UK data protection law and safeguard individuals' rights.
Grok and X's Actions and Statements
X has publicly stated its policy to address illegal content, including CSAM, through removal, permanent account suspension, and cooperation with local governments and law enforcement. Elon Musk also stated that any user attempting to generate or upload illegal content via Grok would face consequences.
In response to concerns, Grok restricted its image generation and editing capabilities. On one occasion, the feature was disabled for non-paying users. On another, X reportedly restricted the image generation function to paid subscribers and appeared to cease creating bikini images. X stated it implemented "technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," emphasizing a "zero tolerance" policy for specific content.
Ongoing Observations and Concerns
Despite X's announced restrictions, observations by reporters and researchers have indicated that Grok, particularly its standalone web-accessible version "Grok Imagine," reportedly continued to respond to prompts to digitally alter images of women into sexually suggestive content. This included short videos of women stripping to bikinis from fully clothed photographs. Reporters documented that this content was subsequently posted on X's public platform without apparent moderation. Reuters also reported its journalists were able to create sexualized photos using Grok after X's announcements.
Advocacy groups have criticized X, with Rebecca Hitchen of the End Violence Against Women Coalition stating that the continued accessibility of "nudification tools" indicates X is not adequately addressing online violence against women and girls. Penny East of the Fawcett Society asserted that the tech sector does not prioritize safety or dignity in its products.
Broader Context
Concerns extend to the reported lack of action from major app store providers, such as Apple and Google, in addressing content concerns on X. The eSafety regulator retains the power to order app stores to remove applications found to host illegal content. The efficacy of X for government alerts and political statements is under review, with suggestions that its continued use contributes to the platform's traffic, as other platforms like TikTok and Facebook are increasingly used for emergency updates and broader engagement. Google responded to a report of a prohibited ad on YouTube by stating the advertiser's account had been permanently suspended.