The online network X has caused significant backlash after individuals used its AI chatbot Grok to manipulate pictures of females by stripping their garments.
Multiple instances have emerged where the tool has been employed to digitally undress women to show them in swimwear non-consensually, as well as to insert them in sexualized situations.
Company Reaction and Regulatory Position
XAI, the firm responsible for Grok, offered no comment to inquiries, except for an standard message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to outlaw nudification technology. Under a proposed criminal offence, anyone who supplied such technology would "be subject to a prison sentence and substantial penalties".
"The platform could prevent these forms of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their networks, but stopped short of confirming whether it was at this time examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' prompts when they reference it in a tweet.
It is commonly used to give commentary or further background to other users' remarks, but people on X are also capable of edit an posted picture through its AI image editing tool.
History of Criticism and Policy
The system has been faulted for permitting users to create photos and videos with explicit content and sexually suggestive content, and it was earlier accused of generating graphic content of public figures.
"The platform has been enabling the production and sharing of these images for months without taking any action and we have yet to see any intervention by regulators," the expert added.
XAI's own usage rules prohibits "portraying images of persons in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "create or distribute non-consensual intimate images or child sexual abuse material" and confirmed this included explicit AI images generated with AI.
It said platforms such as X were mandated to take "necessary steps" to "reduce the danger" of users encountering unlawful content on their platforms and to delete it promptly when they become aware of it.
