The online platform X has sparked widespread outrage after users used its artificial intelligence assistant Grok to manipulate photos of women by stripping their attire.
Several cases have surfaced where the technology has been used to virtually strip women to show them in swimwear without permission, as well as to place them in sexualized contexts.
Developer Reaction and Regulatory Stance
XAI, the firm behind Grok, offered no comment to questions, except for an auto-reply response stating "legacy media lies".
A government official stated that new laws is being brought forward to ban nudification technology. Under a new criminal offence, anyone who supplied such tools would "be subject to a prison sentence and substantial penalties".
"The platform could prevent these forms of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "consider the danger" of people seeing illegal content on their platforms, but did not specify whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some premium functions—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to give commentary or more background to other posters' comments, but people on X are also can modify an posted image through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The chatbot has been faulted for allowing users to create pictures and videos with nudity and sexually suggestive material, and it was earlier accused of creating inappropriate material of celebrities.
"X has been enabling the generation and distribution of these pictures for months without taking any measures and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of people in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "create or distribute deepfakes or child sexual abuse material" and stated this covered explicit AI images made with AI.
It said networks such as X were obligated to take "suitable actions" to "reduce the danger" of users encountering unlawful material on their services and to take down it promptly when they are notified of it.
