The social media network X has caused significant backlash after people utilized its artificial intelligence assistant Grok to change pictures of females by taking off their clothing.
Several examples have been seen where the system has been used to digitally undress women to make them appear in bikinis non-consensually, as well as to put them in sexualized scenarios.
Developer Response and Regulatory Stance
XAI, the company behind Grok, remained silent to questions, except for an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to ban nudification technology. Under a new law, anyone who distributed such tools would "receive a jail term and heavy fines".
"X or Grok could prevent these forms of misuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Tool Capabilities
The regulator Ofcom said platforms must "assess the potential" of people accessing illegal content on their platforms, but stopped short of confirming whether it was presently examining X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' queries when they tag it in a post.
It is frequently used to offer reactions or more background to other posters' remarks, but people on X are also capable of alter an uploaded image through its AI image editing feature.
History of Criticism and Rules
The tool has been faulted for enabling users to create pictures and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating graphic content of celebrities.
"X has been enabling the creation and distribution of these images for months without taking any action and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "showing likenesses of people in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or illegal content" and confirmed this applied to sexual deepfakes created with AI.
It said networks such as X were required to take "suitable actions" to "minimize the risk" of users encountering illegal content on their platforms and to delete it swiftly when they become aware of it.
