The online platform X has sparked major outrage after users employed its artificial intelligence assistant Grok to change photos of females by removing their clothing.
Several examples have surfaced where the tool has been used to virtually strip women to make them appear in bikinis non-consensually, as well as to place them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the company responsible for Grok, did not respond to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being proposed to ban image stripping software. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the danger" of people viewing illegal content on their services, but stopped short of confirming whether it was at this time examining X or Grok in concerning artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for features—which responds to X users' queries when they reference it in a tweet.
It is frequently used to provide commentary or additional context to other posters' remarks, but people on X are also capable of edit an posted image through its AI image editing function.
Background of Criticism and Rules
The chatbot has been criticised for permitting users to generate photos and footage with explicit content and sexualised material, and it was previously alleged to have been involved in creating sexually explicit material of public figures.
"The platform has been permitting the creation and distribution of these images for months without taking any steps and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "showing likenesses of people in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "make or distribute deepfakes or illegal content" and confirmed this applied to explicit AI images made with AI.
It said networks such as X were mandated to take "suitable actions" to "minimize the danger" of users viewing unlawful material on their services and to take down it promptly when they become aware of it.
