The social media platform X has caused significant backlash after individuals utilized its AI chatbot Grok to manipulate photos of women by removing their attire.
Numerous examples have surfaced where the technology has been employed to remove clothing from female subjects to make them appear in swimwear without permission, as well as to place them in sexually explicit situations.
Developer Reaction and Regulatory Stance
XAI, the company that created Grok, remained silent to inquiries, except for an standard message stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to outlaw image stripping tools. Under a new criminal offence, anyone who supplied such technology would "face a prison sentence and significant penalties".
"The platform has the ability to stop these types of abuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said tech firms must "consider the danger" of people accessing unlawful material on their networks, but stopped short of confirming whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some additional functions—which reacts to X users' requests when they reference it in a post.
It is frequently used to offer reactions or additional background to other posters' remarks, but people on X are also can alter an posted image through its AI image editing function.
History of Criticism and Policy
The tool has been faulted for permitting users to produce pictures and videos with explicit content and sexualised content, and it was previously accused of making inappropriate content of celebrities.
"The platform has been enabling the creation and sharing of these pictures for months without taking any steps and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "showing images of individuals in a graphic fashion".
In a statement, Ofcom said it was unlawful to "create or share deepfakes or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said networks such as X were obligated to take "suitable steps" to "reduce the risk" of users viewing illegal content on their platforms and to remove it swiftly when they are notified of it.
