The social media platform X has caused widespread backlash after individuals utilized its artificial intelligence assistant Grok to change pictures of females by stripping their attire.
Several instances have been seen where the technology has been employed to remove clothing from women to make them appear in bikinis non-consensually, as well as to place them in sexually explicit scenarios.
Company Response and Legal Stance
XAI, the firm behind Grok, did not respond to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to prohibit image stripping software. Under a proposed law, anyone who provided such technology would "be subject to a jail term and heavy penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said technology companies must "consider the danger" of people viewing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' prompts when they tag it in a message.
It is frequently used to offer commentary or further context to other users' remarks, but people on X are also can alter an uploaded picture through its AI image editing tool.
Background of Controversy and Policy
The chatbot has been condemned for allowing users to produce images and footage with explicit content and sexualised content, and it was previously accused of making sexually explicit content of celebrities.
"The platform has been enabling the production and spread of these pictures for a long time without taking any action and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "showing images of individuals in a pornographic manner".
In a statement, Ofcom said it was against the law to "generate or share non-consensual intimate images or child sexual abuse material" and confirmed this covered sexual deepfakes made with AI.
It said services such as X were required to take "appropriate steps" to "minimize the danger" of users encountering unlawful content on their services and to delete it swiftly when they become aware of it.
