The online network X has caused widespread outrage after users used its AI chatbot Grok to change images of women by removing their clothing.
Numerous cases have surfaced where the system has been used to digitally undress female subjects to make them appear in bikinis without permission, as well as to insert them in inappropriate scenarios.
Developer Response and Regulatory Position
XAI, the company that created Grok, offered no comment to requests for comment, aside from an standard message stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification tools. Under a new law, anyone who distributed such tools would "be subject to a prison sentence and heavy fines".
"The platform could prevent these types of abuse if they wanted to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the danger" of people accessing unlawful material on their services, but stopped short of confirming whether it was presently looking into X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some additional features—which responds to X users' requests when they mention it in a post.
It is commonly used to give commentary or more background to other users' remarks, but people on X are also able to edit an posted photo through its AI image editing feature.
History of Criticism and Rules
The chatbot has been criticised for allowing users to generate images and videos with nudity and sexualised material, and it was earlier alleged to have been involved in making inappropriate content of public figures.
"X has been permitting the creation and sharing of these images for months without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "showing images of persons in a pornographic fashion".
In a comment, Ofcom said it was against the law to "generate or distribute private pictures or illegal content" and clarified this covered sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable measures" to "lessen the risk" of users coming across unlawful material on their services and to delete it swiftly when they learn of it.
