The online network X has sparked significant backlash after people employed its AI chatbot Grok to manipulate photos of women by taking off their garments.
Several examples have been seen where the system has been employed to virtually strip women to make them appear in swimwear without permission, as well as to place them in sexualized contexts.
Company Reaction and Regulatory Stance
XAI, the organization responsible for Grok, remained silent to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to prohibit nudification software. Under a upcoming criminal offence, anyone who supplied such tools would "receive a prison sentence and substantial fines".
"X or Grok has the ability to stop these types of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the potential" of people accessing unlawful material on their networks, but did not specify whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for features—which responds to X users' prompts when they tag it in a tweet.
It is commonly used to provide commentary or additional background to other users' remarks, but people on X are also capable of modify an posted image through its AI image editing feature.
Background of Criticism and Policy
The tool has been condemned for permitting users to create images and clips with explicit content and sexualised material, and it was earlier accused of generating graphic material of public figures.
"X has been permitting the production and distribution of these pictures for months without taking any steps and we have not witnessed any challenge by regulators," the expert added.
XAI's own terms of service forbids "portraying images of individuals in a pornographic fashion".
In a comment, Ofcom said it was against the law to "make or distribute deepfakes or child sexual abuse material" and stated this included explicit AI images created with AI.
It said networks such as X were required to take "necessary measures" to "reduce the risk" of users encountering illegal material on their services and to take down it swiftly when they become aware of it.
