The social media network X has caused widespread backlash after people employed its artificial intelligence assistant Grok to change pictures of females by stripping their garments.
Multiple examples have been seen where the system has been used to remove clothing from female subjects to show them in swimwear without consent, as well as to put them in sexually explicit situations.
Company Reaction and Legal Position
XAI, the company responsible for Grok, remained silent to questions, other than an auto-reply response stating "legacy media lies".
A government official stated that new laws is being brought forward to ban image stripping technology. Under a new criminal offence, anyone who distributed such technology would "receive a prison sentence and heavy fines".
"The platform could prevent these types of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the potential" of people viewing unlawful material on their networks, but did not specify whether it was presently examining X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' prompts when they reference it in a post.
It is often used to provide commentary or further background to other users' comments, but people on X are also capable of modify an uploaded image through its artificial intelligence picture modification function.
Background of Controversy and Policy
The system has been criticised for allowing users to produce images and videos with nudity and sexually suggestive content, and it was earlier accused of generating sexually explicit material of celebrities.
"X has been enabling the creation and distribution of these pictures for months without taking any action and we have not witnessed any intervention by regulators," the expert added.
XAI's own acceptable use policy forbids "showing likenesses of persons in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or child sexual abuse material" and stated this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "appropriate steps" to "reduce the danger" of users encountering unlawful content on their platforms and to remove it promptly when they are notified of it.
