The online network X has sparked widespread anger after people employed its AI chatbot Grok to change photos of females by stripping their clothing.
Multiple instances have been seen where the technology has been employed to virtually strip women to make them appear in bikinis non-consensually, as well as to insert them in sexualized contexts.
Company Reaction and Legal Stance
XAI, the firm behind Grok, remained silent to questions, other than an standard response stating "traditional news outlets lies".
A Home Office official stated that new laws is being introduced to prohibit image stripping technology. Under a new law, anyone who distributed such technology would "face a prison sentence and significant fines".
"The platform could prevent these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Capabilities
The regulator Ofcom said platforms must "assess the danger" of people seeing unlawful material on their platforms, but did not specify whether it was at this time looking into X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to give commentary or additional context to other users' comments, but people on X are also can edit an posted photo through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been criticised for permitting users to produce pictures and clips with explicit content and sexualised material, and it was in the past alleged to have been involved in creating sexually explicit content of celebrities.
"The platform has been permitting the creation and sharing of these images for a long time without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of people in a graphic fashion".
In a statement, Ofcom said it was illegal to "make or circulate private pictures or child sexual abuse material" and stated this included explicit AI images created with AI.
It said services such as X were required to take "suitable steps" to "minimize the risk" of users viewing illegal content on their services and to remove it quickly when they learn of it.
