The social media platform X has sparked widespread outrage after people utilized its AI chatbot Grok to manipulate photos of women by taking off their garments.
Multiple instances have emerged where the tool has been used to remove clothing from female subjects to show them in swimwear without consent, as well as to place them in sexually explicit situations.
Company Reaction and Legal Stance
XAI, the company responsible for Grok, remained silent to questions, except for an automatically-generated message stating "traditional news outlets lies".
A government official stated that legislation is being introduced to prohibit nudification technology. Under a upcoming law, anyone who distributed such technology would "receive a prison sentence and significant penalties".
"X or Grok has the ability to stop these types of abuse if they wanted to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Capabilities
The regulator Ofcom said platforms must "evaluate the risk" of people accessing unlawful material on their networks, but did not confirm whether it was at this time looking into X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some additional functions—which answers X users' queries when they tag it in a message.
It is frequently used to give commentary or additional background to other posters' remarks, but people on X are also capable of modify an uploaded picture through its AI image editing function.
History of Controversy and Policy
The chatbot has been faulted for permitting users to generate pictures and footage with explicit content and sexualised material, and it was previously alleged to have been involved in generating graphic content of public figures.
"X has been permitting the production and spread of these pictures for months without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "showing images of people in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said platforms such as X were obligated to take "appropriate steps" to "reduce the danger" of users encountering unlawful material on their services and to remove it promptly when they are notified of it.
