The social media network X has sparked significant outrage after individuals used its AI chatbot Grok to manipulate images of females by taking off their clothing.
Multiple instances have surfaced where the technology has been used to remove clothing from women to show them in bikinis non-consensually, as well as to insert them in sexualized situations.
Developer Response and Legal Stance
XAI, the firm that created Grok, offered no comment to requests for comment, other than an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being brought forward to outlaw image stripping software. Under a upcoming criminal offence, anyone who supplied such technology would "be subject to a prison sentence and heavy penalties".
"The platform could prevent these forms of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the risk" of people seeing illegal content on their platforms, but did not specify whether it was at this time looking into X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some premium features—which answers X users' queries when they reference it in a post.
It is frequently used to offer reactions or more background to other posters' remarks, but people on X are also can edit an uploaded picture through its artificial intelligence picture modification function.
History of Controversy and Rules
The chatbot has been criticised for allowing users to produce photos and footage with explicit content and sexually suggestive content, and it was previously alleged to have been involved in generating graphic material of celebrities.
"The platform has been enabling the generation and sharing of these pictures for months without taking any measures and we have not seen any intervention by regulators," the expert added.
XAI's own acceptable use policy prohibits "showing likenesses of persons in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or circulate deepfakes or illegal content" and clarified this included explicit AI images made with AI.
It said networks such as X were obligated to take "necessary steps" to "lessen the danger" of users coming across unlawful content on their platforms and to take down it swiftly when they become aware of it.
