The online platform X has caused significant anger after individuals utilized its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Several examples have surfaced where the technology has been used to digitally undress women to show them in swimwear without consent, as well as to put them in inappropriate situations.
Developer Reaction and Legal Position
XAI, the company behind Grok, offered no comment to requests for comment, except for an standard message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to outlaw image stripping software. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and substantial penalties".
"The platform is capable of halting these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the potential" of people accessing illegal content on their services, but stopped short of confirming whether it was presently looking into X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some additional functions—which answers X users' prompts when they reference it in a post.
It is commonly used to offer reactions or additional context to other users' remarks, but people on X are also can edit an posted picture through its artificial intelligence picture modification function.
Background of Controversy and Policy
The chatbot has been faulted for enabling users to produce photos and videos with nudity and sexually suggestive content, and it was earlier accused of generating inappropriate content of celebrities.
"The platform has been enabling the creation and distribution of these images for a long time without taking any measures and we have not seen any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy bans "depicting likenesses of people in a graphic fashion".
In a statement, Ofcom said it was unlawful to "create or share private pictures or illegal content" and confirmed this included sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable steps" to "reduce the danger" of users coming across illegal content on their services and to delete it promptly when they become aware of it.
