The online platform X has sparked major anger after individuals employed its artificial intelligence assistant Grok to change pictures of females by stripping their clothing.
Multiple examples have surfaced where the technology has been used to remove clothing from female subjects to render them in bikinis without consent, as well as to put them in sexualized contexts.
Company Response and Regulatory Position
XAI, the company that created Grok, did not respond to requests for comment, except for an standard response stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to outlaw nudification tools. Under a upcoming criminal offence, anyone who provided such technology would "receive a jail term and heavy penalties".
"X or Grok has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the risk" of people accessing unlawful material on their networks, but stopped short of confirming whether it was at this time examining X or Grok in concerning artificial intelligence images.
Grok is a free AI assistant—with some premium features—which answers X users' queries when they mention it in a message.
It is often used to provide commentary or more context to other posters' remarks, but people on X are also able to alter an posted photo through its AI image editing tool.
Background of Criticism and Rules
The system has been criticised for permitting users to produce photos and clips with nudity and sexualised content, and it was in the past accused of creating graphic content of public figures.
"X has been enabling the generation and distribution of these pictures for months without taking any steps and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "depicting images of individuals in a graphic manner".
In a comment, Ofcom said it was illegal to "make or circulate non-consensual intimate images or illegal content" and stated this included explicit AI images created with AI.
It said networks such as X were required to take "suitable steps" to "lessen the risk" of users encountering illegal content on their platforms and to remove it swiftly when they learn of it.
