JAKARTA – In a landmark ruling, a Dutch court has ordered Elon Musk’s artificial intelligence company, xAI, to cease the creation and distribution of images that “strip off the clothes” of individuals, including both adults and children, without explicit consent. This interim decision from an Amsterdam court stands as one of Europe’s first legal actions targeting the generative capabilities of AI in producing non-consensual sexual content, a matter that has gained urgency with the rise of “nudifier” technology.
The court determined that Grok, xAI’s chatbot, must refrain from generating or disseminating any images depicting individuals in nude or semi-nude states without their permission. Should xAI disregard this ruling, the company faces potential fines of up to 100,000 euros per day—a daunting prospect for any corporate financial officer.
Moreover, the court mandated that the Grok service must be removed from the X platform while it remains in violation of these legal standards. The case was brought forth by the non-profit organization Offlimits in collaboration with the Victims Support Fund, which has been actively involved in addressing incidents of digital-based sexual violence. During the proceedings, plaintiffs demonstrated that Grok could still manipulate individuals’ images into sexual content without consent, despite xAI’s assertions of improved security measures.
Robbert Hoving, director of Offlimits, emphasized the accountability of companies in preventing misuse of their technology. “The burden is on companies to ensure their technology is not misused,” he stated. In contrast, xAI contended that it cannot entirely eliminate abuse by malicious users and pointed out that it had restricted access to image generation features for paid subscribers only. However, the judge found this measure inadequate.
This ruling comes as the European Union intensifies its oversight of AI technology through the Digital Services Act. The European Commission has previously launched an investigation into the X platform concerning the potential risk of illegal content generated by AI, including the manipulation of explicit images. The European Parliament has also intensified discussions surrounding the need for a ban on AI applications capable of creating fake sexual imagery, indicating a shift from an era of “AI free experimentation” to one of “AI closely monitored.”
The implications of this ruling extend beyond the borders of the Netherlands, potentially setting a significant precedent for the global AI industry. If such legal trends continue, technology firms may no longer be able to merely position themselves as toolmakers. Instead, the law is beginning to recognize them as parties sharing responsibility for the societal impacts of their innovations.
See also
Meta Partners with Entergy for $2 Billion Energy Infrastructure to Power AI Data Center
Germany”s National Team Prepares for World Cup Qualifiers with Disco Atmosphere
95% of AI Projects Fail in Companies According to MIT
AI in Food & Beverages Market to Surge from $11.08B to $263.80B by 2032
Satya Nadella Supports OpenAI’s $100B Revenue Goal, Highlights AI Funding Needs
