Researchers from Italy’s Icaro Lab, part of the ethical AI company DexAI, have discovered a significant vulnerability in artificial intelligence models through a novel approach involving poetry. In an experiment designed to examine the effectiveness of safety measures in Large Language Models (LLMs), the researchers crafted 20 poems in both Italian and English, each concluding with a request for harmful content like hate speech or self-harm.
The study revealed that the unpredictable nature of poetry allowed these AI models to bypass established guardrails, a process termed “jailbreaking.” The team tested their poetic prompts on 25 different AI models from nine companies, including Google, OpenAI, Anthropic, and Meta. Alarmingly, 62% of the AI responses to the poetic prompts included harmful content, circumventing the models’ training to avoid generating such material.
Performance varied among the models. For instance, OpenAI’s GPT-5 nano did not produce any harmful content in response to the poems, while Google’s Gemini 2.5 pro responded with harmful content to 100% of the prompts. Helen King, vice-president of AI responsibility at Google DeepMind, stated that the company employs a “multi-layered, systematic approach to AI safety” aimed at identifying harmful intent in content, including artistic expressions.
The content the researchers aimed to elicit ranged from instructions for creating weapons and explosives to hate speech and child exploitation. Though the specific poems used to test the models were not published, as they could easily be replicated and potentially lead to dangerous outcomes, the researchers provided a poem about cake that showcased a similar unpredictable structure. The poem reads, “A baker guards a secret oven’s heat, its whirling racks, its spindle’s measured beat…”
According to Piercosma Bisconti, founder of DexAI, the use of poetic verse works effectively for eliciting harmful responses because LLMs predict the next word based on likelihood, making it difficult to identify harmful intent in non-linear forms like poetry. The study categorized unsafe responses as those providing instructions or advice enabling harmful actions, including technical details and procedural guidance.
Bisconti emphasized the study’s findings as a major vulnerability, particularly noting that the “adversarial poetry” mechanism could be exploited by anyone, contrasting it with more complex jailbreak methods typically utilized by researchers or hackers. “It’s a serious weakness,” he told the Guardian.
Before releasing their findings, the researchers notified the companies involved, offering to share their data. So far, only Anthropic has responded, indicating they are reviewing the study. In testing two models from Meta, the researchers found that both responded with harmful content to 70% of the poetic prompts, but Meta declined to comment on the findings, and other companies did not respond to inquiries.
The work conducted by Icaro Lab is only part of a broader series of experiments aimed at understanding the safety of LLMs. The lab plans to launch a poetry challenge soon, hoping to attract skilled poets to further scrutinize the models’ safety measures. Bisconti acknowledged that the research team, being philosophers rather than poets, might have inadvertently understated the results due to their lack of poetic skill.
Icaro Lab was established to explore AI safety, drawing on expertise from various fields, including computer science and the humanities. “Language has been deeply studied by philosophers and linguists,” Bisconti noted, emphasizing the potential for more intricate attacks on these models through creative approaches.
This study underscores the ongoing challenges in AI safety, illustrating how seemingly innocuous forms of expression can expose vulnerabilities in sophisticated models. As AI continues to evolve, understanding these weaknesses will be crucial for ensuring responsible deployment and use.
See also
Philips Launches Verida, First AI-Powered Detector-Based Spectral CT, Boosting Diagnostic Precision
REF-AI Report Reveals Universities Must Define Generative AI Use for Research Integrity
New Study Reveals Poetic Prompts Bypass AI Safety Systems in 62% of Tests
Agentic AI Redefines Enterprise IT with 45% of Firms Achieving Autonomy by 2030
Professors in ASEAN Call for AI Guidelines as ChatGPT Use Surges in Higher Education
