The surge in AI adoption over the past five years has heightened concerns among governments and market observers regarding the security risks associated with these evolving systems. Recent evaluations conducted by the UK’s AI Security Institute (AISI) indicate that even the most sophisticated AI models may be susceptible to misuse, prompting a reevaluation of assumptions about vendor trust and model safety.
Established by the UK government in 2024, AISI (formerly the AI Safety Institute) aims to scrutinize the capabilities of frontier AI models along with the risks they pose. The organization has tested numerous models, focusing on their performance in technical tasks such as biological research and software development while assessing their potential for misuse. So far, AISI has published performance evaluations on two notable models: OpenAI o1 and Claude 3.5 Sonnet.
AISI’s evaluation finds that OpenAI’s first reasoning model, o1, performs comparably to the firm’s internal reference model, GPT-4o. Nonetheless, AISI noted similar cybersecurity vulnerabilities in both models, with o1 exhibiting various reliability and tooling issues. While o1 generally underperformed in reasoning and coding tasks compared to GPT-4o, the two were nearly equal in areas like biological research.
Conversely, Claude 3.5 Sonnet excelled in biological research and outperformed other models in engineering and reasoning tasks. However, AISI pointed out that the model’s guardrails are not as robust, identifying multiple avenues for ‘jailbreaking’ the system to elicit harmful responses.
Although AISI has published detailed evaluations of only two models, the organization has examined a total of 22 anonymized models, amassing about 1.8 million attempts to bypass safeguards and conduct illicit tasks. Alarmingly, every model tested exhibited vulnerabilities to jailbreaks, leading AISI to identify over 62,000 harmful behaviors.
These findings have significant implications for firms in regulated sectors such as finance, healthcare, legal services, and the public sector. AISI’s results underscore the importance of governance and security in AI deployment, compelling organizations to take a proactive approach rather than relying solely on ‘trusted vendors.’ Businesses must conduct thorough capability assessments, stress tests, and red-teaming exercises to ensure their AI systems are secure.
Prior to the AISI tests, some regulatory bodies, including the Financial Conduct Authority and the NHS, issued guidance on AI deployment tailored to their industries. However, these guidelines are expected to be updated in light of AISI’s findings. Companies across various sectors should heed these insights when formulating an AI strategy, selecting vendors, or integrating technology into their operations, particularly as the market for enterprise scams has expanded and scammers are increasingly adept at exploiting AI frameworks.
Unlike the EU, which enacted the EU AI Act in 2024, the UK currently lacks a unified framework to govern AI usage. Although AISI’s findings are backed by the government, the accompanying guidance is nonbinding. Furthermore, the evaluation methods employed by AISI are not standardized; disparate assessment criteria exist among regulators and safety institutes worldwide. This inconsistency has led some stakeholders to argue that the tests cannot definitively categorize any AI model, or the industry as a whole, as safe or unsafe.
Despite submitting their models for AISI’s tests, OpenAI and Anthropic have raised concerns regarding the lack of standardization between the UK’s AI institute and its U.S. counterpart, the Center for AI Standards and Innovation. As pressure grows on governments to align their evaluation frameworks, firms looking to adopt AI must remain vigilant. The reality is that safety is not guaranteed, even when sourcing from the most reputable providers in the industry.
See also
Effective AI Governance Demands Clear Communication to Build Trust and Accountability
Trump’s Executive Order Targets State AI Regulations, Aims for National Framework
Korea’s AI Basic Act Launches in 2026 Amid Industry Concerns Over Governance Readiness
Residents Urge Holyoke Council on Bridge Safety and Public Comment Reforms
