How Do You Know an AI Model Is Safe? The Push for Independent Testing Standards

What happened?

A growing number of governments and independent research institutes have moved to formalise how artificial intelligence systems are tested before and after they are deployed in high-stakes settings such as healthcare, finance, hiring and public services. Rather than relying solely on developers' internal claims about model safety and performance, several countries have established dedicated AI safety institutes tasked with independently evaluating systems, sharing findings and coordinating international testing approaches.

This move toward independent evaluation reflects growing concern that self-assessment by AI developers is insufficient given the pace at which models are being deployed into consequential decisions. In 2026, several of these institutes have published shared frameworks for evaluating risks such as inaccurate outputs, security vulnerabilities and potential for misuse, marking an early but significant step toward internationally recognised testing standards for AI systems.

Key points

• Multiple governments have established dedicated AI safety institutes to independently evaluate AI systems before and after deployment.
• Shared evaluation frameworks are emerging to assess risks including inaccurate outputs, security vulnerabilities and potential misuse.
• Self-assessment by AI developers alone is increasingly seen as insufficient for systems used in high-stakes decisions.
• International coordination on testing standards remains limited, creating a risk of fragmented and inconsistent requirements across countries.
• Independent evaluation is particularly focused on sectors such as healthcare, finance and public services where errors carry serious consequences.

What we know

AI safety institutes established by several governments are tasked with conducting technical evaluations of advanced AI models, often working directly with developers to test systems before public release, as well as monitoring systems already in use for emerging risks. These institutes typically focus on evaluating whether models produce harmful, biased or factually incorrect outputs under various conditions, and whether they can be manipulated into bypassing safety guardrails through adversarial prompting techniques.

Standards organisations have also begun publishing technical frameworks intended to give organisations structured methods for assessing AI system risk, covering areas such as data quality, model robustness, transparency and ongoing monitoring after deployment. These frameworks are generally voluntary but are increasingly being referenced in procurement requirements by governments and large enterprises, giving them practical weight even without binding legal force in most jurisdictions.

Officials and experts

Officials at national AI safety institutes have described their mission as building the technical capacity to evaluate AI systems independently of the companies that build them, arguing that public trust in AI depends on having credible, arms-length assessment rather than relying solely on vendor assurances. They have also emphasised the importance of sharing evaluation methods and findings internationally, given that AI models developed in one country are typically deployed globally within weeks of release.

Standards bodies involved in developing AI risk management frameworks have stressed that testing cannot be a one-time event, since AI models can behave differently as they are fine-tuned, updated or deployed in new contexts, requiring ongoing monitoring rather than a single pre-deployment check. Researchers in the field have also cautioned that evaluation science for AI remains young, noting that even well-resourced testing regimes struggle to anticipate every way a system might fail or be misused once deployed at scale.

Background

Concerns about AI safety and reliability moved from academic discussion to mainstream policy attention as increasingly capable generative AI systems were deployed rapidly across consumer and enterprise applications, often faster than regulatory and testing frameworks could adapt. High-profile incidents involving AI systems producing inaccurate, biased or harmful outputs in real-world use heightened public and government attention to the need for more rigorous pre-deployment evaluation.

In response, several governments convened international discussions on AI safety, leading to commitments to establish national testing institutes and to coordinate more closely on shared evaluation approaches. These efforts have built on decades of experience in other high-risk technology sectors, such as pharmaceuticals and aviation, where independent testing and certification became standard practice only after early periods of self-regulation proved insufficient to manage emerging risks at scale.

Detailed analysis

The push for independent AI evaluation reflects a broader recognition that the incentives facing AI developers do not always align neatly with the public interest in thorough safety testing. Companies face competitive pressure to release new models quickly, and thorough safety evaluation takes time and resources that can appear to work against speed to market. Independent testing bodies are intended to provide a check on this dynamic, offering assessments that are not shaped by commercial incentives to minimise identified risks or accelerate release timelines.

However, building credible independent evaluation capacity is proving technically demanding. Modern AI models are complex systems whose behaviour can vary significantly depending on how they are prompted, fine-tuned or integrated into larger applications, making it difficult to certify a model as broadly 'safe' in the way a physical product might be certified against a fixed specification. Evaluators are increasingly focusing on testing specific, well-defined risks, such as a model's tendency to generate harmful content in response to adversarial prompts, rather than attempting a single comprehensive safety verdict.

International coordination remains one of the most significant open challenges. AI models are typically developed by a small number of companies but deployed globally within days of release, creating pressure for internationally consistent testing standards to avoid a patchwork of conflicting national requirements that could slow beneficial deployment while still failing to catch genuine risks. Early efforts at coordination, including shared technical frameworks and joint testing exercises between allied countries' safety institutes, represent initial steps, but substantial gaps remain, particularly regarding how findings are shared with the public and how enforcement might work in practice.

There is also an ongoing debate about the appropriate scope of independent testing. Some argue that evaluation should focus primarily on the most powerful, general-purpose AI models given their broad potential for impact, while others contend that testing should be more heavily weighted toward specific high-stakes applications, such as AI used in medical diagnosis or credit decisions, regardless of how technically advanced the underlying model is. This distinction matters practically, since resources for independent testing remain limited relative to the pace of AI deployment across every sector of the economy.

Smaller companies and developers building applications on top of major AI models, rather than building models from scratch, present a further complication for evaluation regimes designed primarily with large model developers in mind. Much of the real-world risk from AI arises not from the underlying model itself but from how it is applied in a specific product, a distinction that testing frameworks are still working out how to address comprehensively without creating impractical burdens for smaller developers.

Why it matters

As AI systems are increasingly used to inform decisions in healthcare, finance, hiring and public administration, the reliability of these systems has direct consequences for people's lives, from medical diagnoses to loan approvals and job opportunities. Independent testing regimes are intended to build justified public confidence in these systems, distinguishing genuinely well-tested tools from those making unverified safety claims.

For the AI industry, credible independent evaluation could ultimately support broader adoption by giving businesses and consumers more confidence in deploying AI for consequential tasks, provided the testing regimes prove genuinely rigorous rather than becoming a compliance formality. For policymakers, getting the balance right between thorough safety evaluation and unnecessary friction that slows beneficial innovation remains a central and unresolved challenge.

What happens next?

Expect continued expansion of national AI safety institutes' technical capacity and further development of shared international evaluation frameworks, though full harmonisation across major jurisdictions remains a distant prospect given differing regulatory philosophies. Sector-specific testing requirements, particularly in healthcare and financial services, are likely to mature faster than general-purpose evaluation standards, given the more clearly defined risks in those domains.

Over the coming years, the credibility of independent AI testing will depend heavily on whether institutes can demonstrate real technical capability to catch meaningful risks, rather than simply formalising processes that developers were already conducting internally, a distinction that will shape public trust in AI systems for years to come.