Model Safety

Testing frontier models for dangerous capabilities before release.

The UK AI Security Institute has evaluated more than 30 frontier models since November 2023 and found universal jailbreaks in every one. China's regulator had cleared 988 generative AI services through its security filing by June 30, 2026.

US CAISI evaluation, DeepSeek against US models: DeepSeek R1, agent hijacked 37%; US models, agent hijacked 4%; DeepSeek, harmful output under jailbreaks 95-100%; US models, same 5-12%. Sep 2025 US CAISI EVALUATION, DEEPSEEK AGAINST US MODELS DeepSeek R1, agent hijacked 37% US models, agent hijacked 4% DeepSeek, harmful output under jailbreaks 95-100% US models, same 5-12% PERCENT OF TESTS SEP 2025
Models UK AISI has evaluated
30+
Apprentice cyber task success, 2023 to 2025
9% to 50%
China generative AI services filed
988
Registry incidents
3

Evidence

Three US labs flagged their own models at new risk levels in mid-2025

Documented cases: model Safety
DateCaseCountryWhat happenedSource
May 22, 2025Anthropic activates ASL-3United StatesFirst deployment under ASL-3 safeguards, for CBRN riskSource
Jul 17, 2025ChatGPT Agent rated High for bioUnited StatesFirst OpenAI product given that ratingSource
Aug 1, 2025Gemini 2.5 Deep ThinkUnited StatesReached the cyber early-warning thresholdSource
Sep 30, 2025CAISI evaluation of DeepSeekUnited States and China19 benchmarks, three DeepSeek models against four US modelsSource
Jun 30, 2026CAC filing totalsChina988 services filed, 598 apps registeredSource

Developers

Only four of 13 developers disclose pre-release testing by an outside evaluator

Frontier developers
DeveloperSafety frameworkThird-party testingFLI grade, Summer 2026
OpenAI United StatesYes Preparedness Framework v2, Apr 2025Yes UK AISI pre-deployment accessC
Anthropic United StatesYes Responsible Scaling Policy v3.4, Jul 2026Yes UK AISI cyber evaluationC+
Google DeepMind United StatesYes Frontier Safety Framework v3.1, Apr 2026Yes UK AISI, Apollo Research and othersC
Meta United StatesYes Superintelligence Scaling Framework v2.1, Oct 2026Yes Apollo ResearchD+
xAI United StatesYes Risk Management Framework, Aug 2025UnknownF
Microsoft United StatesYes Frontier Governance Framework, Feb 2025UnknownNot graded
Amazon United StatesYes Frontier Model Safety Framework, Feb 2025Partial independent evaluations publishedNot graded
Mistral FranceNoUnknownF
DeepSeek ChinaNoNo CAISI evaluated after releaseF
Alibaba (Qwen) ChinaNoUnknownD-
Zhipu AI (Z.ai) ChinaNoUnknownD-
Moonshot AI ChinaNoNo UK AISI assessed after releaseNot graded
ByteDance (Seed) ChinaNoUnknownNot graded

Tag colors: green yes, amber partial, red no, gray not found.

Registry

Three registry incidents bear on model safety

Registry incidents where this discipline was tested in practice. Each links to its full record.

Registry incidents linked to this page
IncidentDateDeveloperRegulator action
Claude Opus 4 Blackmails an Engineer in a TestMay 2025AnthropicNone.
Chinese State Group Runs Espionage Through Claude CodeSep 2025AnthropicOn November 26, 2025, the House Homeland Security Committee asked Anthropic CEO Dario Amodei to testify.
OpenAI Test Models Breach Hugging FaceJul 2026OpenAISenator Josh Hawley wrote to OpenAI on September 9, 2026. The White House Accord on Super Intelligence, signed September 29, 2026, commits signers to controls that keep models from hacking systems in unintended ways. No enforcement action was found.

Rules

Lab frameworks set the thresholds; China requires filing before launch

Rules and frameworks: model safety
RuleJurisdictionWhat it requiresStatus
Anthropic Responsible Scaling PolicyCompanyCapability thresholds trigger stronger safeguardsv3.4, Jul 2026
OpenAI Preparedness Framework v2CompanyHigh bio or chem rating requires safeguardsApr 2025
Google Frontier Safety FrameworkCompanyCritical capability levels with early warningsv3.1, Apr 2026
Interim Measures for Generative AI ServicesChinaSecurity assessment and filing before launchSince Aug 2023

Status colors: green in force, blue voluntary standard or framework, amber adopted and phasing in, gray draft or pending, red withdrawn.

Record

Page Record

Datasets
ai_trust_evidence.csv (5 rows); registry incidents (3 linked); developer scorecard (13 developers, static table)
Key sources
Anthropic activates ASL-3
ChatGPT Agent rated High for bio
Gemini 2.5 Deep Think
CAISI evaluation of DeepSeek
CAC filing totals
Last verified
October 2026