Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.
NIKOLAI elementN4 · Claims and argumentProposednikolai-v0.3

Safety Case (Assurance Argument)

Definition

A safety case is a structured argument, supported by evidence, that a system is acceptably safe for a specific use in a specific environment. In frontier AI it breaks a top-level safety claim into sub-claims, each backed by evaluations or other evidence, so that reviewers can test the reasoning.

Also called: Assurance argument, Safety argument, Safeguards case, Affirmative case

NIKOLAI's proposed operational definition

NIKOLAI proposes to define a Safety Case as a structured, reviewable argument, composed of one or more Claim records and their supporting evidence, that a model's risks are acceptably low in a stated deployment or development context. This is a NIKOLAI editorial proposal because the sources use the label inconsistently -- some (Google DeepMind) give it a full glossary definition and instance it against a report, others (OpenAI, Meta) use an adjacent undefined term ('Safeguards case', 'safety cases') without specifying format, author, or reviewer, and Anthropic's own Risk Report does not use the label at all for what NIKOLAI would otherwise recognise as the same argument structure. It is not endorsed by, and does not restate, any single source's own definition.

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

  • Google DeepMind's misalignment safety case "relies on two evaluation suites and assumes that AI deployments will feature oversight similar to that of human employees", which it calls "a relatively concentrated safety case" (Gemini 3.7 Flash FSF Report, p.41)
  • A cyber safety case template argues that a model does not pose unacceptable cyber risk "by breaking down the main claim into progressively specific sub-claims, each supported by evidence" (Goemans et al., Safety case template for frontier AI: A cyber inability argument, 2024, Abstract)
  • A control safety case sketch argues "that a hypothetical LLM agent deployed internally at an AI company won't exfiltrate sensitive information", using evidence from a red-team control evaluation (Korbak et al., A sketch of an AI control safety case, 2025, Abstract)

Counter-examples

Not to be confused with

  • A safety framework is not a safety case: the framework sets organisation-wide policies and thresholds, while a safety case argues that one particular system meets them (UK AISI, How can safety cases be used to help with frontier AI safety?, 2025).
  • A model card or system card is not by itself a safety case: it reports capabilities and evaluation results, but need not contain an explicit argument linking that evidence to the conclusion that risk is acceptable.

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

Safety case report
A safety case report is a document that summarises the arguments and evidence of a safety case and records progress against the safety programme, often issued at milestones. The term comes from UK defence safety practice (Def Stan 00-56).
Source: UK Ministry of Defence, SMP12: Safety Case and Safety Case Report (v2.2)
Claims-Arguments-Evidence (CAE)
Claims-Arguments-Evidence (CAE) is a notation for safety cases with three components: claims put forward for acceptance, evidence supporting them, and arguments showing how the evidence substantiates the claims. UK AISI's cyber template uses it.
Source: Goemans, Buhl, Schuett, Korbak, Wang, Hilton and Irving, Safety case template for frontier AI: A cyber inability argument (arXiv 2411.08088, 2024)
Goal Structuring Notation (GSN)
Goal Structuring Notation (GSN) is a graphical safety case notation that links goals, strategies, solutions, context, assumptions and justifications to show how a top-level goal breaks into sub-goals supported by evidence.
Source: Goemans, Buhl, Schuett, Korbak, Wang, Hilton and Irving, Safety case template for frontier AI: A cyber inability argument (arXiv 2411.08088, 2024)
Affirmative case
An affirmative case is a developer's positive argument that a high-capability model is safe enough to proceed. UK AISI cites Anthropic's use of "affirmative cases" in its sketch of ASL-4 policies as an example of frontier developers adopting safety cases.
Source: UK AI Security Institute blog: Safety cases at AISI (Irving, 23 Aug 2024)
Safety case sketch
A safety case sketch is a detailed but partial outline of the arguments and evidence a future safety case might use for one safety method. UK AISI builds sketches as building blocks because full safety cases for advanced models are not yet possible.
Source: UK AI Security Institute blog: Safety cases at AISI (Irving, 23 Aug 2024)
Safety case template
A safety case template sets out the argument structure and evidence for one part of an AI safety case, such as a cyber inability argument, so it can serve as a building block for a full safety case later.
Source: Goemans, Buhl, Schuett, Korbak, Wang, Hilton and Irving, Safety case template for frontier AI: A cyber inability argument (arXiv 2411.08088, 2024)
Holistic safety case
A holistic safety case is a comprehensive argument that addresses all potential risks across an AI system's lifecycle, as opposed to a template covering one risk. GovAI and UK AISI researchers wrote in 2024 that one did not yet seem feasible for a frontier model.
Source: Goemans, Buhl, Schuett, Korbak, Wang, Hilton and Irving, Safety case template for frontier AI: A cyber inability argument (arXiv 2411.08088, 2024)
Safety case review
A safety case review is a pre-launch analysis, run by Google DeepMind when a model reaches a relevant Critical Capability Level, that demonstrates how risks have been reduced to manageable levels; it also covers large internal deployments for advanced ML R&D levels.
Source: Google DeepMind blog: Strengthening our Frontier Safety Framework

Source of record

Where this definition comes from

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
OrganisationTheir term, as publishedMatch & verificationSource
AnthropicShadow mapping
Anthropic Risk Report, August 2026
“Claim graph with aggregation types (§2.5-2.7). The Risk Report does not use the label "safety case": CASRAI searched a full text extraction of the public report on 2026-11-13 and found no occurrence. Its §2 nonetheless has the structure of one: a top-level conclusion ("Risk is therefore low", §2.6) supported by numbered claims and sub-claims.”closeCL
confidence: medium
Anthropic Risk Report, August 2026
OpenAIShadow mapping
OpenAI Preparedness Framework v2
“"Safeguards case" appears in Table 2 (Undermining Safeguards: "require the Safeguards case to be robust to the discovered capability and/or propensity"). CASRAI found no separate definition of the term in the framework.”noneDU
confidence: medium
OpenAI Preparedness Framework v2
Google DeepMindShadow mapping
Google DeepMind Frontier Safety Framework v3.1 / Gemini 3.7 Flash FSF Report
“Safety case definition (glossary); "Residual risk assessments will be supplemented with a safety case when a model reaches a CCL" (glossary). The report says "Our safety case relies on two evaluation suites and assumes that AI deployments will feature oversight similar to that of human employees" (p.41) but does not define it there.”exactEQ
confidence: high
Google DeepMind Frontier Safety Framework v3.1
MetaShadow mapping
Meta Advanced AI Scaling Framework v2
“"Developing safety cases in advance of models exhibiting capabilities related to our enhanced evaluations." (§4.2.3); no format, author or reviewer given.”noneDU
confidence: medium
Meta Advanced AI Scaling Framework v2
UK AI Security Institute (AISI)Shadow mapping
UK AI Security Institute blog: Safety cases at AISI
“"By a safety case, we mean a structured argument that an AI system is safe within a particular training or deployment context." The post also quotes the UK Ministry of Defence Defence Standard 00-56: "A structured argument, supported by a body of evidence, that provides a compelling, comprehensible, and valid case that a system is safe for a given application in a given environment."”
Research blog post by the Institute (then the UK AI Safety Institute; renamed AI Security Institute in February 2025). It describes AISI's research programme on safety case sketches, not a regulatory requirement.
exactEQ
confidence: high
UK AI Security Institute blog: Safety cases at AISI (Irving, 23 Aug 2024)
Frontier Model ForumShadow mapping
Frontier Model Forum: Third-Party Assessments
“Safety case analysis as a form of holistic review (see C1 above).”closeCL
confidence: high
Frontier Model Forum: Third-Party Assessments
Clymer, Gabrieli, Krueger and Larsen (academic paper)Shadow mapping
Safety Cases: How to Justify the Safety of Advanced AI Systems (arXiv 2403.10462)
“"A safety case is a structured rationale that a system is unlikely to cause significant harm if it is deployed to a particular setting." (§1); in the report's own scope, the probability that the AI systems cause a catastrophe "is estimated below an acceptable threshold (for example, 0.1%)" (§3).”
Academic paper by researchers at Columbia University, Harvard University, the University of Cambridge and the Center for AI Policy; not an institutional standard.
exactEQ
confidence: high
Clymer, Gabrieli, Krueger and Larsen, Safety Cases: How to Justify the Safety of Advanced AI Systems (arXiv 2403.10462, 2024)
UK AI Security Institute (AISI)Shadow mapping
Safety case template for frontier AI: A cyber inability argument (Goemans et al., Centre for the Governance of AI and UK AISI, 2024)
“"a safety case: a structured, evidence-based argument aimed at demonstrating why the risk associated with a safety-critical system is acceptable" (Abstract); "Common notations for safety cases are Claims Arguments Evidence (CAE) and Goal Structuring Notation (GSN)" (§3.1).”
Co-authored research template; the Institute was the UK AI Safety Institute at publication.
exactEQ
confidence: high
Goemans, Buhl, Schuett, Korbak, Wang, Hilton and Irving, Safety case template for frontier AI: A cyber inability argument (arXiv 2411.08088, 2024)
What do these codes mean?
exact
The source term is equivalent to this element
close
The source term is close but not equivalent to this element
broad
The source term is broader than this element
narrow
The source term is narrower than this element
none
No mapping claim — used for false-friend and declared-but-undefined rows
EQ
Equivalent
CL
Close
BR
Source is broader than the element
NR
Source is narrower than the element
FF
False friend — same or similar label, different meaning
DU
Declared but undefined by the source
UV
Unverified

Divergence

Where sources materially disagree

"Safety case" has an uneven operational status across sources. Google DeepMind defines it in its framework glossary and refers to one in a model report; OpenAI ("Safeguards case") and Meta ("safety cases") use the term without a stated format, author or reviewer; Anthropic's Risk Report has the claim-and-evidence structure of a safety case without using the label. Outside the labs, UK AISI, its co-authors and Clymer et al. define it as a structured, evidence-backed argument, following safety-case practice in UK defence and other regulated industries. Definitions also differ on the bar: Google DeepMind's is risk "reduced to an acceptable level"; Clymer et al.'s is a catastrophe probability "below an acceptable threshold".

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →