Safety Cases: The Missing Methodology in Frontier AI Governance

Two different research groups have both graded frontier AI safety frameworks, and it is easy to conflate them. SaferAI, a Paris-based nonprofit, publishes a Frontier Risk Management Tracker that scores labs’ overall risk-management practices across four dimensions — identification, analysis, treatment, governance — and CASRAI already covers that rubric in a separate guide. This guide is about something narrower and, so far, uncovered on this site: the safety case as its own methodology — a specific argumentative structure, borrowed from safety-critical engineering, for justifying a single deployment decision — and the body of work the University of Oxford’s Centre for the Governance of AI (GovAI) has built around it, plus the 2023 paper that anticipated much of the regulatory language now showing up in California SB 53 and the EU AI Act.

Do not confuse the two. SaferAI’s rubric grades a company’s overall risk-management program. A safety case is the document a company would write to justify one specific claim — “this model is safe enough to deploy” — and it can exist whether or not that company has ever heard of SaferAI’s Tracker.

What a safety case actually is

A safety case is not a new invention for AI. Aviation certification, nuclear plant licensing, and medical device approval have used the same basic structure for decades: a structured, evidence-based argument that a specific system is acceptably safe for a specific use, in a specific context — not a checklist, not a certification stamp, and not a bare assertion of confidence. GovAI’s own definition, from its October 2024 paper on the topic, is precise about this: a safety case is “a report that makes a structured argument, supported by evidence, that a system is safe enough in a given operational context.”

The methodology most of GovAI’s work builds on is Claims, Arguments, and Evidence (CAE): state the top-level safety claim, break it into progressively narrower sub-claims, and support each sub-claim with concrete evidence — evaluation results, red-team findings, technical mitigations — until the argument bottoms out in something checkable. The output is a document a developer, an internal safety team, or eventually a regulator can actually interrogate, rather than a paragraph of reassurance.

GovAI’s safety-case body of work

GovAI has published three pieces of work on this specific methodology within a few months of each other in late 2024, and each does a different job:

• “A Grading Rubric for AI Safety Frameworks” (Jide Alaga, Jonas Schuett, Markus Anderljung, September 2024) — a scoring instrument with seven evaluation criteria and 21 indicators for judging the published safety frameworks companies like Anthropic, OpenAI, and Google DeepMind already release, graded A through F, meant to be applied via surveys, Delphi studies, or audits.
• “Safety Cases for Frontier AI” (Marie Davidsen Buhl, Gaurav Sett, Leonie Koessler, Jonas Schuett, Markus Anderljung, October 2024) — the methodology paper itself: what a safety case borrowed from safety-critical engineering would look like applied to a frontier model, how it could function inside both company self-governance and government regulation, and what practical groundwork (risk modeling, evaluation infrastructure, independent review capacity) still has to exist before safety cases can actually carry regulatory weight.
• “Safety Case Template for Frontier AI: A Cyber Inability Argument” (Arthur Goemans, Marie Davidsen Buhl, Jonas Schuett, Tomek Korbak, Jessica Wang, Benjamin Hilton, Geoffrey Irving, November 2024) — the methodology made concrete: a worked template arguing a model is incapable of catastrophic cyberattacks, using the CAE structure to chain a top-level claim down through risk models, proxy tasks, and capability-evaluation evidence into a single reviewable document.

Read together, the three form a stack: the grading rubric evaluates what labs publish today, the methodology paper proposes the more rigorous alternative, and the cyber-inability template shows that alternative actually built out for one specific risk domain.

How GovAI’s rubric differs from SaferAI’s

Both organizations score frontier AI safety practices, and both have graded the same handful of labs, which is exactly why the two get conflated. The actual scope is different:

• SaferAI’s Tracker grades a company’s overall risk-management program — whether it identifies risks, sets tolerances, treats them, and governs the whole process — as a standing, periodically updated scorecard across named companies.
• GovAI’s grading rubric scores a company’s published safety framework document specifically — the RSP, Preparedness Framework, or FSF itself — against seven criteria and 21 indicators, as a one-off academic instrument rather than a maintained public tracker.
• GovAI’s safety-case methodology is not a grading instrument at all. It does not score anything that exists today; it specifies what a rigorous safety argument for a single deployment decision should contain, as groundwork for a practice that GovAI’s own October 2024 paper says does not yet exist in a fully workable form at any lab.

In short: SaferAI asks “how good is this company’s overall safety program?” GovAI’s rubric asks “how good is this company’s published framework document?” GovAI’s safety-case work asks a third, different question entirely — “if you had to prove this one deployment decision was safe, what would that proof actually need to contain?” — and that third question is what this guide is about.

The 2023 paper behind the current regulatory language

GovAI’s safety-case work builds on an earlier paper from the same research community: “Frontier AI Regulation: Managing Emerging Risks to Public Safety” (Markus Anderljung, Joslyn Barnhart, Anton Korinek, Jade Leung, Cullen O’Keefe, Jess Whittlestone, and 17 co-authors; submitted July 2023, final version November 2023). It is not currently covered as its own guide anywhere on CASRAI, and it is worth understanding on its own terms — not because CASRAI can verify a direct citation trail into any specific bill, but because its proposed structure and the requirements two major 2025 laws actually adopted line up closely enough to be worth naming explicitly.

The paper proposed three “regulatory building blocks” for frontier AI models, two years before either law existed:

• Standard-setting processes — a mechanism to define what frontier developers are actually required to do.
• Registration and reporting — giving regulators visibility into frontier development activity that they would not otherwise have.
• Compliance mechanisms — enforcement powers for a supervisory authority, up to and including licensing regimes.

It also recommended specific safety practices: pre-deployment risk assessment, external scrutiny of model behavior, using that risk assessment to actually inform deployment decisions, and ongoing post-deployment monitoring.

Compare that to what California SB 53 (effective January 1, 2026) actually requires of large frontier developers: a published frontier AI framework describing catastrophic-risk identification and management (standard-setting, self-administered), transparency reports disclosing the catastrophic-risk assessments run on each model before deployment (pre-deployment risk assessment plus reporting), and mandatory critical-safety-incident reporting to California’s Office of Emergency Services within fixed deadlines (post-deployment monitoring, formalized). Or the EU AI Act’s Article 55 obligations on systemic-risk general-purpose AI models: risk management across the model lifecycle, documented mitigation and safety practices, and serious-incident reporting to the AI Office — the same registration-and-reporting-plus-standard-setting shape, run through a different institution. Neither law is a literal implementation of the 2023 paper’s proposal, and CASRAI has not found a primary source establishing that either bill’s drafters cited it directly. What is verifiable is the structural resemblance itself: three building blocks proposed in mid-2023, largely absent from binding law at the time, all three now present in some form in the two most significant frontier-AI statutes to pass since.

NIKOLAI angle: N4’s safety-case element, honestly framed

NIKOLAI is CASRAI’s own independent, unendorsed reference dictionary for frontier-AI-safety terminology — not a standard, and not something any lab, evaluator, or regulator has endorsed. Its N4 track, “Claims and Argument,” already contains an element for exactly this concept: Safety Case (Assurance Argument), defined there as “a structured, reviewable argument composed of one or more Claim records and supporting evidence demonstrating that a model’s risks are acceptably low in a stated deployment or development context” — the same CAE-style structure GovAI’s papers describe.

That element currently carries five crosswalk rows (Anthropic, OpenAI, Google DeepMind, Meta, and the UK AI Security Institute, plus one for the Frontier Model Forum), and every one of them is a shadow mapping: CASRAI’s own reading of each organization’s published document, not a mapping any of those organizations has reviewed, confirmed, or endorsed. The element’s own divergence note already flags that usage is inconsistent across labs — DeepMind defines “safety case” as a glossary term, OpenAI and Meta use adjacent undefined language like “safeguards case,” and Anthropic’s own Risk Report makes structurally equivalent arguments without ever using the label. None of that has been formalized into a declared row yet; no organization has used NIKOLAI’s Mapping Declaration process to confirm how its own safety-case practice actually maps to this element.

GovAI’s rubric and its three safety-case papers are exactly the kind of material that could sharpen those rows in the future — a rigorous, independent methodology for what a safety-case argument should contain is a natural yardstick against which to re-read each lab’s actual document. This guide treats them as candidate future crosswalk material for N4, not as an existing mapped row: nothing here should be read as claiming GovAI, or any lab named above, has endorsed or been consulted on NIKOLAI’s current safety-case element.

Frequently asked questions

Is a “safety case” the same thing as an AI safety framework like an RSP?

No. A framework (Responsible Scaling Policy, Preparedness Framework, Frontier Safety Framework) is a standing policy document describing how a company will test and respond to risk in general. A safety case is a specific, one-off argument justifying one deployment decision, built using evidence the framework’s process generated. A framework is the policy; a safety case is the proof that the policy was actually satisfied for this model, this time.

Does SaferAI’s Tracker use GovAI’s safety-case methodology?

No, and CASRAI has found no evidence the two are formally linked. SaferAI’s Tracker grades overall risk-management programs against SaferAI’s own four-dimension rubric. GovAI’s safety-case work is a separate methodology for structuring a single deployment argument, developed independently and not incorporated into SaferAI’s scoring criteria as far as this guide could verify.

Has any frontier lab actually published a formal safety case?

Not in the full CAE-structured sense GovAI’s papers describe, as of this guide’s publication. Several labs (Anthropic’s Risk Reports, DeepMind’s safety case reviews under its Frontier Safety Framework) publish structurally similar evidence-backed arguments without using GovAI’s specific template or terminology — part of why NIKOLAI’s N4 safety-case element currently documents inconsistent usage rather than a settled practice.

Related CASRAI coverage

See also: how SaferAI grades frontier AI safety frameworks, Responsible Scaling Policy: what it is and how the major labs compare, California SB 53, the EU AI Act’s GPAI Code of Practice, capability thresholds across labs and regulators, and evaluator independence in the AI safety ecosystem.