Likelihood Term: No Shared Probability Scale

When a frontier AI lab’s safety framework says a risk is “plausible” or “unlikely,” it is very rarely saying what probability that word stands for. Anthropic’s Responsible Scaling Policy, Google DeepMind’s Frontier Safety Framework, and the EU AI Act’s GPAI Code of Practice all use likelihood language to decide whether a model gets released, delayed, or shipped with extra safeguards — and all three let that language stay qualitative, undefined, and non-comparable across labs. NIKOLAI, CASRAI’s own frontier-AI-safety dictionary, catalogues this as a single element — Likelihood Term, in its N4 track — and its own page states the finding plainly: “No source in the mapped corpus anchors its likelihood terminology to numerical probabilities.”

The gap: none of the three frameworks examined here defines what percentage probability its own likelihood words mean.
Anthropic’s RSP v3.4 (effective July 8, 2026) uses “plausible” as an actual policy trigger — a safety case is allowed to be simpler “if it is first required when the risk is merely plausible” — without ever defining plausible as a probability range.
Google DeepMind’s Frontier Safety Framework requires safety cases to weigh “the likelihood and consequences” of misuse and to show “the likelihood of deceptive alignment risk would be low enough,” without a scale for either.
The EU AI Act’s GPAI Code of Practice is the one framework examined that names the format explicitly — its own text gives “probability: unlikely” x “impact: high” as a valid example of a qualitative systemic-risk matrix, alongside quantitative and semi-quantitative alternatives, with no requirement that “unlikely” be pinned to a number.
The IPCC has already solved this, for a different domain: its own calibrated-uncertainty language defines “likely” as an assessed probability of 66% to 100%, as one fixed rung on a shared, cross-report scale climate scientists all use the same way.
NIKOLAI’s own proposed fix: a controlled ladder of likelihood terms paired with a mandatory likelihood.scheme field naming whose scale is in use — plus an explicit “unanchored” value for exactly the qualitative-language-with-no-defined-range case this page documents.

Anthropic’s RSP: “plausible” as a trigger, not a probability

Anthropic’s Responsible Scaling Policy, version 3.4 (effective July 8, 2026), leans on likelihood language throughout its industry-wide recommendations without ever converting that language into a number. Describing what a safety case needs to show for models with “moderate capacity for autonomous, goal-directed operation and subterfuge,” the RSP’s own text says: “This case may initially be relatively simple and rely heavily on capability limitations, if it is first required when the risk is merely plausible.” Here, “plausible” is not colour commentary — it is the specific condition that sets how rigorous a safety case has to be. A lower evidentiary bar applies precisely because the risk is judged “merely plausible” rather than something stronger. Nowhere in the document is “plausible” assigned a probability range, nor is it distinguished numerically from the document’s other likelihood-adjacent words: “likely enough,” used earlier in the RSP’s Introduction to describe the odds that a competitor will race ahead without matching safety investment (“Although this situation has not yet arisen, it looks likely enough that we want to prepare for it”), or “significantly exceed,” “substantially higher,” and “moderate,” each doing similar unquantified work at different points in the capability-threshold table.

The RSP’s threat-specific risk assessment section asks Anthropic to analyze “remaining absolute risk” for each threat model and state whether a threshold has been crossed — but the assessment itself is argument-based (“a strong argument that catastrophic risk is contained”), not probability-based. The document is explicit about why: at this stage of the field, Anthropic writes, “we cannot presently give highly specific advance detail on what evaluations will determine whether risk thresholds have been passed.” That is a defensible position for a document that says outright it favors flexible argument over rigid pre-specification. It is also, by construction, a likelihood vocabulary that cannot be compared word-for-word against another lab’s.

Google DeepMind’s FSF: “likelihood” invoked, never defined

Google DeepMind’s Frontier Safety Framework (the published Version 2.0 document, dated 4 February 2025; DeepMind has since revised the framework further, most recently to v3.1 per CASRAI’s own RSP vs. Preparedness Framework vs. FSF comparison) uses “likelihood” as a load-bearing word in exactly the same undefined way. Its deployment-mitigation process requires developers to assess “the robustness of these mitigations against the risk posed through assurance evaluations and threat modeling research,” where “the assessment takes the form of a safety case, taking into account factors such as the likelihood and consequences of misuse.” For deceptive-alignment risk specifically, the framework sets out two safety cases a developer can build a deployment decision on — including one that requires showing “the likelihood of deceptive alignment risk would be low enough for safe deployment, even if the model were trying to meaningfully undermine human control.” In both passages, “likelihood” is the operative word deciding whether a model ships. In neither passage does the Framework attach a number, a band, or even an ordinal scale to it. The document’s Critical Capability Levels are defined in capability terms (what a model can do), and its security-level recommendations borrow an external RAND scale — but likelihood itself is left to the developer’s own judgment, undocumented and unstandardized, exactly the pattern NIKOLAI’s Likelihood Term element was built to name.

The EU’s GPAI Code of Practice: the one framework that names the format

Of the sources checked for this page, the EU AI Act’s GPAI Code of Practice (Safety and Security chapter) is the only one that states outright what a qualitative probability term looks like in practice — while still leaving that term undefined. Its Measure 3.4, Systemic risk estimation, requires Signatories to estimate “the probability and severity of harm for the systemic risk,” and specifies the estimate “will be expressed as a risk score, risk matrix, probability distribution, or in other adequate formats, and may be quantitative, semi-quantitative, and/or qualitative.” The Code then gives its own worked examples: “(1) a qualitative systemic risk score (e.g. ‘moderate’ or ‘critical’); (2) a qualitative systemic risk matrix (e.g. ‘probability: unlikely’ x ‘impact: high’); and/or (3) a quantitative systemic risk matrix (e.g. ‘X-Y%’ x ‘X-Y EUR damage’).”

That second example — “probability: unlikely” — is the Code explicitly sanctioning an unanchored qualitative probability term as compliant, in the same breath as sanctioning a fully numeric one. Nothing in Measure 3.4 requires a Signatory using the qualitative option to say what percentage range “unlikely” covers, or to use the same range another Signatory would use for the same word. The Code’s incident-reporting provisions (Measure 9.3) repeat the pattern in a different context: a serious-incident report is required when a Signatory “suspect[s] with reasonable likelihood” a causal link between its model and a harm — a phrase that sets a real, binding reporting deadline (as fast as two days for critical-infrastructure disruption) off a threshold (“reasonable likelihood”) the Code never quantifies either.

What a calibrated scale actually looks like: the IPCC comparison

The gap these three frameworks share is not a hard problem to solve — it has already been solved, in a different field, for decades. The Intergovernmental Panel on Climate Change assigns its own likelihood words fixed, published probability ranges, used consistently across its assessment reports specifically so that “likely” means the same thing in one IPCC chapter as it does in another. Per the IPCC’s own calibrated-language framework, “likely” is defined as an assessed probability of 66% to 100% — one rung on a seven-step ladder running from “exceptionally unlikely” up to “virtually certain,” each rung tied to a specific range rather than left to the individual author’s judgment.

The contrast with frontier-AI safety frameworks is direct. An IPCC author cannot write “likely” to mean whatever they personally intend that day — the term is fixed by the report’s own guidance note before a single sentence is drafted, and a reader can look up exactly what confidence that word carries. None of the three AI-safety documents examined on this page do that. Anthropic’s “plausible,” DeepMind’s “likelihood,” and the GPAI Code’s “unlikely” are each left to the document’s own author (or, downstream, to whichever lab is applying the framework) to interpret case by case — which means the same word can carry a different real-world probability every time it’s used, even within a single lab’s own published framework.

Why this keeps coming up in the research literature

The gap is not just a CASRAI observation. A September 2026 workshop paper, Open Problems in AI Risk Modeling: Insights from a Workshop on the Technical Foundations of AI Risk Modeling (Jackson, Raman, Kryś, Fillingham, Kengott, Lohn, Madkour, Papadatos, Sykes, Wisakanto, and Murray; arXiv:2609.03178, posted 2 September 2026), synthesizes exactly this problem from the technical-modeling side. The paper draws on a workshop of 22 experts working across five separate research traditions in quantitative AI-risk modeling, and lays out a structured agenda of open questions covering model design, evidence use, and validation for the kind of probability estimates a rigorous risk model would need. That a 22-expert workshop convened specifically to work through the technical foundations of quantifying AI risk is itself a data point for how unresolved the underlying probability-modeling problem still is — the same gap that shows up, in plainer form, as an undefined “plausible” or “unlikely” in the frameworks labs have already published and are using today to decide what ships.

Where NIKOLAI fits: N4, Likelihood Term

NIKOLAI is CASRAI’s own frontier-AI-safety dictionary — an independent, unendorsed reference work, not an official standard or a body any lab has agreed to be bound by. Its N4 track, Claims and Argument, is where it catalogues the vocabulary safety cases are actually built from: claim, safety case, likelihood term, risk level, confidence adjustment, and related concepts. We checked the live Likelihood Term element page directly before writing this page. As of this writing it carries a “Proposed (nikolai-v0.1)” status, and its own definition proposes exactly the fix this page’s comparison points toward: “a controlled ladder of ordinal terms (or numeric bands where sources permit) paired with a required likelihood.scheme reference field that identifies which organization’s scale is in use,” including “an explicit ‘unanchored’ value for cases where sources use qualitative language without defining corresponding probability ranges.” The element page’s own stated rationale cites the same structural gap this page independently verified against Anthropic’s RSP, DeepMind’s FSF, and the EU’s GPAI Code: “No source in the mapped corpus anchors its likelihood terminology to numerical probabilities — a field-wide absence that strengthens NIKOLAI’s case for mandatory scheme documentation.” That’s a genuine research finding stated on NIKOLAI’s own page, not a crosswalk table asserting how any specific lab’s document maps onto the element — no such per-organization crosswalk exists yet on the live Likelihood Term page, and this page does not claim otherwise.

Frequently asked questions

Do any frontier AI labs define what “likely” or “unlikely” means numerically in their safety frameworks?

Not among the three primary sources checked for this page. Anthropic’s RSP v3.4, Google DeepMind’s Frontier Safety Framework, and the EU AI Act’s GPAI Code of Practice all use likelihood language as an operative part of their risk decisions, and none of the three attaches a probability range to the terms it uses.

Is the EU’s GPAI Code of Practice more rigorous about probability than the lab frameworks?

It’s more explicit about the format, not more rigorous about the number. Its Measure 3.4 is the only one of the three sources that spells out, in its own worked examples, that a qualitative term like “unlikely” is an acceptable way to express systemic-risk probability — alongside quantitative and semi-quantitative alternatives — but it does not require “unlikely” to be pinned to a percentage range.

How does the IPCC’s likelihood scale actually work?

The IPCC assigns fixed, published probability ranges to a set of standard likelihood terms and applies them consistently across its assessment reports. Per its own calibrated-language framework, “likely” specifically means an assessed probability of 66% to 100%.

What is NIKOLAI’s Likelihood Term element proposing?

A controlled ladder of likelihood terms (or numeric bands, where the source material permits them) paired with a mandatory field naming which organization’s scale is being used, plus an explicit “unanchored” value for the common case where a source uses qualitative language without ever defining a probability range for it.

Is NIKOLAI an official or endorsed mapping of these frameworks?

No. NIKOLAI is CASRAI’s own independent, unendorsed reference dictionary. No lab, evaluator, or regulator covered on this page has reviewed, endorsed, or been consulted on NIKOLAI’s Likelihood Term element or any other part of the dictionary.

Sources

Anthropic, Responsible Scaling Policy, Version 3.4, effective July 8, 2026 — “plausible” as a safety-case trigger; “likely enough” in the Introduction; capability-threshold table language.
Google DeepMind, Frontier Safety Framework, Version 2.0, 4 February 2025 — “the likelihood and consequences of misuse” (Deployment Mitigations) and “the likelihood of deceptive alignment risk would be low enough” (Safety Cases and Control Evaluations).
European Commission, GPAI Code of Practice, Safety and Security chapter, Measure 3.4 (Systemic risk estimation) and Measure 9.3 (Reporting timelines) — the qualitative-risk-matrix example (“probability: unlikely” x “impact: high”) and the “reasonable likelihood” incident-reporting threshold.
Jackson, K., Raman, D., Kryś, J., Fillingham, S. P., Kengott, J., Lohn, A. J., Madkour, N., Papadatos, H., Sykes, J., Wisakanto, A. K., & Murray, M. (2026). Open Problems in AI Risk Modeling: Insights from a Workshop on the Technical Foundations of AI Risk Modeling. arXiv:2609.03178, posted 2 September 2026.
IPCC calibrated-uncertainty language framework — “likely” defined as an assessed probability of 66% to 100%, per the IPCC’s own guidance note on consistent treatment of uncertainties.
CASRAI’s own NIKOLAI Likelihood Term element page (N4 track), verified live at time of writing.

Why This Matters for Research Administration

US IRB and human-subjects-review offices already live inside exactly the gap this guide documents in frontier-AI frameworks. The Common Rule’s own core risk-classification term, “minimal risk,” is defined at 45 CFR 46.102 as risk where “the probability and magnitude of harm or discomfort anticipated in the research are not greater in and of themselves than those ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests” — a qualitative, comparison-based standard with no numeric probability attached, structurally identical to the “unanchored” likelihood language NIKOLAI’s Likelihood Term element flags in Anthropic’s “plausible,” DeepMind’s “likelihood,” and the EU GPAI Code’s “unlikely.” IRB panels make the same undocumented judgment call, protocol by protocol, deciding what “ordinarily encountered in daily life” means with no IPCC-style calibrated ladder to anchor the term to a percentage range. A research-administration office training new IRB members could use this guide’s IPCC contrast directly as a ready-made illustration of why “minimal risk” calls differ across institutions reviewing comparable studies.

Likelihood Term: Three Frameworks, No Shared Probability Scale