Threat Scenario in AI Safety: Definition

Examples

Examples from published documents

Meta's scenario TS.1.1 under outcome Cyber 1: "An AI system substantially contributes to the compromise of an environment resulting in large-scale casualties or significant financial loss" (Meta Advanced AI Scaling Framework v2, section 3.4.1)
Meta's scenario TS.1.1 under outcome CB 1: "Small cells of low or moderate skill actors acquire sufficient scientific capabilities to succeed at complex chemical or biological workflows" (Meta Advanced AI Scaling Framework v2, section 3.4.2)

Counter-examples

Not to be confused with

A threat model is broader than a scenario: it covers a whole risk, such as non-novel chemical and biological weapons, and can contain several scenarios. A risk pathway is the causal route; a scenario is one worked case along it.

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

Harm journey: A harm journey, in Google DeepMind's Gemini 3.7 Flash report, is "A breakdown of key stages and substages required to carry out such an attack end-to-end", used to test whether a model gives help at each stage of a threat scenario.; Source: Gemini 3.7 Flash FSF report
Bottleneck: A bottleneck is a step in a threat scenario that blocks most attempts. Google DeepMind tests models on "bottleneck" sub-stages "that present the greatest barriers to a threat actor's success"; xAI's framework calls such inhibiting steps bottlenecks.; Source: Gemini 3.7 Flash FSF report
Enabling capability: An enabling capability, in Meta's framework, is one of "a set of capabilities that are identified as essential to enabling the realization of a threat scenario", such as autonomous offensive operations in Meta's cyber scenarios.; Source: Meta Advanced AI Scaling Framework v2
Catastrophic outcome: A catastrophic outcome, in Meta's framework, is an outcome "that would have large scale, devastating, and potentially irreversible harmful impacts on humanity". Meta builds its threshold system by working backwards from these outcomes to threat scenarios.; Source: Meta Advanced AI Scaling Framework v2

Source of record

Where this definition comes from

Meta Advanced AI Scaling Framework v2, Appendix I, p.41
“"Threat scenarios describe the real-world events – including enabling capabilities, deployment context, and threat actors (as relevant) – that may be sufficient to produce a catastrophic outcome."”
https://ai.meta.com/static-resource/Meta_Advanced-AI-Scaling-Framework-v2
EU GPAI Code of Practice, Safety and Security Chapter, Glossary, p.33; Measure 2.2, p.11
“"'systemic risk scenario' a scenario in which a systemic risk stemming from a model might materialise." Measure 2.2: "Signatories will develop appropriate systemic risk scenarios, including regarding the number and level of detail of these systemic risk scenarios, for each identified systemic risk (pursuant to Measure 2.1)."”
https://ec.europa.eu/newsroom/dae/redirection/document/118119

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.

Organisation	Their term, as published	Match & verification	Source
MetaShadow mapping Meta Advanced AI Scaling Framework v2	“"Threat scenarios describe the real-world events – including enabling capabilities, deployment context, and threat actors (as relevant) – that may be sufficient to produce a catastrophic outcome." (Appendix I). Scenarios carry identifiers, e.g. "TS.1.1: An AI system substantially contributes to the compromise of an environment resulting in large-scale casualties or significant financial loss" (§3.4.1).”	exactEQ confidence: high	Meta Advanced AI Scaling Framework v2
EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter	“"'systemic risk scenario' a scenario in which a systemic risk stemming from a model might materialise" (Glossary); "Signatories will develop appropriate systemic risk scenarios ... for each identified systemic risk" (Measure 2.2).”	exactEQ confidence: high	EU GPAI Code of Practice, Safety and Security Chapter
Frontier Model ForumShadow mapping FMF Risk Taxonomy and Thresholds	“"the community develops detailed threat scenarios that specify how adversaries might use frontier AI to achieve severe outcomes, identifying specific tasks, model capabilities to exploit, and complementary tools" (s2.1, p.8).”	exactEQ confidence: high	FMF Risk Taxonomy and Thresholds
Google DeepMindShadow mapping Gemini 3.7 Flash FSF report	“"prioritize scenarios (combinations of threat actor types and agents, weapons, or attacks)"; "For each detailed scenario, a corresponding detailed harm journey is established" (p.8).”	closeCL confidence: high	Gemini 3.7 Flash FSF report
MicrosoftShadow mapping Microsoft Frontier Governance Framework (Feb 2026)	“Risk models "are designed to prioritize specific scenarios, threat actors, unique ways an AI model may increase or decrease risk, technical bottlenecks, and the severity and likelihood of the scenario resulting in significant impacts" (section 2, p.4).” Scenarios are named as a component of risk models; no scenario list is published.	closeCL confidence: high	Microsoft Frontier Governance Framework (Feb 2026)
xAIShadow mapping xAI Frontier AI Framework (30 Jun 2026, draft-marked)	“xAI "has developed systemic risk scenarios that enumerate the causal factors, potential harms, and mitigations" (s.2.1). The scenarios themselves are not published.” PDF metadata /Title reads "Privileged/Confidential DRAFT working FRAMEWORK DOC"; no xAI statement disambiguating draft vs. final was found. As of 13 Nov 2026 the cited URL returns an S3 "NoSuchKey" error; the quote is verified against the local capture made on 16 Sep 2026.	noneDU confidence: medium	xAI Frontier AI Framework (30 Jun 2026, draft-marked)

What do these codes mean?

exact: The source term is equivalent to this element
close: The source term is close but not equivalent to this element
broad: The source term is broader than this element
narrow: The source term is narrower than this element
none: No mapping claim — used for false-friend and declared-but-undefined rows
EQ: Equivalent
CL: Close
BR: Source is broader than the element
NR: Source is narrower than the element
FF: False friend — same or similar label, different meaning
DU: Declared but undefined by the source
UV: Unverified

Divergence

Where sources materially disagree

The granularity differs. Meta's threat scenarios are real-world event sequences that "may be sufficient to produce a catastrophic outcome", each with an identifier. Google DeepMind's scenarios are combinations of a threat actor type with an agent, weapon or attack, each broken into a harm journey. The EU chapter leaves "the number and level of detail" of scenarios to each signatory.

Gap

Among the sources reviewed, published scenario identifiers (TS.1.1 style) were found only in Meta's framework, which withholds "full details of the constituent steps and tasks within a threat scenario" (§3.4). xAI says its systemic risk scenarios exist but has not published them. The EU chapter requires scenarios but does not require their publication beyond the Model Report.