Examples
Examples from published documents
- Under California law a critical safety incident includes "Loss of control of a frontier model causing death or bodily injury" and a frontier model using "deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior" (California SB 53, Cal. Bus. & Prof. Code 22757.11(d))
- In July 2026 a review of 141,006 evaluation runs found "three incidents in which a model accessed the internet from within or while interacting with the evaluation environment" and then gained unauthorised access to real organisations' systems (Anthropic, Investigating three real-world incidents in our cybersecurity evaluations)
- On 28 July 2026 AISI's security monitoring flagged data leaving a testing system; AISI "declared a security incident, stopped all related evaluations" and found agents had taken unsanctioned actions against real people and organisations (UK AI Security Institute, Incident Report, 4 Aug 2026)
Counter-examples
Not to be confused with
- A hazard or near miss is not an incident: the OECD defines an AI hazard as an event that "could plausibly lead to an AI incident", and the EU Code of Practice treats a near miss as a serious incident that "could have, but ultimately did not, materialise".
- A capability-threshold crossing found in a pre-deployment evaluation is not by itself an incident; SB 53's deception limb expressly excludes behaviour inside "an evaluation designed to elicit this behavior".
Related terms
Related AI safety terms
Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.
- Critical safety incident
- Critical safety incident is the reportable-event term in California SB 53 and New York's RAISE Act: unauthorised weight access causing death or injury, harm from a materialised catastrophic risk, loss of control causing death or injury, or deceptive evasion of developer controls raising catastrophic risk.
- Source: California SB 53 (Transparency in Frontier Artificial Intelligence Act), Chapter 138, Statutes of 2025
- Serious incident
- Serious incident is the EU AI Act's term (Article 3(49)) for an AI incident or malfunction leading to death or serious harm to health, serious and irreversible disruption of critical infrastructure, infringement of fundamental-rights law, or serious harm to property or the environment.
- Source: EU AI Act (Regulation (EU) 2024/1689), Article 3
- AI incident
- AI incident, in the OECD's definition, is an event, circumstance or series of events where the development, use or malfunction of one or more AI systems directly or indirectly leads to harm to health, critical infrastructure, human rights, property, communities or the environment.
- Source: OECD, Defining AI incidents and related terms (OECD Artificial Intelligence Papers No. 16, 2024)
- AI safety incident
- AI safety incident, in US Senate bill S. 5061 (introduced 2026, not enacted), is an event that materially increases the risk that operating an AI system leads to a state in which human life, health, property or the environment is endangered.
- Source: S. 5061 (119th Congress)
- Serious AI incident
- Serious AI incident is the OECD's term for the subset of AI incidents that lead to death or serious harm to health, serious and irreversible disruption of critical infrastructure, serious human-rights violations, or serious harm to property, communities or the environment.
- Source: OECD, Defining AI incidents and related terms (OECD Artificial Intelligence Papers No. 16, 2024)
- AI disaster
- AI disaster, in the OECD's terminology, is a serious AI incident that disrupts the functioning of a community or a society and may test or exceed its capacity to cope using its own resources.
- Source: OECD, Defining AI incidents and related terms (OECD Artificial Intelligence Papers No. 16, 2024)
- Incident response
- Incident response is the coordinated process of containing, mitigating and recovering from an incident once it has been identified. It is real-time and operational, and is led mainly by the affected developer's technical, legal and security teams.
- Source: Frontier Model Forum -- Information Sharing Issue Brief
- Incident response plan
- Incident response plan is a developer's documented procedure for identifying and responding to incidents. OpenAI's AI Safety Incident Response Plan, for example, sets steps for detection, triage, severity, investigation, mitigation and deciding whether an incident must be reported externally.
- Source: OpenAI Frontier Governance Framework
Source of record
Where this definition comes from
EU GPAI Code of Practice, Safety and Security Chapter, Commitment 9 / Measure 9.2
“Signatories track, document and report "relevant information about serious incidents along the entire model lifecycle and possible corrective measures". Measure 9.2 lists nine required fields (start/end dates; "the resulting harm and the victim or affected group"; chain of events; "the model involved"; description of the model's involvement; response taken; recommendation to authorities; "a root cause analysis ..., including ... any failures or circumventions of systemic risk mitigations"; related post-market-monitoring patterns).”
https://ec.europa.eu/newsroom/dae/redirection/document/118119California SB 53 (Transparency in Frontier Artificial Intelligence Act), Chapter 138, Statutes of 2025, Cal. Bus. & Prof. Code 22757.11(d)
“"Critical safety incident" means any of the following: (1) Unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury. (2) Harm resulting from the materialization of a catastrophic risk. (3) Loss of control of a frontier model causing death or bodily injury. (4) A frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk.”
https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53Anthropic Advanced AI Framework, p.4 (definitions)
“"Critical Safety Incident. Refers to any of the following: (a) unauthorized exfiltration of a covered model's weights, or unauthorized, deliberate, malicious modification of a covered model's weights; (b) harm resulting from the materialization of a Catastrophic Risk; (c) loss of control of a model causing death or bodily injury; (d) a model that uses deceptive techniques against the Covered Developer to subvert its controls or monitoring outside the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased Catastrophic Risk."”
https://www-cdn.anthropic.com/files/4zrzovbb/website/0a58d567024a8b448ff15158ebc3625328dfcc1f.pdf
Crosswalk
How named organisations use this concept
| Organisation | Their term, as published | Match & verification | Source |
|---|---|---|---|
| AnthropicShadow mapping Anthropic Advanced AI Framework; Anthropic Risk Report (August 2026) | “AAF defines "Critical Safety Incident" in four limbs (p.4). The Risk Report keeps an incident register in §4.5.8, which "tries to paint a reasonably comprehensive picture of notable risk acceptances or safety-related incidents pertaining to our blocking bio classifiers and access controls", and a separate list of "Safety process failures": "historical cases where Anthropic's safety and security posture fell short of our ideal in some way" (§5.2.1).” The definition is from the Advanced AI Framework (p.4), a legislative proposal; the two register quotes are from Anthropic's Risk Report (August 2026), §5.2.1: https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdf | closeCL confidence: medium | Anthropic Advanced AI Framework |
| OpenAIShadow mapping OpenAI Frontier Governance Framework | “"AI Safety Incident Response Plan (AIRP)" (§2.6). The FGF uses the terms "AI safety incident", "potential AI safety incident", "critical safety incident" and "serious incident" without defining them; for critical safety incidents it points to the TFAIA (California SB 53).” | noneDU confidence: medium | OpenAI Frontier Governance Framework |
| Google DeepMindShadow mapping Google DeepMind Frontier Safety Framework v3.1 | “"incidents relating to our frontier safety risk domains" (s.1.3.2); no definition.” | noneDU confidence: medium | Google DeepMind Frontier Safety Framework v3.1 |
| xAIShadow mapping xAI Frontier AI Framework, 30 June 2026 | “"serious incident" (trigger point) and "Serious AI safety incidents: used, not defined" (s.2, s.3).” The FAIF26 PDF's own metadata /Title reads "Privileged/Confidential DRAFT working FRAMEWORK DOC"; no xAI statement disambiguating draft vs. final status was found. Treat this row's provenance as a draft document, not a confirmed-final xAI policy, until that is resolved. | noneDU confidence: medium | xAI Frontier AI Framework, 30 June 2026 |
| MetaShadow mapping Meta Advanced AI Scaling Framework v2 | “"major incident" and "reporting critical incidents as appropriate" (§2.2.2, §2.3.2); undefined.” | noneDU confidence: medium | Meta Advanced AI Scaling Framework v2 |
| EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter | “"serious incident" is not given a one-sentence definition in the chapter, but Commitment 9 fully operationalises it: Signatories track, document and report "relevant information about serious incidents along the entire model lifecycle and possible corrective measures". Measure 9.2 lists nine required fields (start/end dates; "the resulting harm and the victim or affected group"; chain of events; "the model involved"; description of the model's involvement; response taken; recommendation to authorities; "a root cause analysis ..., including ... any failures or circumventions of systemic risk mitigations"; related post-market-monitoring patterns).” | exactEQ confidence: high | EU GPAI Code of Practice, Safety and Security Chapter |
| California SB 53Shadow mapping California SB 53 | “"Critical safety incident" means any of the following: (1) Unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury. (2) Harm resulting from the materialization of a catastrophic risk. (3) Loss of control of a frontier model causing death or bodily injury. (4) A frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk. (22757.11(d)).” | exactEQ confidence: high | California SB 53 (Transparency in Frontier Artificial Intelligence Act), Chapter 138, Statutes of 2025 |
| UK AI Security Institute (AISI)Shadow mapping UK AI Security Institute -- Incident Report: unsanctioned agent behaviour during cyber testing (4 Aug 2026) | “AISI published an incident report on its own cyber testing: "During a routine cyber evaluation, AISI identified an incident in which AI agents took sustained, unsanctioned action directed at real people and organisations." It "declared a security incident" and states: "We are treating this as a serious incident". The word "incident" is used without a definition.” AISI uses "serious incident" in its ordinary sense; the post does not say whether the event met the EU or SB 53 legal definitions. | noneDU confidence: medium | UK AI Security Institute -- Incident Report: unsanctioned agent behaviour during cyber testing (4 Aug 2026) |
| Frontier Model ForumShadow mapping Frontier Model Forum -- Information Sharing, Incident Reporting and Incident Response Issue Brief | “"Incident" is not defined. "The definition and appropriate scope of what is considered an 'incident' depends on regulatory context, risk categories, and the capabilities of the systems involved."” | noneDU confidence: medium | Frontier Model Forum -- Information Sharing Issue Brief |
| EUShadow mapping EU AI Act (Regulation (EU) 2024/1689), Article 3(49) and Article 55(1)(c) | “"'serious incident' means an incident or malfunctioning of an AI system that directly or indirectly leads to any of the following: (a) the death of a person, or serious harm to a person's health; (b) a serious and irreversible disruption of the management or operation of critical infrastructure; (c) the infringement of obligations under Union law intended to protect fundamental rights; (d) serious harm to property or the environment" (Art. 3(49)). Providers of general-purpose AI models with systemic risk must "keep track of, document, and report, without undue delay, to the AI Office and, as appropriate, to national competent authorities, relevant information about serious incidents and possible corrective measures to address them" (Art. 55(1)(c)).” Article 3(49) is written for "an AI system"; the GPAI Code of Practice applies the same term to general-purpose AI models with systemic risk. | exactEQ confidence: high | EU AI Act (Regulation (EU) 2024/1689), Article 3 |
| OECDShadow mapping OECD, Defining AI incidents and related terms (2024) | “"An AI incident is an event, circumstance or series of events where the development, use or malfunction of one or more AI systems directly or indirectly leads to any of the following harms: (a) injury or harm to the health of a person or groups of people; (b) disruption of the management and operation of critical infrastructure; (c) violations of human rights or a breach of obligations under the applicable law intended to protect fundamental, labour and intellectual property rights; (d) harm to property, communities or the environment." The paper also defines "serious AI incident", "AI hazard", "serious AI hazard" and "AI disaster".” A definition offered for international interoperability, not a legal reporting obligation. | closeCL confidence: high | OECD, Defining AI incidents and related terms (OECD Artificial Intelligence Papers No. 16, 2024) |
| US Congress (S. 5061, 119th Congress, not enacted)Shadow mapping S. 5061, Sec. 2(3) | “"artificial intelligence safety incident" means "an event that materially increases the risk that operation of an artificial intelligence system leads to a state in which human life, health, property, or the environment is endangered." (Sec. 2(3))” S. 5061 is a bill introduced in the US Senate on 21 July 2026; it has not been enacted. | closeCL confidence: medium | S. 5061 (119th Congress) |
What do these codes mean?
- exact
- The source term is equivalent to this element
- close
- The source term is close but not equivalent to this element
- broad
- The source term is broader than this element
- narrow
- The source term is narrower than this element
- none
- No mapping claim — used for false-friend and declared-but-undefined rows
- EQ
- Equivalent
- CL
- Close
- BR
- Source is broader than the element
- NR
- Source is narrower than the element
- FF
- False friend — same or similar label, different meaning
- DU
- Declared but undefined by the source
- UV
- Unverified
Related, not mapped
Pointers that are not crosswalk claims
These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.
- METR
Independent incident investigation scoped to "seven specific questions" (a related pointer, not an incident-record-shape mapping).
METR -- OpenAI/Hugging Face Incident Investigation
Divergence
Where sources materially disagree
The word 'incident' collides across sources. California SB 53 and New York's RAISE Act give 'critical safety incident' a closed four-limb legal definition, the EU AI Act defines 'serious incident' by harm type, and the OECD offers a broader interoperable definition with separate tiers (AI incident, serious AI incident, AI disaster). OpenAI's Frontier Governance Framework, Google DeepMind's Frontier Safety Framework, xAI's framework, Meta's framework and UK AISI's incident report use the word without defining it, and the Frontier Model Forum states that the appropriate scope of 'incident' depends on regulatory context, risk categories and system capabilities. A record that says only 'incident' is therefore not comparable across sources unless it also names the definition it was assessed against.







