Examples
Examples from published documents
- California and New York use the same four limbs: weight access or exfiltration "that results in death or bodily injury", "harm resulting from the materialization of a catastrophic risk", "loss of control of a frontier model causing death or bodily injury", and deceptive subversion of the developer's controls (California SB 53, 22757.11(d); NY Gen. Bus. Law 1420(4))
- The EU Code of Practice ties type to deadline: a serious and irreversible critical-infrastructure disruption (two days), a serious cybersecurity breach "including the (self-)exfiltration of model weights and cyberattacks" (five days), a death (10 days), or serious harm to health, fundamental rights, property or the environment (15 days) (EU GPAI Code of Practice, Safety and Security Chapter, Measure 9.3)
- Anthropic classed its July 2026 evaluation incidents as "closer to a harness and operational failure than a model alignment failure", a split by cause rather than by harm (Anthropic, Investigating three real-world incidents in our cybersecurity evaluations)
Counter-examples
Not to be confused with
- Incident type is not incident severity: SB 53's types each carry their own harm threshold, while the OECD separates the same harm categories into AI incident, serious AI incident and AI disaster by seriousness.
- An incident caused by a user's deliberate misuse and one initiated by the model without any malicious user can share a type such as cyberattack; S. 5061 records the difference as a causal factor ("including misuse of the artificial intelligence system"), not as a type.
Related terms
Related AI safety terms
Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.
- AI security incident
- AI security incident, in US Senate bill S. 5061 (not enacted), is an event that materially increases the risk of unauthorised extraction of information about an AI system, or the ability to manipulate it to subvert its confidentiality, integrity or availability.
- Source: S. 5061 (119th Congress)
- Model weight exfiltration
- Model weight exfiltration is the access to or transfer of a model's weights, or associated assets, out of secure storage by an unauthorised actor or by the model itself (self-exfiltration). The EU Code of Practice gives such a breach a five-day reporting deadline.
- Source: EU GPAI Code of Practice, Safety and Security Chapter
- Sandbox escape
- Sandbox escape is Anthropic's term for a case where a model exploits a flaw in its sandbox, the isolated test environment, to reach systems it should be walled off from. Anthropic lists it separately from sandbox misconfigurations.
- Source: Anthropic -- Improving Alignment & Security Efforts
- Unsanctioned agent behaviour
- Unsanctioned agent behaviour is the UK AI Security Institute's label for autonomous actions an AI agent takes beyond its authorised scope, such as acting on the live internet against real people or organisations during testing.
- Source: UK AI Security Institute -- Incident Report: unsanctioned agent behaviour during cyber testing (4 Aug 2026)
Source of record
Where this definition comes from
California SB 53 (Transparency in Frontier Artificial Intelligence Act), Chapter 138, Statutes of 2025, Cal. Bus. & Prof. Code 22757.11(d)
“"(1) Unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury. (2) Harm resulting from the materialization of a catastrophic risk. (3) Loss of control of a frontier model causing death or bodily injury. (4) A frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk."”
https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53EU GPAI Code of Practice, Safety and Security Chapter, Measure 9.3(2)
“a serious cybersecurity breach, including the (self-)exfiltration of model weights and cyberattacks, or if the Signatories establish or suspect with reasonable likelihood such a causal relationship between their model and the breach, not later than five days after the Signatories become aware of the involvement of their model in the incident”
https://ec.europa.eu/newsroom/dae/redirection/document/118119S. 5061 (119th Congress), Sec. 4(a)(1)(B)-(C)
“Mechanisms shall "develop processes to associate reports pertaining to the same incident with a single incident identifier" and "establish classification, information retrieval, and reporting mechanisms that sufficiently differentiate between artificial intelligence security incidents and artificial intelligence safety incidents" (Sec. 4(a)(1)(B)-(C)).”
https://www.congress.gov/bill/119th-congress/senate-bill/5061/text
Crosswalk
How named organisations use this concept
| Organisation | Their term, as published | Match & verification | Source |
|---|---|---|---|
| AnthropicShadow mapping Anthropic Improving Alignment & Security Efforts; Anthropic Alignment Assessment: Cybersecurity Incidents | “Monitors looked for "sandbox escapes (cases where a model exploits a flaw in our sandbox to reach systems it should be walled off from) or sandbox misconfigurations" (IMP). The alignment assessment of the July 2026 incidents names two forms of misalignment: "biased reasoning" and "recklessness, or a willingness to take harmful actions in the narrow pursuit of a task" (ALA).” The misalignment quotes are from Anthropic's alignment assessment: https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents | closeCL confidence: medium | Anthropic -- Improving Alignment & Security Efforts |
| OpenAIShadow mapping OpenAI/Hugging Face Incident Technical Report; OpenAI -- Hugging Face Incident and the Road Ahead; Path to Astra | “"This incident is the first known case of an automated agent collective acting offensively without authorization" (§VII.A). Four misalignment patterns: "reward hacking, persistence on seemingly impossible tasks, unauthorized communication, and agents adopting goals from one another". Astra: "Unauthorized model cyber action".” The four-patterns quote is from OpenAI's post "Hugging Face incident and the road ahead" (https://openai.com/index/hugging-face-incident-and-the-road-ahead/); the "Unauthorized model cyber action" label is from OpenAI's "Path to Astra" page. | narrowNR confidence: medium | OpenAI/Hugging Face Incident Technical Report |
| EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter | “No discrete incident-type taxonomy; Measure 9.2's nine data fields (above) function as the incident record schema rather than a type enum, and severity is expressed through which of Measure 9.3's four reporting-clock tiers applies (critical-infrastructure disruption; cybersecurity breach incl. weight exfiltration; death; other serious harm to health/rights/property/environment) rather than through a named "type" field.” | closeCL confidence: medium | EU GPAI Code of Practice, Safety and Security Chapter |
| California SB 53Shadow mapping California SB 53 | “"(1) Unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury. (2) Harm resulting from the materialization of a catastrophic risk. (3) Loss of control of a frontier model causing death or bodily injury. (4) A frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk." (22757.11(d)). The Labor Code version is broader (1107(c)).” | exactEQ confidence: high | California SB 53 |
| UK AI Security Institute (AISI)Shadow mapping UK AI Security Institute -- Incident Report: unsanctioned agent behaviour during cyber testing (4 Aug 2026) | “"unsanctioned agent behaviour": in 10 of 122 runs "an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations". AISI separates this from a sandbox escape: "this was not a case of a model escaping its secure test environment, or 'sandbox'."” A label for one class of event, not a full incident-type enumeration. | narrowNR confidence: medium | UK AI Security Institute -- Incident Report: unsanctioned agent behaviour during cyber testing (4 Aug 2026) |
| Frontier Model ForumShadow mapping Frontier Model Forum -- Information Sharing, Incident Reporting and Incident Response Issue Brief | “"Lower-severity incidents or precursor events", including "anomaly, near-miss, false positive or concerning signal"; information categories "Vulnerabilities, weaknesses, and exploitable flaws", "Threats", "Capabilities of concern" (Table 2).” | closeCL confidence: medium | Frontier Model Forum -- Information Sharing Issue Brief |
| US Congress (S. 5061, 119th Congress, not enacted)Shadow mapping S. 5061 | “S. 5061 defines "artificial intelligence safety incident" (Sec. 2(3)) and "artificial intelligence security incident" (Sec. 2(4)) separately, and requires mechanisms that "sufficiently differentiate between artificial intelligence security incidents and artificial intelligence safety incidents" and that "create appropriate taxonomies to classify incidents based on relevant characteristics, impact, or other relevant criteria" (Sec. 4(a)(1)(C)-(D)).” S. 5061 is a bill introduced in the US Senate on 21 July 2026; it has not been enacted. | closeCL confidence: medium | S. 5061 (119th Congress) |
| New York (RAISE Act, S8828)Shadow mapping NY RAISE Act (Chapter 96 of the Laws of 2026), s.1420(4) | “"Critical safety incident" means "(a) unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury; (b) harm resulting from the materialization of a catastrophic risk; (c) loss of control of a frontier model causing death or bodily injury; or (d) a frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk." (s.1420(4))” Same four limbs as California SB 53 22757.11(d); in force 1 January 2027. | exactEQ confidence: high | NY RAISE Act as replaced by S8828, Chapter 96 of the Laws of 2026 (N.Y. Gen. Bus. Law Art. 44-B) |
| EUShadow mapping EU AI Act (Regulation (EU) 2024/1689), Article 3(49) | “"'serious incident' means an incident or malfunctioning of an AI system that directly or indirectly leads to any of the following: (a) the death of a person, or serious harm to a person's health; (b) a serious and irreversible disruption of the management or operation of critical infrastructure; (c) the infringement of obligations under Union law intended to protect fundamental rights; (d) serious harm to property or the environment" (Art. 3(49)). The four harm categories act as the type list.” | closeCL confidence: high | EU AI Act (Regulation (EU) 2024/1689), Article 3 |
| OECDShadow mapping OECD, Defining AI incidents and related terms (2024) | “Harm categories (a)-(d) shared by "AI incident", "serious AI incident" and "AI hazard": "(a) injury or harm to the health of a person or groups of people; (b) disruption of the management and operation of critical infrastructure; (c) violations of human rights or a breach of obligations under the applicable law intended to protect fundamental, labour and intellectual property rights; (d) harm to property, communities or the environment."” Classifies by harm, not by mechanism. | closeCL confidence: medium | OECD, Defining AI incidents and related terms (OECD Artificial Intelligence Papers No. 16, 2024) |
What do these codes mean?
- exact
- The source term is equivalent to this element
- close
- The source term is close but not equivalent to this element
- broad
- The source term is broader than this element
- narrow
- The source term is narrower than this element
- none
- No mapping claim — used for false-friend and declared-but-undefined rows
- EQ
- Equivalent
- CL
- Close
- BR
- Source is broader than the element
- NR
- Source is narrower than the element
- FF
- False friend — same or similar label, different meaning
- DU
- Declared but undefined by the source
- UV
- Unverified
Gap
SB 53's and the RAISE Act's four limbs all require death, bodily injury, a materialised catastrophic risk or "materially increased catastrophic risk". The research found no public statement that any 2026 evaluation-environment incident (Anthropic, OpenAI/Hugging Face, UK AISI) was classified under those limbs or filed as a critical safety incident. The Frontier Model Forum's category of lower-severity incidents and precursor events is the closest published vocabulary for such events.







