Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.
NIKOLAI elementN7 · IncidentsProposednikolai-v0.3

Incident type

Definition

An incident type is the category an AI incident falls into, such as unauthorised access to model weights, harm from a catastrophic risk, loss of control of a model, deceptive subversion of a developer's controls, or a security breach. Some regimes, such as the EU Code of Practice, set the reporting deadline by type.

Also called: Incident classification

NIKOLAI's proposed operational definition

NIKOLAI editorial proposal (unsourced): Incident type is a controlled value classifying an Incident by mechanism and severity class, so that similar events can be compared across developers. Candidate value groupings observed across sources include weight/model exfiltration or unauthorized access, loss-of-control or deceptive-subversion events, materialized catastrophic-risk harms, and lower-severity "precursor" or near-miss events -- but no two sources currently share one enumeration, so NIKOLAI's own ladder is a proposal, not a transcription of any single source's list.

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

  • California and New York use the same four limbs: weight access or exfiltration "that results in death or bodily injury", "harm resulting from the materialization of a catastrophic risk", "loss of control of a frontier model causing death or bodily injury", and deceptive subversion of the developer's controls (California SB 53, 22757.11(d); NY Gen. Bus. Law 1420(4))
  • The EU Code of Practice ties type to deadline: a serious and irreversible critical-infrastructure disruption (two days), a serious cybersecurity breach "including the (self-)exfiltration of model weights and cyberattacks" (five days), a death (10 days), or serious harm to health, fundamental rights, property or the environment (15 days) (EU GPAI Code of Practice, Safety and Security Chapter, Measure 9.3)
  • Anthropic classed its July 2026 evaluation incidents as "closer to a harness and operational failure than a model alignment failure", a split by cause rather than by harm (Anthropic, Investigating three real-world incidents in our cybersecurity evaluations)

Counter-examples

Not to be confused with

  • Incident type is not incident severity: SB 53's types each carry their own harm threshold, while the OECD separates the same harm categories into AI incident, serious AI incident and AI disaster by seriousness.
  • An incident caused by a user's deliberate misuse and one initiated by the model without any malicious user can share a type such as cyberattack; S. 5061 records the difference as a causal factor ("including misuse of the artificial intelligence system"), not as a type.

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

AI security incident
AI security incident, in US Senate bill S. 5061 (not enacted), is an event that materially increases the risk of unauthorised extraction of information about an AI system, or the ability to manipulate it to subvert its confidentiality, integrity or availability.
Source: S. 5061 (119th Congress)
Model weight exfiltration
Model weight exfiltration is the access to or transfer of a model's weights, or associated assets, out of secure storage by an unauthorised actor or by the model itself (self-exfiltration). The EU Code of Practice gives such a breach a five-day reporting deadline.
Source: EU GPAI Code of Practice, Safety and Security Chapter
Sandbox escape
Sandbox escape is Anthropic's term for a case where a model exploits a flaw in its sandbox, the isolated test environment, to reach systems it should be walled off from. Anthropic lists it separately from sandbox misconfigurations.
Source: Anthropic -- Improving Alignment & Security Efforts
Unsanctioned agent behaviour
Unsanctioned agent behaviour is the UK AI Security Institute's label for autonomous actions an AI agent takes beyond its authorised scope, such as acting on the live internet against real people or organisations during testing.
Source: UK AI Security Institute -- Incident Report: unsanctioned agent behaviour during cyber testing (4 Aug 2026)

Source of record

Where this definition comes from

  • California SB 53 (Transparency in Frontier Artificial Intelligence Act), Chapter 138, Statutes of 2025, Cal. Bus. & Prof. Code 22757.11(d)

    “"(1) Unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury. (2) Harm resulting from the materialization of a catastrophic risk. (3) Loss of control of a frontier model causing death or bodily injury. (4) A frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk."”

    https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53
  • EU GPAI Code of Practice, Safety and Security Chapter, Measure 9.3(2)

    “a serious cybersecurity breach, including the (self-)exfiltration of model weights and cyberattacks, or if the Signatories establish or suspect with reasonable likelihood such a causal relationship between their model and the breach, not later than five days after the Signatories become aware of the involvement of their model in the incident”

    https://ec.europa.eu/newsroom/dae/redirection/document/118119
  • S. 5061 (119th Congress), Sec. 4(a)(1)(B)-(C)

    “Mechanisms shall "develop processes to associate reports pertaining to the same incident with a single incident identifier" and "establish classification, information retrieval, and reporting mechanisms that sufficiently differentiate between artificial intelligence security incidents and artificial intelligence safety incidents" (Sec. 4(a)(1)(B)-(C)).”

    https://www.congress.gov/bill/119th-congress/senate-bill/5061/text

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
OrganisationTheir term, as publishedMatch & verificationSource
AnthropicShadow mapping
Anthropic Improving Alignment & Security Efforts; Anthropic Alignment Assessment: Cybersecurity Incidents
“Monitors looked for "sandbox escapes (cases where a model exploits a flaw in our sandbox to reach systems it should be walled off from) or sandbox misconfigurations" (IMP). The alignment assessment of the July 2026 incidents names two forms of misalignment: "biased reasoning" and "recklessness, or a willingness to take harmful actions in the narrow pursuit of a task" (ALA).”
The misalignment quotes are from Anthropic's alignment assessment: https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents
closeCL
confidence: medium
Anthropic -- Improving Alignment & Security Efforts
OpenAIShadow mapping
OpenAI/Hugging Face Incident Technical Report; OpenAI -- Hugging Face Incident and the Road Ahead; Path to Astra
“"This incident is the first known case of an automated agent collective acting offensively without authorization" (§VII.A). Four misalignment patterns: "reward hacking, persistence on seemingly impossible tasks, unauthorized communication, and agents adopting goals from one another". Astra: "Unauthorized model cyber action".”
The four-patterns quote is from OpenAI's post "Hugging Face incident and the road ahead" (https://openai.com/index/hugging-face-incident-and-the-road-ahead/); the "Unauthorized model cyber action" label is from OpenAI's "Path to Astra" page.
narrowNR
confidence: medium
OpenAI/Hugging Face Incident Technical Report
EUShadow mapping
EU GPAI Code of Practice, Safety and Security Chapter
“No discrete incident-type taxonomy; Measure 9.2's nine data fields (above) function as the incident record schema rather than a type enum, and severity is expressed through which of Measure 9.3's four reporting-clock tiers applies (critical-infrastructure disruption; cybersecurity breach incl. weight exfiltration; death; other serious harm to health/rights/property/environment) rather than through a named "type" field.”closeCL
confidence: medium
EU GPAI Code of Practice, Safety and Security Chapter
California SB 53Shadow mapping
California SB 53
“"(1) Unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury. (2) Harm resulting from the materialization of a catastrophic risk. (3) Loss of control of a frontier model causing death or bodily injury. (4) A frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk." (22757.11(d)). The Labor Code version is broader (1107(c)).”exactEQ
confidence: high
California SB 53
UK AI Security Institute (AISI)Shadow mapping
UK AI Security Institute -- Incident Report: unsanctioned agent behaviour during cyber testing (4 Aug 2026)
“"unsanctioned agent behaviour": in 10 of 122 runs "an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations". AISI separates this from a sandbox escape: "this was not a case of a model escaping its secure test environment, or 'sandbox'."”
A label for one class of event, not a full incident-type enumeration.
narrowNR
confidence: medium
UK AI Security Institute -- Incident Report: unsanctioned agent behaviour during cyber testing (4 Aug 2026)
Frontier Model ForumShadow mapping
Frontier Model Forum -- Information Sharing, Incident Reporting and Incident Response Issue Brief
“"Lower-severity incidents or precursor events", including "anomaly, near-miss, false positive or concerning signal"; information categories "Vulnerabilities, weaknesses, and exploitable flaws", "Threats", "Capabilities of concern" (Table 2).”closeCL
confidence: medium
Frontier Model Forum -- Information Sharing Issue Brief
US Congress (S. 5061, 119th Congress, not enacted)Shadow mapping
S. 5061
“S. 5061 defines "artificial intelligence safety incident" (Sec. 2(3)) and "artificial intelligence security incident" (Sec. 2(4)) separately, and requires mechanisms that "sufficiently differentiate between artificial intelligence security incidents and artificial intelligence safety incidents" and that "create appropriate taxonomies to classify incidents based on relevant characteristics, impact, or other relevant criteria" (Sec. 4(a)(1)(C)-(D)).”
S. 5061 is a bill introduced in the US Senate on 21 July 2026; it has not been enacted.
closeCL
confidence: medium
S. 5061 (119th Congress)
New York (RAISE Act, S8828)Shadow mapping
NY RAISE Act (Chapter 96 of the Laws of 2026), s.1420(4)
“"Critical safety incident" means "(a) unauthorized access to, modification of, or exfiltration of, the model weights of a frontier model that results in death or bodily injury; (b) harm resulting from the materialization of a catastrophic risk; (c) loss of control of a frontier model causing death or bodily injury; or (d) a frontier model that uses deceptive techniques against the frontier developer to subvert the controls or monitoring of its frontier developer outside of the context of an evaluation designed to elicit this behavior and in a manner that demonstrates materially increased catastrophic risk." (s.1420(4))”
Same four limbs as California SB 53 22757.11(d); in force 1 January 2027.
exactEQ
confidence: high
NY RAISE Act as replaced by S8828, Chapter 96 of the Laws of 2026 (N.Y. Gen. Bus. Law Art. 44-B)
EUShadow mapping
EU AI Act (Regulation (EU) 2024/1689), Article 3(49)
“"'serious incident' means an incident or malfunctioning of an AI system that directly or indirectly leads to any of the following: (a) the death of a person, or serious harm to a person's health; (b) a serious and irreversible disruption of the management or operation of critical infrastructure; (c) the infringement of obligations under Union law intended to protect fundamental rights; (d) serious harm to property or the environment" (Art. 3(49)). The four harm categories act as the type list.”closeCL
confidence: high
EU AI Act (Regulation (EU) 2024/1689), Article 3
OECDShadow mapping
OECD, Defining AI incidents and related terms (2024)
“Harm categories (a)-(d) shared by "AI incident", "serious AI incident" and "AI hazard": "(a) injury or harm to the health of a person or groups of people; (b) disruption of the management and operation of critical infrastructure; (c) violations of human rights or a breach of obligations under the applicable law intended to protect fundamental, labour and intellectual property rights; (d) harm to property, communities or the environment."”
Classifies by harm, not by mechanism.
closeCL
confidence: medium
OECD, Defining AI incidents and related terms (OECD Artificial Intelligence Papers No. 16, 2024)
What do these codes mean?
exact
The source term is equivalent to this element
close
The source term is close but not equivalent to this element
broad
The source term is broader than this element
narrow
The source term is narrower than this element
none
No mapping claim — used for false-friend and declared-but-undefined rows
EQ
Equivalent
CL
Close
BR
Source is broader than the element
NR
Source is narrower than the element
FF
False friend — same or similar label, different meaning
DU
Declared but undefined by the source
UV
Unverified

Gap

SB 53's and the RAISE Act's four limbs all require death, bodily injury, a materialised catastrophic risk or "materially increased catastrophic risk". The research found no public statement that any 2026 evaluation-environment incident (Anthropic, OpenAI/Hugging Face, UK AISI) was classified under those limbs or filed as a critical safety incident. The Frontier Model Forum's category of lower-severity incidents and precursor events is the closest published vocabulary for such events.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →