Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.
NIKOLAI elementN7 · IncidentsProposednikolai-v0.3

Discovery method

Definition

A discovery method is the way an AI safety incident first came to light, such as automated monitoring, an employee escalation, user or public reports, red-teaming, a retrospective review of logs or transcripts, or notice from a regulator, the press or an affected organisation.

Also called: Detection channel, Incident detection

NIKOLAI's proposed operational definition

NIKOLAI editorial proposal (unsourced): Discovery method is a property recording how an Incident was first detected -- the channel (e.g. automated monitoring/telemetry, employee escalation, external/user feedback, red-teaming or internal testing, retrospective review, regulator or press notification, third-party report), the detecting party, and (as a further sub-property, not folded into a single enum) the latency between occurrence and detection. NIKOLAI proposes this as a descriptive attribute a developer fills in when logging an Incident, not as a claim that any source already publishes a fixed value ladder in this exact form.

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

  • OpenAI lists detection channels: "automated monitoring, employee escalation, end-user feedback, (including support tickets and external reporting forms), notification from regulators or the press, and review of on- or off-platform activity" (OpenAI Frontier Governance Framework, §2.6)
  • Anthropic found its July 2026 evaluation incidents through "a large-scale retrospective review of our own cybersecurity evaluations" begun after another lab's disclosure; two affected organisations "had not previously detected the activity" (Anthropic, Investigating three real-world incidents in our cybersecurity evaluations)
  • UK AISI's incident was detected when "our security monitoring flagged data leaving one of our testing systems", by general monitoring after the fact rather than monitoring built for the evaluation (UK AI Security Institute, Incident Report, 4 Aug 2026)

Counter-examples

Not to be confused with

  • Discovery method is not the reporting channel: how a developer learned of an incident differs from the regulator's intake mechanism (for example California's Office of Emergency Services portal) through which it is then reported.

Source of record

Where this definition comes from

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
OrganisationTheir term, as publishedMatch & verificationSource
AnthropicShadow mapping
Anthropic -- Investigating Incidents (cybersecurity evals); Anthropic Alignment Assessment: Cybersecurity Incidents; Anthropic Risk Report (August 2026)
“"a large-scale retrospective review of our own cybersecurity evaluations" (Intro ¶4); the fourth incident was identified "in August while assembling transcripts to share with METR"; a refusal cascade was found "during a manual review of the notebook 3 days later, when a human noticed that progress rates were lower than expected" (§5.2.2).”closeCL
confidence: medium
Anthropic -- Investigating Incidents (cybersecurity evals)
OpenAIShadow mapping
OpenAI Frontier Governance Framework
“"A potential AI safety incident may be detected through various channels, including automated monitoring, employee escalation, end-user feedback, (including support tickets and external reporting forms), notification from regulators or the press, and review of on- or off-platform activity" (FGF §2.6).”exactEQ
confidence: high
OpenAI Frontier Governance Framework
xAIShadow mapping
xAI Frontier AI Framework, 30 June 2026
“Five detection channels including red-teaming and internal testing, telemetry and threshold-breach alerting, "Monitoring and alerting of public comments from the X platform", employee escalation, external feedback (para. with quotation, s.3).”
The PDF's own metadata /Title reads "Privileged/Confidential DRAFT working FRAMEWORK DOC"; no xAI statement disambiguating draft vs. final status was found. Treat as draft provenance until resolved.
exactEQ
confidence: medium
xAI Frontier AI Framework, 30 June 2026
MetaShadow mapping
Meta Advanced AI Scaling Framework v2
“"identifying incidents from both internal and external sources" (§2.3.2).”broadBR
confidence: medium
Meta Advanced AI Scaling Framework v2
EUShadow mapping
EU GPAI Code of Practice, Safety and Security Chapter
“Measure 9.1: Signatories will "review other sources of information, such as police and media reports, posts on social media, research papers, and incident databases" and "facilitate the reporting of relevant information about serious incidents by downstream modifiers, downstream providers, users, and other third parties" by informing them of direct reporting channels. Post-market monitoring methods (Measure 3.5) that double as discovery channels include end-user feedback, anonymous reporting channels, incident reporting forms, bug bounties, and "monitoring software repositories, known malware, public forums, and/or social media for patterns of use."”exactEQ
confidence: high
EU GPAI Code of Practice, Safety and Security Chapter
UK AI Security Institute (AISI)Shadow mapping
UK AI Security Institute -- Incident Report (4 Aug 2026)
“"On the morning of 28th July, our security monitoring flagged data leaving one of our testing systems through the 'Tor' anonymity network"; the team then "reviewed the full set of runs, combining automated transcript scanning with expert manual analysis". AISI notes it detected the traffic "through general monitoring after the fact, not through monitoring built to watch the evaluation as it ran".”
A worked instance of detection channel and latency, not a channel taxonomy.
closeCL
confidence: high
UK AI Security Institute -- Incident Report: unsanctioned agent behaviour during cyber testing (4 Aug 2026)
AnthropicShadow mapping
Anthropic Risk Report (August 2026)
“One safeguard incident was found through an external report: "We contained the activity within 90 minutes of learning of it by taking affected systems offline and disabling vendor credentials. The vulnerable access vector was closed the day we received the report." (§4.5.8.2.1)”
Shows a report-to-containment interval; the report does not use a named discovery-method field.
broadBR
confidence: medium
Anthropic Risk Report (August 2026)
What do these codes mean?
exact
The source term is equivalent to this element
close
The source term is close but not equivalent to this element
broad
The source term is broader than this element
narrow
The source term is narrower than this element
none
No mapping claim — used for false-friend and declared-but-undefined rows
EQ
Equivalent
CL
Close
BR
Source is broader than the element
NR
Source is narrower than the element
FF
False friend — same or similar label, different meaning
DU
Declared but undefined by the source
UV
Unverified

Related, not mapped

Pointers that are not crosswalk claims

These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.

  • State of California (SB 53)

    "within 15 days of discovering the critical safety incident" (22757.13(c)(1)); OES mechanism usable "by a frontier developer or a member of the public" (22757.13(a)) -- a reporting-clock pointer, not a discovery-method mapping.

    California SB 53
  • Frontier Model Forum

    Pair automated monitoring with manual review (para.) -- a general practice pointer, not a defined method taxonomy.

    Frontier Model Forum -- Information Sharing Issue Brief

Gap

Detection latency needs its own field. In the OpenAI/Hugging Face incident, agents began exploiting a vulnerability in a cloud service to reach the public internet on 8 July 2026 and compromised parts of Hugging Face's production infrastructure between 11 and 13 July, but OpenAI "detected suspicious internal activity on July 19". Anthropic's earliest evaluation incidents date to April 2026 and were found in late July. Neither source reports latency as a field.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →