Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.
NIKOLAI elementN6 · Mitigations and securityProposednikolai-v0.3

Monitor

Definition

A monitor is an automated or human process that watches a model's inputs, outputs, reasoning or actions to detect a specified behaviour, such as misuse or sabotage, and escalates what it flags. It is described by what it sees, how much traffic it samples and who reviews its flags.

Also called: Usage Monitoring, Offline monitoring, Asynchronous monitoring, Monitor AI

NIKOLAI's proposed operational definition

NIKOLAI proposes to define a Monitor record as: an automated or human process that observes model inputs, outputs, reasoning, actions, or internal state to detect a specified behavior, carrying a stated coverage scope, sampling rate, and escalation path when a detection fires. This is an unsourced NIKOLAI editorial synthesis of how Anthropic, OpenAI, and Google DeepMind each independently operate chain-of-thought and output monitors as a distinct mitigation category from access-control safeguards.

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

  • Anthropic runs "A Claude Opus 4.6 completion-only monitor covering completions with extended thinking by most internal users", and second-stage filtering surfaces cases that AI control researchers "manually review on the order of 50 trajectories per week" (Anthropic Risk Report, August 2026, §2.23.1.1)
  • OpenAI lists as an example misalignment safeguard: "Monitor AI supervises agent actions to detect agent in the case of trying to take harmful actions, scheming, or similar" (OpenAI Preparedness Framework v2, Appendix C.2)

Counter-examples

Not to be confused with

  • A blocking classifier that stops a response in real time is recorded as a classifier; a monitor may only flag activity for later review. Many systems use both.

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

Chain-of-thought monitoring
Chain-of-thought monitoring is reading the step-by-step reasoning a model writes out, often with another model, to catch signs of intent to misbehave. Its proponents call it promising but imperfect, and fragile if development choices reduce how monitorable the reasoning is.
Source: Korbak et al., Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety (arXiv 2507.11473)
Post-market monitoring
Post-market monitoring is the ongoing collection of information about a model after release, through user feedback, incident reports, bug bounties and usage analysis, to judge whether its systemic risks remain acceptable. The term is used in the EU GPAI Code of Practice.
Source: EU GPAI Code of Practice, Safety and Security Chapter

Source of record

Where this definition comes from

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
OrganisationTheir term, as publishedMatch & verificationSource
AnthropicShadow mapping
Anthropic Risk Report, August 2026 / Training a Misaligned Reward Seeker
“"A Claude Opus 4.6 completion-only monitor covering completions with extended thinking by most internal users, with no subsampling for traffic coming from the most common agent scaffolds"; "A Claude Opus 4.8 prompt+completion monitor ... This covers a 25% subsample of prompts and completions in which a tool call might take an impactful action"; reviewers "manually review on the order of 50 trajectories per week" (§2.23.1.1); "we monitor the Chain-of-Thought to detect and measure reward hacking, but importantly do not reward or penalize the model based on it, in order to preserve monitorability"”
Term also draws on Anthropic's "Training a Misaligned Reward Seeker" for the CoT-monitoring/severity-flagging quote.
exactEQ
confidence: high
Anthropic Risk Report, August 2026
OpenAIShadow mapping
OpenAI Preparedness Framework v2 / Pacing Model Development (Cyber), Aug 2026
“"Monitor AI supervises agent actions to detect agent in the case of trying to take harmful actions, scheming, or similar" (PF v2, App. C.2); OpenAI's Pacing Model Development post (Aug 2026) describes monitoring as a safeguard "which detects and allows us to respond to concerning behavior", with activation classifiers, "detectors that inspect a model's internal activity", escalating to "increasingly sophisticated and high-compute automated investigators"”
Only the first quote is from PF v2. The other quotes are from OpenAI's Aug 2026 post, read through a reader proxy because openai.com refused automated requests; CASRAI has not verified them against the live page.
exactEQ
confidence: high
OpenAI Preparedness Framework v2
Google DeepMindShadow mapping
Frontier Safety Framework v3.1
“Deployment mitigations include "input/output/chain-of-thought monitoring and analysis" (glossary); safeguards "such as chain-of-thought monitoring on high-risk deployments"”closeCL
confidence: medium
Google DeepMind Frontier Safety Framework v3.1
xAIShadow mapping
Frontier AI Framework, 30 Jun 2026
“Incident detection channel "Monitoring and alerting of public comments from the X platform"; security measures include real-time monitoring (described, not quoted)”
xAI's Frontier AI Framework (30 Jun 2026) carries PDF metadata reading "Privileged/Confidential DRAFT working FRAMEWORK DOC" with no xAI statement found disambiguating draft from final; the cited URL also returned HTTP 404 (NoSuchKey) on 13 Nov 2026, so the quote is verified against the local capture only. Treat as provisional.
narrowNR
confidence: medium
xAI Frontier AI Framework, 30 June 2026 (draft-labeled PDF metadata)
MetaShadow mapping
Meta Advanced AI Scaling Framework v2
“LlamaFirewall "chain-of-thought auditing" and input classifiers”narrowNR
confidence: medium
Meta Advanced AI Scaling Framework v2
EUShadow mapping
EU GPAI Code of Practice, Safety and Security Chapter
“Measure 3.5 "Post-market monitoring": Signatories "will conduct appropriate post-market monitoring to gather information relevant to assessing whether the systemic risk could be determined to not be acceptable ... and to inform whether a Model Report update is necessary", via methods including end-user feedback, incident reporting forms, bug bounties, "monitoring software repositories, known malware, public forums, and/or social media", and "monitoring aspects of models that are relevant for assessing and mitigating systemic risk and are not transparent to third parties, e.g. hidden chains-of-thought"”
This confirms xAI's chapter-derived s.2.2(5) wording; xAI's framework carries the draft/final metadata caveat above.
exactEQ
confidence: high
EU GPAI Code of Practice, Safety and Security Chapter
Frontier Model ForumShadow mapping
FMF Information Sharing / Incident Reporting Issue Brief
“"Monitoring and Detection Systems: Enhancing systems that detect anomalous behavior, unauthorized access, or potential misuse, for safety and security purposes only" (Table 3)”closeCL
confidence: medium
Frontier Model Forum, Information Sharing / Incident Reporting Issue Brief
What do these codes mean?
exact
The source term is equivalent to this element
close
The source term is close but not equivalent to this element
broad
The source term is broader than this element
narrow
The source term is narrower than this element
none
No mapping claim — used for false-friend and declared-but-undefined rows
EQ
Equivalent
CL
Close
BR
Source is broader than the element
NR
Source is narrower than the element
FF
False friend — same or similar label, different meaning
DU
Declared but undefined by the source
UV
Unverified

Related, not mapped

Pointers that are not crosswalk claims

These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.

  • SB 53 (California)

    Framework topic "(10) ... including risks resulting from a frontier model circumventing oversight mechanisms" (22757.12(a)) — related to monitoring but a pointer, not a mapping.

    California SB 53
  • UK AISI / Google DeepMind

    "CoT monitoring helps us understand how an AI system produces its answers, complementing interpretability research." — related, not a direct monitor-record mapping.

    DeepMind, Deepening Our Partnership with UK AI Security Institute

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →