Examples
Examples from published documents
- In the AI Control paper's trusted-monitoring protocol, developers "monitor GPT-4's outputs with a trusted but less capable model, GPT-3.5" and show suspicious code to a human (Greenblatt et al., arXiv 2312.06942, §1)
- OpenAI's example misalignment safeguards include "System forces human approval of agent actions (either by default, or by escalation of the monitor)" (OpenAI Preparedness Framework v2, Appendix C.2)
Counter-examples
Not to be confused with
- Alignment training tries to make the model not want to cause harm; AI control assumes it might and limits what it can do anyway.
Related terms
Related AI safety terms
Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.
- Trusted monitoring
- Trusted monitoring is an AI control protocol in which a less capable but trusted model reviews a more capable untrusted model's outputs and sends suspicious ones to human review. It was introduced in the AI Control paper on code backdooring.
- Source: Greenblatt, Shlegeris, Sachan, Roger, AI Control: Improving Safety Despite Intentional Subversion (arXiv 2312.06942)
- Untrusted monitoring
- Untrusted monitoring is an AI control protocol in which another instance of the same powerful, untrusted model checks the first one's work, using techniques to prevent the two instances from colluding.
- Source: Greenblatt, Shlegeris, Sachan, Roger, AI Control: Improving Safety Despite Intentional Subversion (arXiv 2312.06942)
- Control evaluation
- A control evaluation estimates how safe a control protocol is by letting a red team replace the untrusted model with an attack policy that tries to cause harm, then measuring how often the protocol stops it.
- Source: Greenblatt, Shlegeris, Sachan, Roger, AI Control: Improving Safety Despite Intentional Subversion (arXiv 2312.06942)
- Compartmentalisation
- Compartmentalisation is restricting what each person or AI system can access, so no single actor can do excessive damage. Anthropic's RSP calls for internal compartmentalisation, restriction and code review to limit sabotage opportunities for AI models.
- Source: Anthropic Responsible Scaling Policy v3.4, effective 8 Jul 2026
Source of record
Where this definition comes from
Greenblatt, Shlegeris, Sachan, Roger, AI Control: Improving Safety Despite Intentional Subversion (arXiv 2312.06942), Abstract
“In this paper, we develop and evaluate pipelines of safety techniques ("protocols") that are robust to intentional subversion.”
https://arxiv.org/abs/2312.06942Google DeepMind, An Approach to Technical AGI Safety and Security (arXiv 2504.01849), Abstract
“Second, system-level security measures such as monitoring and access control can mitigate harm even if the model is misaligned.”
https://arxiv.org/abs/2504.01849
Crosswalk
How named organisations use this concept
Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
| Organisation | Their term, as published | Match & verification | Source |
|---|---|---|---|
| AI Control research (Greenblatt et al., arXiv 2312.06942)Shadow mapping AI Control: Improving Safety Despite Intentional Subversion | “"we develop and evaluate pipelines of safety techniques ('protocols') that are robust to intentional subversion" (abstract, inner quotation marks normalised); "We call this methodology for estimating the safety of a protocol, where the scenario is simulated using a red-team-provided implementation for the untrusted model, a control evaluation." (§1)” Research paper that introduced the term; not a framework or statute. | exactEQ confidence: high | Greenblatt, Shlegeris, Sachan, Roger, AI Control: Improving Safety Despite Intentional Subversion (arXiv 2312.06942) |
| Google DeepMindShadow mapping An Approach to Technical AGI Safety and Security / Frontier Safety Framework v3.1 | “"system-level security measures such as monitoring and access control can mitigate harm even if the model is misaligned" (arXiv 2504.01849); FSF §3.1.2 safeguards "may include measures such as limiting affordances, monitoring and escalation, auditing, and alignment training"” FSF v3.1 cites arXiv 2504.01849 (section 6) for its misalignment mitigations. | exactEQ confidence: high | Google DeepMind, An Approach to Technical AGI Safety and Security (arXiv 2504.01849) |
| OpenAIShadow mapping OpenAI Preparedness Framework v2 | “Safeguards against a misaligned model: "Reliable and Robust System Oversight: Effective AI and human oversight of model actions detects and prevents execution of harm or subversion of safeguards."; "System Architecture: The model can't take actions that cause harm because it lacks access to output channels or mechanisms to persist sufficiently to execute the harm"” | closeCL confidence: high | OpenAI Preparedness Framework v2 |
| AnthropicShadow mapping Anthropic Responsible Scaling Policy v3.4 / Risk Report, August 2026 | “For misaligned AI in high-stakes settings: "Internal compartmentalization, restriction, and code review to prevent excessive sabotage opportunities for AI models." and "Monitoring and/or restricting AI behavior and usage internally." (RSP §1); the Risk Report routes flagged trajectories "to AI control researchers" (§2.23.1.1)” | closeCL confidence: high | Anthropic Responsible Scaling Policy v3.4, effective 8 Jul 2026 |
| EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter | “Measure 5.1 example: "other emerging safety mitigations, such as for achieving transparency into chain-of-thought reasoning or defending against a model's ability to subvert its other safety mitigations"” | broadBR confidence: medium | EU GPAI Code of Practice, Safety and Security Chapter |
What do these codes mean?
- exact
- The source term is equivalent to this element
- close
- The source term is close but not equivalent to this element
- broad
- The source term is broader than this element
- narrow
- The source term is narrower than this element
- none
- No mapping claim — used for false-friend and declared-but-undefined rows
- EQ
- Equivalent
- CL
- Close
- BR
- Source is broader than the element
- NR
- Source is narrower than the element
- FF
- False friend — same or similar label, different meaning
- DU
- Declared but undefined by the source
- UV
- Unverified
Related, not mapped
Pointers that are not crosswalk claims
These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.
- METR
Anthropic evaluated its monitors "using attack attempts by a METR staff member who red-teamed our Opus 4.6 prompt+completion monitor, several static AI control monitor benchmarks" (§2.23.1.2). Related external red-teaming of a control measure.
Anthropic Risk Report, August 2026
Gap
None of the lab frameworks uses "AI control" as a defined term; the mapping rests on their descriptions of safeguards against a misaligned model. Anthropic's Risk Report says it "did not perform end-to-end evaluations of our automated offline monitoring pipeline" (§2.23.3).
