Examples
Examples from published documents
- The EU Code of Practice lists Q&A sets, task-based evaluations, benchmarks, red-teaming and other adversarial testing, human uplift studies, model organisms, simulations and proxy evaluations for classified materials as example evaluation methods (EU GPAI Code of Practice, Safety and Security Chapter, Measure 3.2)
- OpenAI pairs automated scalable evaluations, which carry pre-set indicative thresholds, with deep dives such as human expert red-teaming and bio wet lab studies (OpenAI Preparedness Framework v2, section 3.1)
- Microsoft documents each capability evaluation with its assumptions about the model, the capability evaluated, the method and the results, plus a statement on the method's robustness and any concerns about its validity (Microsoft Frontier Governance Framework, p.6)
Counter-examples
Not to be confused with
- A threat model describes how harm could happen and guides which evaluations to run, but it produces no evidence about a model by itself.
- A system card or risk report is a document that reports evaluation results; the evaluation is the procedure that produced them.
Related terms
Related AI safety terms
Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.
- Dangerous capability evaluation
- A dangerous capability evaluation is a test of whether an AI model can do things that could enable severe harm, such as helping with weapons or cyberattacks, used to judge how close it is to a risk threshold.
- Source: Google DeepMind Frontier Safety Framework v3.1
- Propensity evaluation
- A propensity evaluation measures what an AI model tends to do rather than what it can do, for example its tendency toward deception or other misaligned behaviour, because capability alone does not show whether harm is likely.
- Source: Meta Advanced AI Scaling Framework v2
- Pre-deployment evaluation
- A pre-deployment evaluation is testing of an AI model before release, often including early access for government or independent evaluators, so that the findings can inform the decision to deploy.
- Source: GPT-5.6 deployment safety card
- Open-ended testing
- Open-ended testing is exploratory evaluation without a fixed task list, aimed at finding unexpected behaviours, capability boundaries or emergent properties that predefined evaluations would miss.
- Source: EU GPAI Code of Practice, Safety and Security Chapter
- Model organism
- A model organism, in AI safety, is a model deliberately trained to have a known problem, such as a hidden misaligned behaviour, so that evaluators can check whether their auditing and evaluation methods detect it.
- Source: Anthropic Risk Report (August 2026)
- Interpretability evidence
- Interpretability evidence is information about a model's internal computations, gathered with tools such as natural-language autoencoders or activation steering, used alongside behavioural tests to look for hidden capabilities or misalignment.
- Source: Anthropic Risk Report (August 2026)
Source of record
Where this definition comes from
OpenAI Preparedness Framework v2, §3.1
“Scalable Evaluations: automated evaluations designed to measure proxies that approximate whether a capability threshold has been crossed.”
https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbddebcd/preparedness-framework-v2.pdfGoogle DeepMind Frontier Safety Framework v3.1, Glossary
“Early Warning Evaluations: are evaluations which measure the dangerous capabilities of a model. They specifically target the threats and risk scenarios identified through our threat modeling to determine a model's proximity to a CCL or TCL.”
https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/strengthening-our-frontier-safety-framework/frontier-safety-framework_3-1.pdfEU GPAI Code of Practice, Safety and Security Chapter, Measure 3.2
“Signatories will conduct at least state-of-the-art model evaluations in the modalities relevant to the systemic risk to assess the model's capabilities, propensities, affordances, and/or effects.”
https://ec.europa.eu/newsroom/dae/redirection/document/118119
Crosswalk
How named organisations use this concept
| Organisation | Their term, as published | Match & verification | Source |
|---|---|---|---|
| AnthropicShadow mapping Anthropic Risk Report (August 2026) | “CoBench: "an internal evaluation measuring how well a model, placed at a historical point in Anthropic's infrastructure ..., can diagnose the root causes of issues that Anthropic engineers actually solved" (§3.4.3).” | exactEQ confidence: high | Anthropic Risk Report (August 2026) |
| AnthropicShadow mapping Anthropic Advanced AI Framework | “the safety framework must describe "capability evaluations performed" (p.5).” | exactEQ confidence: high | Anthropic Advanced AI Framework |
| OpenAIShadow mapping OpenAI Preparedness Framework v2 | “"Scalable Evaluations: automated evaluations designed to measure proxies that approximate whether a capability threshold has been crossed." "Deep Dives: designed to provide additional evidence validating the scalable evaluations' findings ..." (§3.1)” | exactEQ confidence: high | OpenAI Preparedness Framework v2 |
| Google DeepMindShadow mapping Frontier Safety Framework v3.1 | “"Early Warning Evaluations: are evaluations which measure the dangerous capabilities of a model. They specifically target the threats and risk scenarios identified through our threat modeling to determine a model's proximity to a CCL or TCL" (glossary).” | exactEQ confidence: high | Google DeepMind Frontier Safety Framework v3.1 |
| xAIShadow mapping xAI Frontier AI Framework (30 Jun 2026) | “Model evaluations: "state-of-the-art model evaluations relevant to the systemic risk to assess the model's capabilities, propensities, affordances, and/or effects, which may include Q&A sets, task-based evaluations, benchmarks, red-teaming and other methods of adversarial testing, human uplift studies, model organisms, simulations, and/or proxy evaluations for classified materials" (s.2.2(2)).” This row cites xAI's Frontier AI Framework (30 Jun 2026), whose PDF metadata title reads "Privileged/Confidential DRAFT working FRAMEWORK DOC"; no xAI statement settling draft or final status was found, and the cited URL returned HTTP 404 on 13 Nov 2026. Treat as provisional. | exactEQ confidence: medium | xAI Frontier AI Framework (30 Jun 2026) |
| xAIShadow mapping Grok 4.6 model card | “"safety-threshold evaluations"” | exactEQ confidence: high | Grok 4.6 model card |
| MetaShadow mapping Meta Advanced AI Scaling Framework v2 | “"Evaluation(s) refers to the assessments we do to understand capabilities and performance. We use this term to describe automated and human evaluations that assess capabilities, as well as evaluations to assess potential for misuse, such as red teaming and uplift studies." (Appendix I, p.43)” | exactEQ confidence: high | Meta Advanced AI Scaling Framework v2 |
| EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter | “Measure 3.2: "Signatories will conduct at least state-of-the-art model evaluations in the modalities relevant to the systemic risk to assess the model's capabilities, propensities, affordances, and/or effects", including "open-ended testing ... with a view to identifying unexpected behaviours, capability boundaries, or emergent properties"; Appendix 3.1 requires "internal validity", "external validity" and "reproducibility".” | exactEQ confidence: high | EU GPAI Code of Practice, Safety and Security Chapter |
| EU / xAI (cross-reference)Shadow mapping EU CoP Appendix 3 example methods, echoed in xAI FAIF s.2.2(2) | “The example methods in the EU Code of Practice (Measure 3.2 / Appendix 3) are echoed almost verbatim in xAI s.2.2(2): Q&A sets, task-based evaluations, benchmarks, red-teaming, human uplift studies, model organisms, simulations, proxy evaluations for classified materials.” This document's PDF metadata /Title reads "Privileged/Confidential DRAFT working FRAMEWORK DOC"; no xAI statement was found disambiguating draft vs. final status. Treat as provisional. | exactEQ confidence: medium | xAI Frontier AI Framework (30 Jun 2026) |
| California SB 53Shadow mapping California SB 53 | “Large developers summarise "(A) Assessments of catastrophic risks from the frontier model conducted pursuant to the large frontier developer's frontier AI framework" (22757.12(c)(2)).” | broadBR confidence: medium | California SB 53 |
| US Government (NIST CAISI)Shadow mapping NIST CAISI bulletin | “CAISI "will conduct pre-deployment evaluations and targeted research" (para. 1) — undefined.” | noneDU confidence: medium | NIST CAISI bulletin |
| Frontier Model ForumShadow mapping FMF Third-Party Assessments technical report | “"Capability Assessments : the process begins with frontier capability assessments that evaluate whether a model crosses any enabling capability thresholds or outcomes-based thresholds" (1.2).” | closeCL confidence: medium | Frontier Model Forum, Third-Party Assessments |
| MicrosoftShadow mapping Microsoft Frontier Governance Framework | “"Capability evaluation: This involves robust evaluation of whether a model possesses tracked capabilities at high or critical levels, including through adversarial testing and systematic measurement using state-of-the-art methods. Evaluations are documented in a consistent fashion setting out the assumptions regarding the model, capability being evaluated, the method used, and evaluation results." (Deeper capability assessment, p.6)” | exactEQ confidence: high | Microsoft Frontier Governance Framework |
What do these codes mean?
- exact
- The source term is equivalent to this element
- close
- The source term is close but not equivalent to this element
- broad
- The source term is broader than this element
- narrow
- The source term is narrower than this element
- none
- No mapping claim — used for false-friend and declared-but-undefined rows
- EQ
- Equivalent
- CL
- Close
- BR
- Source is broader than the element
- NR
- Source is narrower than the element
- FF
- False friend — same or similar label, different meaning
- DU
- Declared but undefined by the source
- UV
- Unverified
Related, not mapped
Pointers that are not crosswalk claims
These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.
- METR
"Full Capability Elicitation During Evaluations"; "Timing and Frequency of Evaluations" — named as process elements, not an evaluation definition (not a mapping).
METR







