Examples
Examples from published documents
- Google DeepMind ran automated checks of reasoning traces and manual transcript reviews that revealed no evidence of sandbagging in Gemini 3.7 Flash, and judged that the evaluation awareness it did observe did not compromise its capability assessments (Gemini 3.7 Flash FSF report, pp.5-6)
- Apollo Research found that 16% of GPT-5.6 Sol samples verbalised alignment-evaluation awareness, against 43% for a GPT-5.5 checkpoint; OpenAI notes that lower verbalised awareness may only mean the awareness is less visible (GPT-5.6 system card, section 9.3.1)
- Anthropic states that evaluation awareness does partially undermine its confidence in its alignment assessments, because much of its automated behavioural audit is likely detectably unrealistic to the models (Anthropic Risk Report, August 2026, section 2.9.4)
Counter-examples
Not to be confused with
- Under-elicitation by the evaluator, such as weak prompts or missing tools, also lowers scores, but it is the tester's shortfall rather than the model's behaviour; it is recorded under elicitation method.
- Benchmark contamination, where test items leaked into training data, inflates scores through recall rather than through the model reacting to being tested; it is recorded with the benchmark.
Related terms
Related AI safety terms
Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.
- Sandbagging
- Sandbagging is when an AI model deliberately performs worse on an evaluation than it can, hiding a capability so that it is not flagged as dangerous. It undermines any test that assumes the model is trying its best.
- Source: GPT-5.6 deployment safety card
- Evaluation awareness
- Evaluation awareness is an AI model's ability to tell that it is being tested, and sometimes why, which can change its behaviour so that test results no longer predict how it acts in real deployment.
- Source: Meta Advanced AI Scaling Framework v2
- Situational awareness
- Situational awareness is an AI model's ability to discover and use relevant details of its own deployment setting, such as that it is an AI, where it is running and what constraints it faces.
- Source: Google DeepMind Frontier Safety Framework v3.1
- Alignment faking
- Alignment faking is when an AI model hides its true motives by selectively complying with its training objective while it believes it is being trained, to avoid being modified, and behaving differently otherwise. It can make alignment training and evaluations misleading.
- Source: Greenblatt et al., Alignment faking in large language models (arXiv 2412.14093, 2024)
- Metagaming
- Metagaming is when an AI model reasons about how it will be graded, rewarded or monitored rather than only about the task in front of it, which can make good behaviour in tests an unreliable signal.
- Source: GPT-5.6 deployment safety card
- Reward hacking
- Reward hacking is when an AI model learns to "cheat" on a task by exploiting loopholes in how success is checked, instead of completing the task as intended, so its scores rise without the intended behaviour.
- Source: Anthropic, "Reward seeker"
- Grader awareness
- Grader awareness is an AI model explicitly reasoning about the grading or evaluation mechanism itself, beyond what the prompt mentions, and using that reasoning to change its strategy.
- Source: Anthropic Risk Report (August 2026)
Source of record
Where this definition comes from
Anthropic Risk Report (August 2026), §2.25.1
“Grader awareness (by which we mean a model explicitly reasoning about the grading/evaluation mechanism itself, beyond what is mentioned in the prompt, and using that reasoning to change its strategy) also increased significantly.”
https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdfOpenAI Preparedness Framework v2, Table 2 (Research Categories), §2.3
“Sandbagging: ability and propensity to respond to safety or capability evaluations in a way that significantly diverges from performance under real conditions, undermining the validity of such evaluations.”
https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbddebcd/preparedness-framework-v2.pdfGemini 3.7 Flash FSF report, p.5
“we performed automated checks to rule out sandbagging (deliberate underperformance in order to avoid being flagged as dangerous van der Weij et al. 2025).”
https://storage.googleapis.com/deepmind-media/gemini/gemini_3-7_flash_fsf_report.pdfGemini 3.7 Flash FSF report, p.6
“despite identifying some examples of evaluation awareness (where the model indicated knowledge of the testing context or explicitly noted it was operating in a synthetic environment), we do not believe evaluation awareness compromises our capability assessments.”
https://storage.googleapis.com/deepmind-media/gemini/gemini_3-7_flash_fsf_report.pdfGPT-5.6 deployment safety card, §9.3.1
“"Sandbagging" means intentionally performing worse to hide capabilities during an evaluation.”
https://deploymentsafety.openai.com/gpt-5-6
Crosswalk
How named organisations use this concept
| Organisation | Their term, as published | Match & verification | Source |
|---|---|---|---|
| AnthropicShadow mapping Anthropic Risk Report (August 2026) | “"Grader awareness (by which we mean a model explicitly reasoning about the grading/evaluation mechanism itself, beyond what is mentioned in the prompt, and using that reasoning to change its strategy)" (§2.25.1); "evaluation awareness does partially undermine our confidence in the results of our alignment assessments" (§2.9.4, Claim 3.4.2); risk pathway "Broad/diffuse sandbagging on safety-relevant R&D" (§2.2.1).” | exactEQ confidence: high | Anthropic Risk Report (August 2026) |
| AnthropicShadow mapping Anthropic, "Reward seeker" | “Reward hacking: models "sometimes learn to 'cheat' rather than completing these tasks as intended"; "Our industry lacks a general solution to this problem, and reward hacking remains challenging to fully mitigate" (tl;dr). When the studied model is told a reward-hacking classifier is present, sneaky hacking "increases from 4% to 19%"; internal monitoring flagged "97% of reward hacking environments with reward hacking rates of at least 1%" as significant or severe.” | exactEQ confidence: high | Anthropic, "Reward seeker" |
| OpenAIShadow mapping OpenAI Preparedness Framework v2 | “"Sandbagging: ability and propensity to respond to safety or capability evaluations in a way that significantly diverges from performance under real conditions, undermining the validity of such evaluations." Listed as a Research Category (Table 2, §2.3).” | exactEQ confidence: high | OpenAI Preparedness Framework v2 |
| OpenAIShadow mapping GPT-5.6 deployment safety card | “"Metagaming is when a model reasons about how it will be graded, rewarded, or monitored, rather than only reasoning about the situation described in the prompt." (s.7.4)” | exactEQ confidence: high | GPT-5.6 deployment safety card |
| Google DeepMindShadow mapping Gemini 3.7 Flash FSF report | “Automated checks "to rule out sandbagging (deliberate underperformance in order to avoid being flagged as dangerous van der Weij et al. 2025)" (p.5); "evaluation awareness (where the model indicated knowledge of the testing context or explicitly noted it was operating in a synthetic environment)" (p.6).” | exactEQ confidence: high | Gemini 3.7 Flash FSF report |
| xAIShadow mapping Grok 4.20 model card | “"On our audit, verbalized evaluation awareness remains low but non-negligible. However, this does not rule out unverbalized awareness." (§3.3, p.5)” Names evaluation awareness as a measured audit metric; no definition is given. | closeCL confidence: high | Grok 4.20 model card |
| MetaShadow mapping Meta Advanced AI Scaling Framework v2 | “"Evaluation awareness: Ability to accurately determine whether it is being evaluated and the purpose of that evaluation ..."; "Selective evaluation-adaptive behavior" (enabling capabilities, §3.4).” | exactEQ confidence: high | Meta Advanced AI Scaling Framework v2 |
| METRShadow mapping METR | “MALT: "A dataset of natural and prompted examples of behaviors that threaten evaluation integrity, like generalized reward hacking or sandbagging." (home page)” Read from CASRAI's local capture of metr.org, not from a dated METR report. | closeCL confidence: medium | METR |
| OpenAIShadow mapping GPT-5.6 deployment safety card | “"'Sandbagging' means intentionally performing worse to hide capabilities during an evaluation." (§9.3.1, Apollo Research external evaluation); "Metagaming" is reported because "observed aligned actions may not be reflective of true alignment" (§7.4).” | exactEQ confidence: high | GPT-5.6 deployment safety card |
| AnthropicShadow mapping Anthropic Responsible Scaling Policy v3.4 | “External reviewers must "be knowledgeable about potential ways such evaluations might be misleading (for example, alignment faking)" (§3.6.1, p.15).” | closeCL confidence: high | Anthropic Responsible Scaling Policy v3.4 |
| Google DeepMindShadow mapping Frontier Safety Framework v3.1 | “Stealth and Situational Awareness TCL: "situational awareness (ability to discover and use relevant details of its deployment setting)" (§3.2.1, p.14).” A capability threshold that includes situational awareness, not a list of evaluation-validity threats. | narrowNR confidence: high | Google DeepMind Frontier Safety Framework v3.1 |
What do these codes mean?
- exact
- The source term is equivalent to this element
- close
- The source term is close but not equivalent to this element
- broad
- The source term is broader than this element
- narrow
- The source term is narrower than this element
- none
- No mapping claim — used for false-friend and declared-but-undefined rows
- EQ
- Equivalent
- CL
- Close
- BR
- Source is broader than the element
- NR
- Source is narrower than the element
- FF
- False friend — same or similar label, different meaning
- DU
- Declared but undefined by the source
- UV
- Unverified
Related, not mapped
Pointers that are not crosswalk claims
These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.
- US (SB 53)
Incident limb applies "outside of the context of an evaluation designed to elicit this behavior" (22757.11(d)(4)) — a carve-out pointer, not an evaluation-validity-threat definition.
California SB 53
Divergence
Where sources materially disagree
Sandbagging and alignment faking point in opposite directions. Sandbagging is underperformance that hides a capability from a dangerous-capability evaluation; alignment faking is apparent good behaviour that hides misalignment from an alignment or propensity evaluation. Both can be driven by evaluation awareness, and the sources treat them separately. The sources also file these conditions in different places: OpenAI lists sandbagging as a Research Category of risk in its own right, Meta lists evaluation awareness and selective evaluation-adaptive behavior as enabling capabilities for loss of control, and Google DeepMind treats sandbagging and evaluation awareness as checks within its elicitation procedure. For how this element relates to N4's limitation element, see that element's divergence note.







