Examples
Examples from published documents
- Anthropic found that giving Mythos 5 a 900,000-token budget instead of 300,000 tokens on its CoBench R&D evaluation improved scores by only about 3 percentage points (Anthropic Risk Report, August 2026, section 3.4.3)
- The EU Code of Practice gives at least 20 business days as an appropriate evaluation period for most systemic risks and methods, and requires adequate compute budgets for long runs, parallel execution and re-runs (EU GPAI Code of Practice, Safety and Security Chapter, Appendix 3.4)
- Microsoft states that resources applied to elicitation should be extrapolated to those available to the threat actors relevant to each tracked high-risk capability (Microsoft Frontier Governance Framework, p.6)
Counter-examples
Not to be confused with
- The compute used to train a model is not an elicitation budget: it is fixed before testing, whereas the elicitation budget is what the evaluator spends at test time.
Related terms
Related AI safety terms
Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.
- Token budget
- A token budget is the maximum amount of text an AI model may read and generate during one evaluation attempt; Meta gives agents a budget up to the maximum context length or until performance plateaus.
- Source: Meta Advanced AI Scaling Framework v2
- Test-time compute
- Test-time compute is the computation an AI model spends while answering, for example by reasoning for longer or sampling more candidate answers; raising it can raise scores, so evaluators adjust it during elicitation.
- Source: EU GPAI Code of Practice, Safety and Security Chapter
- Attempt budget
- An attempt budget is the maximum number of tries an AI model is allowed on a task; under Google DeepMind's TAC score, one success within the allocated attempts counts the challenge as solved.
- Source: Gemini 3.7 Flash FSF report
Source of record
Where this definition comes from
Meta Advanced AI Scaling Framework v2, §4.2, p.27
“Agents will be provided with a generous token budget, up to the maximum context length or to the point at which performance plateaus.”
https://ai.meta.com/static-resource/Meta_Advanced-AI-Scaling-Framework-v2EU GPAI Code of Practice, Safety and Security Chapter, Appendix 3.4
“adequate compute budgets, including to allow for sufficiently long model evaluation runs, parallel execution, and re-runs”
https://ec.europa.eu/newsroom/dae/redirection/document/118119Microsoft Frontier Governance Framework, Deeper capability assessment, p.6
“Resources applied to elicitation should be extrapolated out to those available to actors in threat models relevant to each tracked high-risk capability.”
https://cdn-dynmedia-1.microsoft.com/is/content/microsoftcorp/microsoft/msc/documents/presentations/CSR/Frontier-Governance-Framework-Feb-2026.pdf
Crosswalk
How named organisations use this concept
| Organisation | Their term, as published | Match & verification | Source |
|---|---|---|---|
| MetaShadow mapping Meta Advanced AI Scaling Framework v2 | “"Agents will be provided with a generous token budget, up to the maximum context length or to the point at which performance plateaus." (§4.2, p.27)” | closeCL confidence: high | Meta Advanced AI Scaling Framework v2 |
| EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter | “Elicitation "such as by adapting test-time compute, rate limits, scaffolding, and tools" (Appendix 3.2); teams get "adequate compute budgets" and time, e.g. "a period of at least 20 business days is appropriate for most systemic risks and model evaluation methods" (Appendix 3.4).” | closeCL confidence: high | EU GPAI Code of Practice, Safety and Security Chapter |
| MicrosoftShadow mapping Microsoft Frontier Governance Framework | “"Resources applied to elicitation should be extrapolated out to those available to actors in threat models relevant to each tracked high-risk capability." (p.6)” | closeCL confidence: high | Microsoft Frontier Governance Framework |
| AnthropicShadow mapping Anthropic Risk Report (August 2026) | “CoBench results are "from the best-performing of a number of scaffolds used", and "giving Mythos 5 a 900,000 token budget instead of the 300,000 token budget used in the figure above only improves scores by about 3 percentage points" (§3.4.3).” | closeCL confidence: high | Anthropic Risk Report (August 2026) |
| Google DeepMindShadow mapping Frontier Safety Framework v3.1 | “Evaluations use "appropriate scaffolding, inference compute, and other augmentations" (§1.3.2, p.5).” | closeCL confidence: medium | Google DeepMind Frontier Safety Framework v3.1 |
| Google DeepMindShadow mapping Gemini 3.7 Flash FSF report | “TAC score requires "only one success per challenge within the maximum allocated attempt budget" (p.17).” Attempt budget only. | narrowNR confidence: high | Gemini 3.7 Flash FSF report |
| OpenAIShadow mapping GPT-5.6 deployment safety card | “Automated red-teaming for universal jailbreaks totalled "over 700,000 A100e GPU hours of compute effort" (§9.4.4).” Compute effort for a red-team exercise rather than for a capability evaluation. | narrowNR confidence: high | GPT-5.6 deployment safety card |
What do these codes mean?
- exact
- The source term is equivalent to this element
- close
- The source term is close but not equivalent to this element
- broad
- The source term is broader than this element
- narrow
- The source term is narrower than this element
- none
- No mapping claim — used for false-friend and declared-but-undefined rows
- EQ
- Equivalent
- CL
- Close
- BR
- Source is broader than the element
- NR
- Source is narrower than the element
- FF
- False friend — same or similar label, different meaning
- DU
- Declared but undefined by the source
- UV
- Unverified
Related, not mapped
Pointers that are not crosswalk claims
These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.
- OpenAI
Elicitation uses the highest-capability tier of system settings and "the best presently-available scaffolds" to "approximate the high end of expected elicitation by threat actors" (§3.1) - a qualitative budget anchor without numbers.
OpenAI Preparedness Framework v2
Divergence
Where sources materially disagree
Sources anchor the budget differently: Microsoft and the EU Code of Practice tie it to the resources of relevant threat actors, Meta to the point at which performance plateaus, Google DeepMind to an allocated attempt budget per challenge, and Anthropic reports a sensitivity check (tripling the token budget added about 3 percentage points)
Gap
The research did not find a requirement to report the budget as numbers alongside each result; most frameworks describe it qualitatively ('generous', 'adequate', 'appropriate')
