Skip to main content
v2026.11,986 entries · CC-BY 4.0
CASRAIAsk CASRAI · AI assistant for researchAsk a research question. Answers come with their sources.Funder rules, compliance, methods and publishing. Answers link the sources they rest on.
NIKOLAI elementN5 · Evidence and evaluationsProposednikolai-v0.3

Elicitation method

Definition

An elicitation method is a technique used during an AI evaluation to draw out the most a model can do, such as fine-tuning, prompt optimisation, stronger scaffolding and tools, or removing refusals, so the result reflects its full capability rather than its default behaviour.

Also called: Capability elicitation, Model elicitation, Full capability elicitation, Elicitation

NIKOLAI's proposed operational definition

NIKOLAI proposal: the techniques and conditions used to draw out a model's maximum capability during an evaluation, recorded as a property of the evaluation run, together with the declared interpretation of the result (lower bound vs. ceiling).

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

  • OpenAI tests with the highest-capability tier of system settings, a model variant with a negligible rate of safety-based refusals and the best available scaffolds, and treats any one-time capability elicitation as a lower bound rather than a ceiling (OpenAI Preparedness Framework v2, section 3.1)
  • Anthropic reports that fine-tuning and prompt optimisation raised Mythos 5's SHADE-Arena score from 0% to 3.8% and 9.2% respectively, while stating that it has not provided clear evidence that this elicitation is sufficiently strong (Anthropic Risk Report, August 2026, sections 2.7 and 2.16.1)
  • Meta gives agents a task-optimised scaffold and a generous token budget, up to the maximum context length or the point at which performance plateaus, and reviews transcripts for easily fixable agent failures (Meta Advanced AI Scaling Framework v2, section 4.2)

Counter-examples

Not to be confused with

  • Removing production safeguards, as in xAI's "unrestricted configuration", is one input to elicitation rather than the whole method: on its own it adds no fine-tuning, scaffolding or extra compute.
  • Scores published at default settings show typical behaviour, not an elicited maximum; they cannot be used to rule out a dangerous capability.

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

Capability elicitation
Capability elicitation is the deliberate effort to make an AI model show its full abilities during testing, using techniques such as fine-tuning, prompt optimisation, multi-agent setups and tool access, so that dangerous capabilities are not underestimated.
Source: Microsoft Frontier Governance Framework
Under-elicitation
Under-elicitation is a failure to draw out a model's real capability during an evaluation, for example through weak prompting, missing tools or unhandled refusals, which makes the model look less capable than it is.
Source: EU GPAI Code of Practice, Safety and Security Chapter
Helpful-only model
A helpful-only model is a variant of a production AI model trained not to refuse potentially harmful requests, used so that dangerous-capability evaluations and red-teaming measure what the model can do rather than what it declines to do.
Source: Anthropic Risk Report (August 2026)
Fine-tuning for elicitation
Fine-tuning for elicitation is extra training of a model on task-relevant data before testing, to estimate the upper bound of what someone with the weights or a fine-tuning API could make it do.
Source: Meta Advanced AI Scaling Framework v2
Scaffolding
Scaffolding, also called an agent harness, is the software around a model in an agentic evaluation, such as tools, memory and multi-step or multi-agent loops, that lets it act on tasks; weak scaffolding can understate capability.
Source: Gemini 3.7 Flash FSF report
Lower bound (elicitation)
A lower bound, in elicitation, is the reading of a one-time capability test as the least a model can do, not the most, because better scaffolds and techniques found later may draw out more.
Source: OpenAI Preparedness Framework v2

Source of record

Where this definition comes from

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.
OrganisationTheir term, as publishedMatch & verificationSource
AnthropicShadow mapping
Anthropic Risk Report (August 2026)
“Elicitation experiments "using fine-tuning and prompt optimization, which increase Mythos 5's SHADE-Arena performance from 0% to 3.8% and 9.2% respectively" (§2.7); "we have not provided clear evidence that this elicitation is sufficiently strong" (§2.16.1).”closeCL
confidence: high
Anthropic Risk Report (August 2026)
OpenAIShadow mapping
OpenAI Preparedness Framework v2
“Elicitation aims at "the high end of expected elicitation by threat actors" using highest-capability settings, a variant with a "negligible rate of safety-based refusals", best scaffolds, and fine-tuning if weights will be released; "we regard any one-time capability elicitation in a frontier model as a lower bound, rather than a ceiling" (§3.1).”exactEQ
confidence: high
OpenAI Preparedness Framework v2
OpenAIShadow mapping
GPT-5.6 deployment safety card
“SecureBio evaluated "a railfree version of GPT-5.6 Sol" with filters disabled (s.9.1.1.8).”exactEQ
confidence: high
GPT-5.6 deployment safety card
Google DeepMindShadow mapping
Gemini 3.7 Flash FSF report
“"We use a number of elicitation methods to surface a model's latent capabilities and propensities, and ensure that our risk assessment is based on the model's absolute potential rather than its default behavior"; agentic domains get "state-of-the-art scaffolding and affordances", and "we actively take steps to minimize the risk of under-elicitation" (Elicitation, p.5).”exactEQ
confidence: high
Gemini 3.7 Flash FSF report
Google DeepMindShadow mapping
Frontier Safety Framework v3.1
“evaluations use "appropriate scaffolding, inference compute, and other augmentations" (s.1.3.2).”exactEQ
confidence: high
Google DeepMind Frontier Safety Framework v3.1
xAIShadow mapping
Grok 4.6 model card
“"Unrestricted configuration": capability testing "without the safeguards we use in production, which would otherwise mask the model's full capability" (§7, §7.1) — safeguard removal only, narrower than the general elicitation-method concept.”narrowNR
confidence: medium
Grok 4.6 model card
MetaShadow mapping
Meta Advanced AI Scaling Framework v2
“Helpful-only fine-tuning "(i.e., refusal-free)", domain-specific capability training for open release, task-optimised scaffolds; "Agents will be provided with a generous token budget, up to the maximum context length or to the point at which performance plateaus" (§4.2).”exactEQ
confidence: high
Meta Advanced AI Scaling Framework v2
METRShadow mapping
METR
“"Full Capability Elicitation During Evaluations: Intentions to perform model evaluations in a way that does not underestimate the full capabilities of the model."”exactEQ
confidence: high
METR
EUShadow mapping
EU GPAI Code of Practice, Safety and Security Chapter
“Appendix 3.2 "Model elicitation": evaluations must use "at least a state-of-the-art level of model elicitation that elicits the model's capabilities, propensities, affordances, and/or effects", techniques that "(1) minimise the risk of under-elicitation; and (2) minimise the risk of model deception during model evaluations (e.g. sandbagging)"; Appendix 3.4 requires access to "the model version(s) with the fewest safety mitigations implemented (such as a helpful-only model version, if it exists)" and an indicative time floor of "at least 20 business days ... for most systemic risks and model evaluation methods."”exactEQ
confidence: high
EU GPAI Code of Practice, Safety and Security Chapter
MicrosoftShadow mapping
Microsoft Frontier Governance Framework
“"Capability elicitation: Evaluations include concerted efforts at capability elicitation, i.e., applying capability enhancing techniques to advance understanding of a model's full capabilities"; "Resources applied to elicitation should be extrapolated out to those available to actors in threat models relevant to each tracked high-risk capability" (Deeper capability assessment, p.6).”exactEQ
confidence: high
Microsoft Frontier Governance Framework
G42Shadow mapping
G42 Frontier Safety Framework
“In-depth evaluations "will incorporate capability elicitation", "techniques such as prompt engineering, fine-tuning, and agentic tool usage", to "optimize performance, overcome model refusals, and avoid underestimating model capabilities" (Frontier Capability Threshold evaluations, PDF p.4).”exactEQ
confidence: high
G42 Frontier Safety Framework
What do these codes mean?
exact
The source term is equivalent to this element
close
The source term is close but not equivalent to this element
broad
The source term is broader than this element
narrow
The source term is narrower than this element
none
No mapping claim — used for false-friend and declared-but-undefined rows
EQ
Equivalent
CL
Close
BR
Source is broader than the element
NR
Source is narrower than the element
FF
False friend — same or similar label, different meaning
DU
Declared but undefined by the source
UV
Unverified

Related, not mapped

Pointers that are not crosswalk claims

These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.

  • UK Government / AISI

    UK AISI "Grey box access": "access to chain-of-thought of the safety reasoning monitor, exact policy wording, and real-time feedback on classifier labels that would not be accessible to real-world attackers" — an access-tier pointer, not itself an elicitation-method definition.

    GPT-5.6 deployment safety card
  • Frontier Model Forum

    Access item "version with limited safety training" — a pointer, not a full elicitation-method definition.

    Frontier Model Forum, Third-Party Assessments

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →