Uplift Study in AI Safety: Definition

Uplift study

Definition

An uplift study is an experiment that measures how much access to an AI model improves people's ability to complete a task, usually a risk-relevant one, by comparing them with a control group that has only existing resources such as the internet.

Also called: Uplift trial, Human uplift study, Wet-lab uplift study

NIKOLAI's proposed operational definition

NIKOLAI proposal: a controlled study measuring how much access to an AI model changes people's ability to complete a risk-relevant task compared with a control group using existing resources, recorded with participant expertise, the control condition, the task, sample size and duration, the model version and safeguards in place, the primary endpoint and the result. The risk concept of uplift itself is defined in the N2 element 'uplift'; this element covers the study design that measures it.

The operational definition is CASRAI's own proposal, not a definition any named organisation has agreed to. See what NIKOLAI is and is not, or browse the AI safety glossary.

Examples

Examples from published documents

Hong et al. (2026) ran a randomised controlled trial with 153 novices over 8 weeks on a viral reverse-genetics workflow and found no significant difference in the primary endpoint, with numerically higher success on 4 of 5 tasks (Gemini 3.7 Flash FSF report, p.12)
Anthropic's reading of the same trial adds that a post-hoc analysis suggested about 1.42x uplift on a typical task, with large error bars, and that the study was underpowered (Anthropic Risk Report, August 2026, section 4.4.2.1)
Anthropic ran a catastrophic biological scenario uplift trial with five three-person teams whose outputs were graded by external domain experts against a standardized uplift rubric (Anthropic Risk Report, August 2026, section 4.4.3)

Counter-examples

Not to be confused with

Expert red teaming that compares a model's answers with what is available on the web estimates uplift indirectly; without a control group of participants doing the task, it is not an uplift study.

Related terms

Related AI safety terms

Terms defined under this element that are not NIKOLAI elements themselves, each as the cited document uses it. Every term is also listed in the AI safety glossary.

Randomised controlled trial (uplift): A randomised controlled trial, in AI safety, randomly assigns participants to work with or without AI access on the same tasks, so that differences in success can be attributed to the AI rather than to the participants.; Source: Gemini 3.7 Flash FSF report

Source of record

Where this definition comes from

Meta Advanced AI Scaling Framework v2, Appendix I, p.43
“Uplift studies are experiments that assess the extent to which access to Frontier AI increases a person or group's ability to complete a particular task or scenario in comparison to a control group that only has access to existing resources, such as textbooks, the internet, and existing AI models.”
https://ai.meta.com/static-resource/Meta_Advanced-AI-Scaling-Framework-v2
Anthropic Risk Report (August 2026), §4.4.1
“Evaluations that compare success with and without AI assistance, e.g. uplift trials.”
https://www-cdn.anthropic.com/f61d49fa5596956a5dec75fea0e973bf6a6a8378/Redacted%20Risk%20Report%20August%202026%20.pdf
EU GPAI Code of Practice, Safety and Security Chapter, Measure 3.2
“Examples of model evaluation methods are: Q&A sets, task-based evaluations, benchmarks, red-teaming and other methods of adversarial testing, human uplift studies, model organisms, simulations, and/or proxy evaluations for classified materials.”
https://ec.europa.eu/newsroom/dae/redirection/document/118119

Crosswalk

How named organisations use this concept

Every row below is a shadow mapping. A shadow row is CASRAI's own reading of a published document. No lab, evaluator or regulator named on a shadow row has declared, endorsed, or been consulted on it. That changes only when an organisation files its own Mapping Declaration.

Organisation	Their term, as published	Match & verification	Source
MetaShadow mapping Meta Advanced AI Scaling Framework v2	“"Uplift studies are experiments that assess the extent to which access to Frontier AI increases a person or group's ability to complete a particular task or scenario in comparison to a control group that only has access to existing resources, such as textbooks, the internet, and existing AI models." (Appendix I, p.43)”	exactEQ confidence: high	Meta Advanced AI Scaling Framework v2
AnthropicShadow mapping Anthropic Risk Report (August 2026)	“Informative evidence includes "Evaluations that compare success with and without AI assistance, e.g. uplift trials" (§4.4.1); a "Catastrophic biological scenario uplift trial" with "five three-person teams" graded "against a standardized uplift rubric" (§4.4.3); and the Hong et al. randomized controlled trial on novice uplift (§4.4.2.1).”	closeCL confidence: high	Anthropic Risk Report (August 2026)
Google DeepMindShadow mapping Gemini 3.7 Flash FSF report	“"real world uplift studies offer important insight into the latter - translation of synthesized information into real-world improvement in laboratory performance"; Hong et al. (2026), "a randomized controlled trial (n=153) conducted over 8 weeks" (p.12).”	closeCL confidence: high	Gemini 3.7 Flash FSF report
EUShadow mapping EU GPAI Code of Practice, Safety and Security Chapter	“"human uplift studies" listed among example model evaluation methods (Measure 3.2).”	noneDU confidence: high	EU GPAI Code of Practice, Safety and Security Chapter
OpenAIShadow mapping GPT-5.6 deployment safety card	“"in addition to red-teaming, external testing and previous wet-lab uplift studies, we run four evaluations"; "Additional validation from wet-lab uplift studies may ultimately change this conclusion" (§9.1.1).”	noneDU confidence: high	GPT-5.6 deployment safety card

What do these codes mean?

exact: The source term is equivalent to this element
close: The source term is close but not equivalent to this element
broad: The source term is broader than this element
narrow: The source term is narrower than this element
none: No mapping claim — used for false-friend and declared-but-undefined rows
EQ: Equivalent
CL: Close
BR: Source is broader than the element
NR: Source is narrower than the element
FF: False friend — same or similar label, different meaning
DU: Declared but undefined by the source
UV: Unverified

Related, not mapped

Pointers that are not crosswalk claims

These sources mention this concept but do not define or map it clearly enough to count as a crosswalk row — noted here so the research is visible without overstating it as a mapping.

OpenAI
Deep dives may include "resource-intensive third party evaluations (e.g., bio wet lab studies, assessments by independent third party evaluators)" (§3.1) - a pointer to wet-lab studies, without the control-group design.
OpenAI Preparedness Framework v2

Divergence

Where sources materially disagree

Sources choose different control conditions. Meta's control group has access to existing resources including the internet and existing AI models, so it measures uplift over today's AI; Google DeepMind's CBRN evaluations compare against a web-only baseline; the Hong et al. trial cited by both Anthropic and Google DeepMind compared novices with and without mid-2025 frontier LLMs. Results from studies with different controls are not directly comparable.

Gap

The research did not find an agreed minimum sample size or statistical power for uplift studies; Anthropic notes that the Hong et al. trial had only 36% power to detect an odds ratio of 2.0 in success rates on the core task sequence.