Examples
Worked examples
- Is an instance
A researcher tries three different outlier-exclusion rules and reports only the one producing a significant result, without disclosing the other two were tried.
- Is an instance
A pre-registered study fixes its primary outcome measure and exclusion criteria before data collection, closing off that particular degree of freedom in advance.
Counter-examples
Looks similar, but isn't
- Not an instance
Disclosing every analysis attempted, including the ones that didn’t reach significance, is not itself an exercise of undisclosed researcher degrees of freedom -- the harm comes from the flexibility being hidden, not from flexibility existing at all.
Editorial commentary
Researcher degrees of freedom are the many defensible decision points in designing, running, analysing, and reporting a study — which observations to exclude, which covariates to include, which of several plausible outcome measures to report, when to stop collecting data, which subgroup to examine — each individually reasonable, that collectively give a researcher far more flexibility to reach a significant result than a single pre-specified analysis would allow. Simmons, Nelson and Simonsohn coined the term in 2011 to explain why studies that looked methodologically unremarkable could nonetheless produce false-positive rates well above the nominal 5% threshold, once every combination of those defensible choices is available to be tried.
What this term is not
This is a research-integrity and practice concept, not a statistics tutorial: the point is not how to compute an estimate under a given specification, but why having many undisclosed opportunities to choose a specification after seeing the data undermines the nominal error rate of whichever one gets reported. Researcher degrees of freedom are the underlying substrate that other documented practices act on: they are what p-hacking exploits during analysis, what the garden of forking paths describes as a structural feature of any analysis with branching choices, and what a multiverse analysis exposes by running every branch rather than one.
How practice now constrains them
The structural response is to fix the analytic choices before they can be influenced by the data. Pre-registration and a pre-analysis plan commit a study’s hypotheses, exclusion criteria, and primary analysis in advance, removing the researcher’s ability to select among degrees of freedom after seeing results. Registered Reports go further, having a journal review and provisionally accept the design and analysis plan before any results exist. Many journals now also require disclosure of all analyses attempted, not only those that were ultimately reported, and a growing number ask authors to state explicitly which parts of the reported analysis were planned versus exploratory — treating undisclosed researcher degrees of freedom as a transparency failure to be reported around, not eliminated outright, since some flexibility in real research is unavoidable and even legitimate when disclosed.
References
- Simmons, Nelson, Simonsohn, ‘False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant’ (Psychological Science, 2011); Wicherts et al., ‘Degrees of freedom in planning, running, analyzing, and reporting psychological studies’ (Frontiers in Psychology, 2016).
Also known as
analytic degrees of freedom
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="Researcher degrees of freedom"
vocab-term-identifier="https://casrai.org/dictionary/term/researcher-degrees-of-freedom" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"@id": "https://casrai.org/dictionary/term/researcher-degrees-of-freedom",
"name": "Researcher degrees of freedom",
"identifier": "https://casrai.org/dictionary/term/researcher-degrees-of-freedom",
"description": "The many individually defensible decision points in designing, running, analysing and reporting a study -- exclusions, covariates, outcome measures, stopping rules -- that collectively let a researcher reach a significant result far more easily than one pre-specified analysis would allow.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/reproducibility#set",
"url": "https://casrai.org/dictionary/term/researcher-degrees-of-freedom",
"alternateName": [
"analytic degrees of freedom"
],
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"@id": "https://casrai.org/#organization"
},
"author": {
"@id": "https://casrai.org/#editorial-team"
},
"datePublished": "2026-05-21T02:22:50",
"dateModified": "2026-09-05T14:24:22",
"inLanguage": "en-GB",
"isAccessibleForFree": true
}
