Examples
Worked examples
- Is an instance
A 4,096-cell multiverse over 12 binary processing decisions in a developmental cohort study.
- Is an instance
A multiverse showing the headline effect is positive in 92% of specifications and significant in 71%.
Counter-examples
Looks similar, but isn't
- Not an instance
A single robustness check.
- Not an instance
A meta-analysis across multiple studies (different concept).
Editorial commentary
A multiverse analysis (Steegen, Tuerlinckx, Gelman & Vanpaemel, 2016) runs every reasonable combination of the data-processing and modelling decisions involved in a study — how outliers are handled, how a variable is coded, which covariates are included, which subsample is analysed — rather than committing to one processing pipeline and reporting only its result. Where a conventional paper reports one dataset processed one way, a multiverse analysis reports a “universe” of possible datasets and analyses, one for every combination of the reasonable choices the researchers identify, and summarises how the conclusion varies across that universe: for example, the proportion of specifications in which an effect is positive, or in which it crosses a conventional significance threshold.
The technique generalises both robustness checks and specification curves by varying multiple upstream decisions simultaneously rather than one at a time, and by exhausting the reasonable option space rather than selecting a handful of alternatives to report.
What it is not
A multiverse analysis is not a way of choosing the “correct” specification, and this page does not attempt to teach the statistical machinery of building one. It is a reporting and transparency practice, not an estimation method: its output is a distribution of results and an honest account of how much of the finding is a general pattern versus an artefact of specific, otherwise-invisible choices. It should not be confused with a meta-analysis, which combines results across separate studies rather than across processing choices applied to a single dataset, nor with the garden of forking paths itself, which is the underlying problem a multiverse analysis is a response to, not the technique.
For a research-integrity office, the relevant question when reviewing a paper that reports a multiverse analysis is whether the set of specifications was defined and justified in advance (ideally in a pre-registration or pre-analysis plan), since a multiverse that was itself hand-picked after seeing which combinations flattered the result would reintroduce the exact problem the method exists to solve.
References
- Steegen, Tuerlinckx, Gelman, Vanpaemel, 'Increasing transparency through a multiverse analysis' (Perspectives on Psychological Science, 2016).
Also known as
multiverse
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="Multiverse analysis"
vocab-term-identifier="https://casrai.org/dictionary/term/multiverse-analysis" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"@id": "https://casrai.org/dictionary/term/multiverse-analysis",
"name": "Multiverse analysis",
"identifier": "https://casrai.org/dictionary/term/multiverse-analysis",
"description": "An analytical approach in which all reasonable combinations of data-processing and modelling choices are executed, producing a distribution of results that displays the impact of researcher degrees of freedom on the conclusion.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/reproducibility#set",
"url": "https://casrai.org/dictionary/term/multiverse-analysis",
"alternateName": [
"multiverse"
],
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"@id": "https://casrai.org/#organization"
},
"author": {
"@id": "https://casrai.org/#editorial-team"
},
"datePublished": "2026-05-21T02:22:49",
"dateModified": "2026-09-05T14:24:07",
"inLanguage": "en-GB",
"isAccessibleForFree": true
}
