Examples
Worked examples
- Is an instance
A regression result presented with and without country fixed effects.
- Is an instance
A trial outcome reanalysed under per-protocol and intention-to-treat populations.
Counter-examples
Looks similar, but isn't
- Not an instance
A new experiment on a new sample (replication).
- Not an instance
Re-running the same code (reproducibility).
Editorial commentary
A robustness check is a supplementary analysis, run alongside a study’s headline result, that varies one or more analytical choices — the estimator, the sample, the set of control variables, the functional form of the outcome — to show that the main conclusion does not depend on one specific, arbitrary set of choices. A result that survives a battery of reasonable alternative specifications is more credible than one reported under only a single, unstated set of decisions; a result that flips sign or significance under small, defensible changes is telling reviewers something important about how fragile the original estimate was.
The credibility of a robustness check depends heavily on when the check was chosen. A robustness check specified in advance — as part of a pre-analysis plan or a pre-registration — carries far more evidential weight than one selected after the fact, because a post-hoc robustness check can itself be quietly chosen from among many that were tried, which is exactly the kind of hidden multiplicity described by the garden of forking paths. This is why some journals and reviewers now expect authors to state, explicitly, which robustness checks were planned and which were exploratory.
How it differs from a specification curve or multiverse analysis
A robustness check is typically a handful of discrete, individually chosen alternative analyses reported alongside the main result. A specification curve and a multiverse analysis are more systematic and exhaustive: rather than a researcher hand-picking a small number of alternatives, they enumerate a large, pre-defined set of defensible specifications and report the full distribution of results across all of them. A robustness check answers “does the result survive these specific alternatives I chose to show you”; a specification curve or multiverse answers “what does the result look like across essentially every defensible way of running this analysis.” Neither is a stand-in for a genuine replication on new data, nor for computational reproducibility — re-running the same code on the same data.
References
- Leamer, 'Specification Searches: Ad Hoc Inference with Nonexperimental Data' (Wiley, 1978).
Also known as
sensitivity analysis
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="Robustness check"
vocab-term-identifier="https://casrai.org/dictionary/term/robustness-check" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"@id": "https://casrai.org/dictionary/term/robustness-check",
"name": "Robustness check",
"identifier": "https://casrai.org/dictionary/term/robustness-check",
"description": "An additional analysis, supplementary to the headline result, that varies one or more analytical choices in order to demonstrate that the main conclusion is not artefactual to those choices.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/reproducibility#set",
"url": "https://casrai.org/dictionary/term/robustness-check",
"sameAs": [
"sensitivity analysis"
],
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"@id": "https://casrai.org/#organization"
},
"author": {
"@id": "https://casrai.org/#editorial-team"
},
"datePublished": "2026-05-21T02:22:49",
"dateModified": "2026-08-22T15:41:27",
"inLanguage": "en-GB",
"isAccessibleForFree": true
}
