Examples
Worked examples
- Is an instance
A regression result presented with and without country fixed effects.
- Is an instance
A trial outcome reanalysed under per-protocol and intention-to-treat populations.
Counter-examples
Looks similar, but isn't
- Not an instance
A new experiment on a new sample (replication).
- Not an instance
Re-running the same code (reproducibility).
Editorial commentary
Robustness checks include alternative estimators, alternative samples, dropped covariates, transformed outcomes, and bootstrap resampling. They are most informative when pre-specified; ad hoc robustness checks chosen after seeing the main result can themselves be a form of garden-of-forking-paths.
References
- Leamer, 'Specification Searches: Ad Hoc Inference with Nonexperimental Data' (Wiley, 1978).
Also known as
sensitivity analysis
Machine-readable encodings
Use in your systems
xml
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="Robustness check"
vocab-term-identifier="https://casrai.org/dictionary/term/robustness-check" />json
{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"name": "Robustness check",
"identifier": "https://casrai.org/dictionary/term/robustness-check",
"description": "An additional analysis, supplementary to the headline result, that varies one or more analytical choices in order to demonstrate that the main conclusion is not artefactual to those choices.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/reproducibility-and-computational-research/",
"url": "https://casrai.org/dictionary/term/robustness-check",
"sameAs": [
"sensitivity analysis"
],
"license": "https://creativecommons.org/licenses/by/4.0/"
}
