Direct comparison
Reproducibility vs. Replicability
NASEM's 2019 definitions: reproducibility means same data + same code give the same result; replicability means new, independent data confirms it.
Side-by-side comparison
| Dimension | Reproducibility | Replicability |
|---|---|---|
| NASEM 2019 definition | Consistent results from the same input data, code, and computational steps | Consistent results across independent studies, each collecting its own new data |
| What is held constant | Data and analysis code (both reused as-is) | Method/study design (data is newly collected each time) |
| What is being tested | Whether the reported analysis was executed and reported correctly | Whether the underlying scientific finding holds up in new evidence |
| Also called | Computational reproducibility | Replication; sometimes 'generalizability' when the new study varies population/setting |
| How it's assessed | Re-execute the deposited code against the deposited data; results should match | Run an independent new study; compare effect direction/size, not exact values |
| Typical failure cause | Missing data/code, undocumented software versions, broken pipeline | Original finding was a false positive, context-dependent, or underpowered |
| Primary institutional lever | Data management plans, code deposit, repository/metadata standards | Study funding, pre-registration, publication-bias reform |
| Field-usage consistency | Some fields (parts of CS/engineering) have historically used this closer to NASEM's 'replicability' | Same inconsistency runs in reverse in those fields |
| Relation to open science/RDM | Directly supported by FAIR data, code sharing, persistent identifiers | Supported indirectly, via pre-registration and full methods transparency |
Common questions
FAQ
Is reproducibility the same as replicability?+
No. Under the now-standard NASEM (2019) usage, reproducibility means an independent analyst gets the same result from the same data and code; replicability means an independent new study, with its own newly collected data, reaches a consistent finding. A result can be reproducible but fail to replicate.
Which definition is 'correct'?+
There is no single universally enforced convention, but the NASEM 2019 report's definitions are the most widely cited current reference point and are increasingly the default in funder and journal guidance. Older documents, and some fields such as parts of computer science, have historically used the terms closer to the opposite way, so check how a specific source defines its terms rather than assuming.
Can a study be reproducible but not replicable?+
Yes, and this is common. A reproducible-but-non-replicating result means the original analysis was executed and reported correctly on its own data, but the underlying effect did not hold up when independently tested with new data — which can happen for reasons ranging from a false-positive original finding to genuine context-dependence.
Does depositing data and code guarantee reproducibility?+
It's necessary but not sufficient. Deposit removes the most common practical barriers (missing data, undocumented steps), but full computational reproducibility also depends on capturing the software environment and workflow, which is why containerization and workflow-documentation practices matter alongside raw data/code deposit.
Who is responsible for replicability — the original authors or the field?+
Replicability is generally treated as a property of the accumulated evidence on a question, established through independent replication attempts by other research groups, rather than something a single author or study can unilaterally guarantee. Institutions and funders can support replication (funding tracks, publication-bias reform) but cannot mandate that a specific finding will replicate.
