Results Reproducibility

Ask about Results reproducibility

Answers are drawn from this dictionary entry and the rest of the CASRAI corpus, with a link to every source.

Answers are AI-generated from CASRAI’s own published pages and can be wrong, so check the linked sources before relying on one; your question is logged without personal data — never sold, never used to train a third-party model — to show us what CASRAI is missing, so please do not type personal or confidential details. How we use this

Results reproducibility is the narrower sense in which a study’s reported quantitative results — point estimates, confidence intervals, p-values, effect sizes — can be recreated from the deposited data using the deposited analysis procedures. It answers “if I run what was deposited, do I get the published numbers?”, not “was the study reported in enough detail?” (that’s methods reproducibility) and not “would repeating the underlying experiment give the same observations?” (that’s empirical reproducibility).

Relationship to computational reproducibility

Results reproducibility and computational reproducibility are closely related and often used loosely as if interchangeable, but they emphasise different things. Computational reproducibility asks the narrower technical question — does the deposited code, run against the deposited data on a comparable environment, execute at all and produce the same output? Results reproducibility is the outcome-focused version — do the specific published numerical claims match? A pipeline can be computationally reproducible (it runs, it produces output) while failing results reproducibility if, say, a reporting error means the published table doesn’t match what the code actually outputs. In practice, most reproducibility audits check results reproducibility directly, because it’s the claim readers actually rely on.

What causes it to fail

Common failure points: an undocumented data-cleaning or exclusion step applied before analysis but not reported; a software version difference that changes a statistical routine’s default behaviour; a random seed that wasn’t fixed or recorded; or, less innocently, selective reporting of the analysis that produced the strongest result among several tried. Distinguishing an honest computational-environment mismatch from an undisclosed analytic choice is usually the actual work of a reproducibility check, not just re-running a script.

Sources

Goodman, Fanelli & Ioannidis, “What does research reproducibility mean?” Science Translational Medicine (2016).

Use in your systems

JATS XML <role> element

xml

<role vocab="credit"
      vocab-identifier="https://casrai.org/dictionary/"
      vocab-term="Results reproducibility"
      vocab-term-identifier="https://casrai.org/dictionary/term/results-reproducibility" />

Schema.org DefinedTerm (JSON-LD)

json

{
  "@context": "https://schema.org",
  "@type": "DefinedTerm",
  "@id": "https://casrai.org/dictionary/term/results-reproducibility",
  "name": "Results reproducibility",
  "identifier": "https://casrai.org/dictionary/term/results-reproducibility",
  "description": "The narrow sense in which a study's reported quantitative results can be recreated from the deposited data using the deposited analysis procedures.",
  "inDefinedTermSet": "https://casrai.org/dictionary/domain/reproducibility#set",
  "url": "https://casrai.org/dictionary/term/results-reproducibility",
  "sameAs": [
    "numerical reproducibility (results sense)"
  ],
  "license": "https://creativecommons.org/licenses/by/4.0/",
  "publisher": {
    "@id": "https://casrai.org/#organization"
  },
  "dateModified": "2026-08-22T14:39:13",
  "inLanguage": "en"
}

Results reproducibility