Dictionary domain · Track D · Active

Reproducibility

Operational terminology for the practices, standards, and metadata that support reproducibility, replicability, and robustness of research outputs.

Implementation guide →Working group →Standards hub →Propose a term →

For implementers

Operational deployment checklist for the Reproducibility vocabulary: prerequisites, five deploy steps, integration notes for DSpace, Pure, OJS, Software Heritage, and OSF, plus the pitfalls that recur in the field.

View implementation checklist →

What this domain covers

The Reproducibility domain assembles the vocabulary that lets funders, journals, institutions, and researchers talk about the same things when they invoke “reproducible research”. Its remit spans six adjacent areas that are routinely conflated in practice.

Outcome distinctions — reproducibility, replicability, and robustness as three formally separate constructs, following the National Academies framing (Goodman, Fanelli, and Ioannidis 2016; NASEM 2019).
Reporting standards — the TOP, ARRIVE, CONSORT, PRISMA, and STROBE families curated by the EQUATOR Network and the Center for Open Science.
Methods transparency — preregistration, registered reports, protocol publication, and the data availability statement as the now-canonical disclosure surface.
Outputs — datasets, code, materials, and computational environments captured as containers, Jupyter notebooks, and workflow artefacts under FAIR4RS-compatible practice.
Assessment — replicability rate, robustness checks, multiverse analysis, and specification curve analysis as the techniques that quantify how stable a finding actually is.
Governance — the Reproducibility Crisis discourse and its intersection with the research-integrity domain, including the boundary between “non-replication” and “misconduct”.

19 entries

Terms in this domain

Each entry has an operational definition, worked examples, related terms, and stable URIs. Pages link only where the term entry exists; the rest are populated as the working group ratifies each definition.

Reproducibility: Obtaining consistent results using the same input data, computational steps, methods, and conditions of analysis (NASEM 2019).
Replicability: Obtaining consistent results across studies aimed at answering the same question with new data or methods.
Robustness: Stability of a finding under reasonable variation in modelling choices, samples, or analytic specification.
Pre-registration: Time-stamped declaration of hypotheses, design, and analysis plan prior to data observation.
Registered report: A two-stage publication format in which a protocol is peer-reviewed and accepted in principle before results exist.
Data availability statement: Mandatory disclosure declaring where, how, and under what conditions underlying data are accessible.
Computational reproducibility: Bit-for-bit or equivalent re-execution of an analysis from the same code, data, and computational environment.
Methods transparency: Reporting of materials, procedures, and decisions in sufficient detail to permit independent scrutiny.
Open materials: Public release of study materials (instruments, stimuli, protocols) under a reuse licence.
Open code: Public release of analysis code under an OSI-approved licence, ideally archived with a persistent identifier.
Protocol publication: Formal, citable publication of a study protocol prior to or alongside primary results.
Multiverse analysis: Reporting the distribution of results across the full set of defensible analytic specifications (Steegen et al. 2016).
Robustness check: An auxiliary analysis demonstrating that the headline result survives a substantive change in specification.
Specification curve analysis: A systematic descriptive technique that visualises results across all reasonable analytic specifications.
Replication study: A study explicitly designed to re-test the inference of a prior study, classified as direct or conceptual.
Reporting guideline: A structured checklist defining the minimum reporting required for a study design (ARRIVE, CONSORT, PRISMA, STROBE, etc.).
Authentication of key resources: NIH Rigor & Reproducibility requirement to validate the identity and quality of biological and chemical resources used.
Scientific rigor: The strict application of the scientific method to ensure unbiased, well-controlled experimental design (NIH).
Reproducibility crisis: Discourse term for the empirically observed gap between published findings and successful replication across fields.

Term pages are populated as the Reproducibility working group reviews each definition. See /dictionary/contribute to propose a term.

Cross-domain reference: FAIR principles assessment (Data infrastructure domain).

Stewardship

Reproducibility terminology sits between three communities. The CASRAI Dictionary working group integrates and publishes the vocabulary. The Center for Open Scienceis the originating steward of the TOP Guidelines, the Registered Reports format, and the preregistration registry on the Open Science Framework. The Committee on Publication Ethics (COPE)stewards the boundary cases where non-replication intersects with editorial concern.

Cross-reference to CODATA covers the related FAIR and RDM stewardship that touches reproducibility from the data-management side. The Reproducibility working group also liaises with FORCE11 on FAIR4RS and software-citation practice, with the EQUATOR Network on reporting guidelines, and with NIH and Wellcome on funder-policy alignment.

Related editorial and standards

Reproducibility standards hub — the standards-level companion to this domain
CRediT for authors — how contributor attribution supports reproducibility
CRediT role: Validation — the role most directly mapped to reproducibility checks
CRediT role: Data Curation — the role that makes computational reproducibility possible
FAIR principles assessment — cross-domain reference
CASRAI × CODATA federation — adjacent RDM stewardship

Federation

Cross-walks to external standards

Each entry in this domain carries machine-readable mappings to the upstream standard. Definitions remain canonical at the steward; CASRAI federates rather than re-publishes.

Standard	Steward	Scope
TOP Guidelines	Center for Open Science	Eight modular journal-policy standards covering citation, data, code, materials, design, analysis, preregistration, replication.
ARRIVE 2.0	NC3Rs (du Sert, Ahluwalia, Alam et al. 2020)	Reporting checklist for in vivo animal research; Essential 10 + Recommended Set.
NIH Rigor & Reproducibility	US National Institutes of Health	Four-pillar policy covering scientific premise, rigorous design, biological variables, and authentication of key resources.
CONSORT · PRISMA · STROBE	EQUATOR Network	Reporting guidelines for trials, systematic reviews, and observational studies respectively.

Frequently asked questions

Are reproducibility and replicability the same thing?

No. CASRAI follows the National Academies (NASEM 2019) usage. Reproducibility refers to obtaining the same result from the same data and code; replicability refers to obtaining a consistent result from a new study aimed at the same question. Robustness is a third, distinct construct concerning stability under analytic variation. Conflating the three is the single most common source of definitional drift in the literature, which is why the Dictionary maintains them as separate entries.

Why does CASRAI maintain reproducibility terminology if NIH and COS already do?

NIH defines reproducibility in the context of US biomedical grant policy; COS defines it in the context of TOP-compliant journal policy. Neither is a vendor-neutral controlled vocabulary that a CRIS, repository, or publisher submission system can ingest as machine-readable terms. CASRAI integrates these definitions, federates with their stewards, and exposes the result as Schema.org DefinedTerm markup under CC-BY 4.0.

Where do I propose a new reproducibility term?

Use the contribute flow. The Reproducibility working group reviews proposals during each release cycle (March and September). Accepted contributors receive CRediT attribution on the term entry itself.

How does CRediT support reproducibility?

CRediT provides the contributor-role granularity that reproducibility audits depend on. The Validation role in particular maps directly to reproducibility checks; Data Curation and Software map to the artefacts that make computational reproducibility possible. The Dictionary cross-references CRediT roles from every relevant reproducibility term.