Examples
Worked examples
- Is an instance
Many Labs 2 testing 28 effects across 125 samples and 36 countries.
- Is an instance
ManyBabies coordinated infant-research replications across dozens of labs.
Counter-examples
Looks similar, but isn't
- Not an instance
A single direct replication by one team.
- Not an instance
A many-analysts study (different design).
Editorial commentary
A crowdsourced replication is a coordinated effort in which many independent laboratories or research teams attempt to replicate the same set of studies under a common, pre-specified protocol, pooling the results to estimate how replicable a finding — or a whole field’s typical finding — actually is. Rather than relying on one lab’s attempt to reproduce another lab’s result, which cannot distinguish a genuine failure to replicate from an idiosyncrasy of the replicating team’s sample, setting, or procedure, a crowdsourced design runs the same protocol across dozens or hundreds of sites and reports the distribution of outcomes.
The best-known examples are the Many Labs series in psychology (Many Labs 2 tested 28 effects across 125 samples in 36 countries) and the field-wide Open Science Collaboration Reproducibility Project: Psychology (2015), which had independent teams each replicate one study from the original sample of 100. The same design has since been applied outside psychology, including the Reproducibility Project: Cancer Biology, and to developmental research through the ManyBabies consortia.
Why it matters for a research-integrity assessment
A crowdsourced replication provides an estimate of a field’s replicability that is not confounded with any single lab’s competence, equipment, or local population — a low replication rate across dozens of independent, pre-registered sites is much harder to attribute to any one team’s error than a single failed replication is. Because the protocol is agreed and typically pre-registered before any site collects data, crowdsourced replications also produce open, reusable datasets that downstream meta-research can draw on. They are a distinct design from a many-analysts study, which holds the data fixed and varies the analysts and analytic choices instead of varying the site and sample.
References
- Klein et al., 'Many Labs 2' (Advances in Methods and Practices in Psychological Science, 2018).
- Open Science Collaboration, 'Estimating the reproducibility of psychological science' (Science, 2015).
Also known as
Many Labs (concept) · multi-site replication
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="Crowdsourced replication"
vocab-term-identifier="https://casrai.org/dictionary/term/crowdsourced-replication" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"@id": "https://casrai.org/dictionary/term/crowdsourced-replication",
"name": "Crowdsourced replication",
"identifier": "https://casrai.org/dictionary/term/crowdsourced-replication",
"description": "A coordinated effort in which many independent laboratories or teams attempt to replicate the same set of studies under pre-specified protocols, in order to estimate field-wide replicability.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/reproducibility#set",
"url": "https://casrai.org/dictionary/term/crowdsourced-replication",
"alternateName": [
"Many Labs (concept)",
"multi-site replication"
],
"license": "https://creativecommons.org/licenses/by/4.0/",
"publisher": {
"@id": "https://casrai.org/#organization"
},
"author": {
"@id": "https://casrai.org/#editorial-team"
},
"datePublished": "2026-05-21T02:22:50",
"dateModified": "2026-09-05T14:24:23",
"inLanguage": "en-GB",
"isAccessibleForFree": true
}
