Metascience — also called meta-research, or informally "the science of science" — is the empirical field that studies the research process itself: how studies are designed and powered, how results are reported, how often findings replicate, how peer review and publication shape what becomes visible, and how funding and career incentives shape researcher behavior. CASRAI’s Dictionary entry for metascience covers the core definition and its founding literature; this guide goes further, mapping the field’s structure, methods, institutions, and what it means in practice for research administrators who set or enforce policy that metascience findings are increasingly used to justify.
The five things metascience studies
A widely cited 2015 PLOS Biology paper by John Ioannidis, Daniele Fanelli, Debbie Dunne, and Steven Goodman, "Meta-research: Evaluation and Improvement of Research Methods and Practices," frames metascience around five thematic areas. Understood concretely, they are:
- Methods — statistical power, study design, and pre-analysis choices that determine whether a study can detect a real effect if one exists.
- Reporting — whether a published paper contains enough detail (methods, data, code, materials) for another researcher to understand or attempt to reproduce it.
- Reproducibility — whether re-running the original analysis on the original data returns the same result, and whether an independent replication attempt returns a consistent finding. CASRAI’s Reproducibility entry and the Reproducibility crisis entry cover this concept in its own right; metascience is the field that studies reproducibility empirically across many studies at once, rather than being the phenomenon itself.
- Evaluation — how research and researchers get assessed: peer review quality, journal metrics, and the criteria institutions use for hiring, tenure, and funding decisions.
- Incentives — how the reward structures researchers actually operate under (publish-or-perish pressure, novelty bias in what gets published) shape the choices that produce the problems the other four categories measure.
How the field took shape
Metascience did not emerge from a single founding moment, but a few landmarks are commonly cited as having defined it as a distinct field rather than scattered methodological criticism. John Ioannidis’s 2005 PLOS Medicine paper "Why Most Published Research Findings Are False" argued, from first principles about study power, bias, and the number of teams testing a hypothesis, that a large share of published claims across biomedicine were likely to be false positives. In 2009, Iain Chalmers and Paul Glasziou’s Lancet analysis "Avoidable Waste in the Production and Reporting of Research Evidence" estimated that roughly 85% of biomedical research investment was avoidably wasted through poor question selection, weak design, incomplete reporting, and restricted access to results — reframing research inefficiency as something that could itself be measured and reduced.
Dedicated infrastructure followed within a few years. The Center for Open Science (COS) was founded in Charlottesville, Virginia in January 2013 by Brian Nosek and Jeffrey Spies, and built the Open Science Framework (OSF) as shared infrastructure for pre-registration, version control, and public sharing of data and materials. Stanford’s Meta-Research Innovation Center (METRICS), co-directed by Ioannidis and Steven Goodman, launched in 2014 as a research center dedicated specifically to studying research practice. The UK Reproducibility Network, launched in 2018 and led by University of Bristol biological psychologist Marcus Munafò, coordinates reproducibility-focused work across UK universities. In 2019, an international coalition — the Wellcome Trust, Leiden University’s Centre for Science and Technology Studies (CWTS), the research-technology firm Digital Science, and the University of Sheffield — launched the Research on Research Institute (RoRI) on 30 September 2019 to study research systems and funding at scale. Government has since followed: UKRI has established a dedicated Metascience Unit, reportedly the first government-backed unit of its kind, to apply metascience findings directly to how UK public research funding is designed and administered.
Metascience, the reproducibility crisis, and replication: related but distinct
These three terms are often used loosely as synonyms, but they describe different things. The reproducibility crisis is a specific, observed phenomenon — the finding, across psychology, biomedicine, and other fields, that a large share of published results do not hold up when independently re-tested. A replication study is a specific research design: an independent attempt to reproduce a prior finding, most famously demonstrated at scale by the Center for Open Science’s Reproducibility Project: Psychology, covered in depth in CASRAI’s guide on the psychology replication crisis. Metascience is the broader empirical discipline that studies the reproducibility crisis as one of its central objects, alongside reporting quality, evaluation practices, and incentive structures — it is the field that produces the evidence that a crisis exists and tests what actually fixes it, not the crisis itself. The distinction matters in practice for a related reason: as the U.S. National Academies of Sciences, Engineering, and Medicine set out in its 2019 report Reproducibility and Replicability in Science, reproducibility (consistent results from the original data and methods) and replicability (consistent results from a new study of the same question) are themselves not interchangeable — a point metascience research treats as a starting distinction rather than a technicality.
How metascience research actually gets done
Metascience is empirical, not purely reflective — it studies research practice using many of the same tools researchers use to study anything else, applied reflexively to the research process itself:
- Large-scale coordinated replication — many independent teams each replicate a set of published studies under agreed protocols, as in the Reproducibility Project: Psychology, producing an empirical reproducibility rate for a field rather than a verdict on any single study. See CASRAI’s crowdsourced replication entry.
- Meta-analysis and meta-research audits of the literature — systematically coding hundreds or thousands of published papers for statistical power, pre-registration status, effect-size inflation, or data-sharing rates to characterize a field’s practices in aggregate. CASRAI’s reproducibility audit entry describes one operational form of this.
- Pre-registration and Registered Reports as both intervention and study object — pre-registration and the Registered Report publishing format are reforms that metascience research helped motivate (by demonstrating that undisclosed flexibility in analysis choices inflates false-positive rates), and their subsequent adoption is itself tracked and studied as a metascience outcome.
- Computational and artifact reproducibility checks — verifying that a paper’s code, software environment, and data actually reproduce its published results, formalized in schemes such as the ACM’s Artifact Review and Badging framework. See CASRAI’s computational reproducibility entry.
Why this matters for research administrators — and CASRAI’s own angle
CASRAI is a standards body, not a metascience research group, but a real share of its work exists to supply the standardized, machine-readable infrastructure that metascience research depends on, and that policy built on metascience findings needs to be enforceable rather than aspirational. Concretely:
- CRediT (jointly stewarded by CASRAI and NISO as ANSI/NISO Z39.104-2022) turns authorship into structured, comparable data instead of free-text acknowledgments — exactly the kind of dataset a metascience study of authorship, credit allocation, or contributorship patterns needs to exist before it can be studied at scale. See CASRAI’s CRediT overview.
- Machine-actionable data management plans — the subject of CASRAI’s data management plan (DMP) entry and the broader research data management content — turn planning documents that used to be filed away as static prose into structured records that can be audited for whether commitments (repository choice, licensing, timelines) were actually met, which is itself a metascience-relevant question about research reporting quality.
- Assessment-reform initiatives such as DORA (the San Francisco Declaration on Research Assessment) and CoARA (the Coalition for Advancing Research Assessment), which push institutions away from journal-level metrics like the Journal Impact Factor toward broader evaluation of research and researchers, are themselves grounded directly in metascience findings about how metrics distort incentives — the "evaluation" and "incentives" themes above, applied to real institutional policy.
- Funder policy increasingly encodes metascience findings directly: the NIH’s rigor and reproducibility expectations for grant applications are a direct policy response to the same evidence base metascience research produced. See CASRAI’s NIH Rigor and Reproducibility policy entry.
For a research administrator, the practical upshot is that metascience is no longer confined to methodology seminars — its findings now show up directly as funder requirements, journal policies, and institutional assessment criteria, which means understanding the field’s core claims and evidence base is increasingly part of the job, not adjacent to it.
Frequently asked questions
Is metascience a recognized academic discipline?
It functions as one in practice, even without a single home department: dedicated research centers (METRICS at Stanford, RoRI, the UK Reproducibility Network), a recurring international Metascience conference, and journals that specifically publish meta-research (such as Research Integrity and Peer Review) all treat it as a distinct field with its own methods and literature, even though individual metascience researchers are trained across statistics, psychology, philosophy of science, and their original home disciplines.
Is metascience the same as research integrity?
No, though they overlap. Research integrity is primarily concerned with misconduct, ethics, and compliance — fabrication, falsification, plagiarism, and the policies that prevent and address them. Metascience is broader and largely non-accusatory: it studies how research is designed, reported, and rewarded across an entire field, including in studies with no integrity violation at all, to understand why good-faith research still fails to replicate as often as it should.
Who funds metascience research?
A mix of research funders, philanthropies, and — increasingly — government science-policy bodies fund metascience directly, rather than treating it as incidental to funding substantive science. The Wellcome Trust is a founding partner of the Research on Research Institute; UKRI has established a dedicated Metascience Unit to apply metascience evidence to how UK public research funding itself is designed.
How does metascience relate to CRediT and CASRAI’s other standards?
Metascience needs structured, comparable data about the research process to study it at scale — free-text papers alone are hard to analyze systematically. CRediT, machine-actionable DMPs, and persistent-identifier infrastructure (ORCID, ROR, DOI) are exactly this kind of infrastructure: each converts an aspect of the research process into a form that can be counted, compared, and audited across a field, which is what makes large-scale metascience studies practically possible in the first place.
