Most research is reviewed and published after the results are known. That ordering, so obvious it usually goes unexamined, quietly distorts the literature: studies with striking positive results get published and studies with null results get filed away, and analyses can drift, after the fact, toward whatever story the data happen to tell. Pre-registration and its more rigorous cousin, the registered report, flip the order — committing to the plan before the data exist — and in doing so address some of the deepest threats to reproducibility. They are a central concern of the reproducibility domain and connect directly to the research-integrity domain.
The problems they are designed to solve
Two well-documented distortions motivate planning research in the open.
The first is publication bias: the tendency for positive, “significant” results to be published while null or negative results disappear. The literature that results is not a fair sample of the research that was done — it over-represents flukes and under-represents the disconfirmations that science depends on. A field can end up confidently believing an effect that the full body of evidence, published and unpublished, would not support.
The second is the family of analytic flexibility problems, of which HARKing — Hypothesising After the Results are Known — is the clearest example. When the hypothesis is written after seeing the data, and when there is freedom to choose among many possible analyses, it becomes easy, often unintentionally, to present an exploratory finding as if it had been predicted, and to select the analysis that produces the most publishable result. None of this need involve any intent to deceive; it is the natural consequence of making decisions while looking at the outcome.
Pre-registration: committing to the plan
Pre-registration is the practice of specifying, in a public, time-stamped record before data collection or analysis, what the study will do: its hypotheses, its design, its sampling and stopping rules, its outcome measures, and its planned analysis. The record is created in advance and cannot be quietly altered afterwards, which draws a clean line between what was confirmatory (predicted in advance) and what was exploratory (discovered in the data). Exploratory analysis remains entirely legitimate and valuable — pre-registration does not forbid it; it simply makes it honest by preventing exploratory findings from being dressed up as confirmatory ones.
The Open Science Framework (OSF), maintained by the non-profit Center for Open Science, is the most widely used infrastructure for this. OSF lets researchers create a registration — a frozen, time-stamped, citable snapshot of the study plan — and control when it becomes public. The plan is fixed; the credibility of any later claim to have predicted a result can be checked against it.
Registered reports: review before the results
A registered report takes the logic further and builds it into the publishing process itself, through a two-stage peer review designed and promoted by the Center for Open Science and now offered by a large and growing number of journals.
- Stage 1 is the protocol. Before any data are collected, the authors submit the introduction, the hypotheses, and a detailed methods and analysis plan. Reviewers assess the importance of the question and the soundness of the method — not the results, because there are none yet. If the protocol passes, the journal grants in-principle acceptance: a commitment to publish the completed study regardless of how the results turn out, provided the authors carry out the registered plan and the work is sound.
- Stage 2 is the completed study. The authors execute the plan, report what they found — positive, null, or mixed — clearly distinguish any exploratory analyses from the pre-registered confirmatory ones, and the paper is published.
The consequences are precise. Because the decision to publish is made before the results are known, publication bias is removed at its source — a null result is just as publishable as a positive one. Because the analysis plan is fixed and reviewed up front, HARKing and selective analysis are structurally prevented. And because reviewers shape the design while it can still be improved, peer review does its most useful work before the study is run rather than after, when nothing can be changed.
What this strengthens, and what it does not
Registered reports and pre-registration are powerful but not universal. They suit hypothesis-testing, confirmatory research best; they fit awkwardly onto genuinely exploratory, discovery-driven, or qualitative work, where the questions emerge from the material and a rigid pre-specified plan would be a forced fit. The honest position is that they are an excellent tool for a particular and very common kind of research, not a mandate for all of it. Used where they fit, they directly serve reproducibility: a study whose plan was fixed and public in advance is far easier for others to evaluate, replicate, and build on.
Crediting the planning work
Planning a study rigorously is itself a substantial contribution, and contributor-role metadata can record it. The CRediT taxonomy‘s Conceptualization and Methodology roles capture the intellectual work of formulating the research goals and designing the methods — precisely the work that a registered report front-loads and makes visible. Recording these roles ensures that the design effort, which a registered report elevates from invisible preparation to peer-reviewed output, is credited to the people who did it.
Where shared vocabulary fits
“Pre-registration”, “registered report”, “in-principle acceptance”, “Stage 1 protocol”, and “confirmatory analysis” are used loosely and sometimes interchangeably, which muddies what a given journal or record actually guarantees. A shared, federated vocabulary that defines these terms precisely — and points back to the Center for Open Science and the OSF registration infrastructure — is what lets a registered report in one venue be understood the same way in another. Supplying that definitional layer is the role the CASRAI dictionary is designed to play; the relevant terms sit in the reproducibility domain, with adjacent entries in the research-integrity domain.
Leave a Reply