Peer review’s basic case is straightforward: before a manuscript enters the permanent scholarly record, independent experts check whether its methods, evidence, and conclusions hold up. What’s less often laid out clearly is how that value stacks up against the process’s own documented weaknesses — not as vague complaints, but as things that have actually been studied. This page covers the pros-versus-cons balance directly. For how peer review actually works step by step, see CASRAI’s peer review guide; for how the single-anonymized, double-anonymized, and open review models differ, see this comparison.
The case for peer review
The strongest arguments for peer review are about what happens without it, not just what happens with it:
- Independent quality control. A manuscript is checked by people with no stake in its acceptance, applying domain expertise the handling editor alone usually doesn’t have across every submission.
- A credibility signal readers can rely on. “Peer-reviewed” functions as a floor, not a guarantee — it tells a reader, funder, or employer that independent experts judged the methodology and claims defensible enough to publish, which is why library databases and indexes let users filter to peer-reviewed content specifically.
- It genuinely improves manuscripts. Reviewer feedback routinely catches unclear methodology reporting, unsupported claims, missing controls, or overlooked prior work, and most accepted manuscripts go through at least one revision cycle in response.
- It’s the mechanism the rest of the system is built around. Citation indexes, institutional assessment, funder reporting, and tenure review all lean on peer-reviewed publication as the baseline unit of vetted output — removing it wouldn’t remove the need for some check, it would just relocate where that check happens.
The documented weaknesses
None of this is peer review’s opponents talking — most of what follows is acknowledged by the standards bodies that defend the process. The ICMJE recommendations themselves describe peer review as “an important extension of the scientific process,” while explicitly noting that its actual value has been disputed within the field. The specific weaknesses are worth separating out individually, because they’re different problems with different fixes.
Reviewers routinely disagree with each other
If peer review reliably identified a manuscript’s true quality, independent reviewers assessing the same submission should mostly agree. They don’t, and this has been measured directly. A 2010 multilevel meta-analysis of 48 reliability studies across journal peer review (published in PLOS ONE) found a low overall level of inter-rater agreement — a mean correlation/ICC around 0.34 and a mean Cohen’s kappa around 0.17, the latter falling in the “slight agreement” range on the standard interpretation scale. In practice, this means two qualified reviewers looking at the same manuscript often reach materially different verdicts, and which reviewers a manuscript happens to draw can matter as much as the manuscript’s actual quality.
A poor track record catching fraud and serious error
Peer review is designed to assess plausibility and methodological soundness on the information the authors present — it was never built to detect deliberately fabricated data, and its record on that front reflects it. The clearest illustration is the 1998 Lancet paper by Andrew Wakefield linking the MMR vaccine to autism: it passed peer review and was published, and it took over a decade — plus dogged investigative journalism that uncovered undisclosed conflicts of interest and manipulated case data, not the original review process — before the journal issued a full retraction in 2010 and Wakefield lost his UK medical license. The paper’s damage to vaccine uptake was already done well before the correction caught up. Fraud cases involving other high-profile researchers show the same pattern: the underlying misconduct is typically surfaced by whistleblowers, data sleuths, or post-publication scrutiny, not caught by the reviewers who approved the original manuscript. See CASRAI’s guide to paper mill and fabrication red flags for how detection has shifted toward post-publication and automated screening precisely because pre-publication review alone hasn’t been sufficient.
Bias toward familiar names and institutions
A frequently cited 1982 study by Douglas Peters and Stephen Ceci (Behavioral and Brain Sciences) resubmitted 12 already-published psychology articles back to the same prestigious journals that had originally accepted them, 18–32 months earlier, with the authors’ real names and institutions replaced by fictitious, lower-status ones. Only 3 of 38 editors and reviewers detected the resubmission. Of the 9 papers that went on to full review under their new fictitious identities, 8 were rejected — 16 of 18 reviewers recommended against publication, most commonly citing “serious methodological flaws” in work the same journal had already judged publication-worthy under a well-known author’s name. It’s a small, dated study of one field, but it remains one of the most direct empirical demonstrations that reviewer judgment isn’t purely about the manuscript’s content. This is also the core rationale behind double-anonymized review — see the peer review models comparison for how different journals try to address it.
Reviewer burden, unpaid labor, and turnaround time
Journal peer review runs almost entirely on unpaid volunteer labor from researchers already stretched across their own research, teaching, and administrative work — a structurally different arrangement from, for instance, NIH grant study-section reviewers, who receive an honorarium for that separate, funding-focused review process. That volunteer basis is a major driver of one of the most commonly cited practical complaints about peer review: slow, unpredictable turnaround, since there’s no enforceable deadline behind an unpaid favor. Reviewer fatigue and declining willingness to take on review requests, especially in high-submission-volume fields, compounds the problem over time.
It doesn’t test reproducibility
Peer review evaluates whether a study’s design and reasoning are sound and whether the reported results support the stated conclusions — it does not, and structurally cannot, verify that an independent lab would get the same result by repeating the work, because reviewers are assessing a manuscript, not re-running an experiment. A peer-reviewed result can still fail to replicate, which is one of the direct contributors to what’s now widely referred to as the reproducibility crisis. Formats like Registered Reports, which peer-review the study design before results exist, are a direct response to this specific gap rather than to peer review’s other weaknesses.
How the field is responding
None of the above has produced consensus that peer review should be abandoned — instead, the field has layered additional checks around it rather than relying on the initial review as the last word:
- Open peer review — publishing reviewer identities and reports alongside the article — trades some reviewer candor for reviewer accountability. Covered in full in the review models comparison.
- Post-publication peer review — open commentary and critique after an article is already public — catches problems the original review missed, at the cost of the article already being citable and public when they surface.
- Automated screening tools for statistical errors, image manipulation, and paper mill patterns (e.g. SciScore) are increasingly used to supplement, not replace, human reviewer judgment before or during review.
- Registered Reports shift review earlier, to the study design stage, specifically to address publication bias and the reproducibility gap rather than fraud detection.
- Retraction infrastructure and the retraction process itself function as the field’s acknowledgment that pre-publication review is fallible and a documented correction mechanism has to exist for when it fails.
The bottom line
Peer review’s advantages and weaknesses aren’t really in tension with each other — they describe the same process from different angles. It remains the best broadly available check the field has for filtering out unsound methodology and unsupported claims before publication, and removing it wouldn’t eliminate the need for that check, just relocate it somewhere with even less structure. But it is not a reliability guarantee, a fraud detector, or a reproducibility test, and treating it as any of those three overstates what the evidence — including evidence from standards bodies that use and defend the process — actually supports.
Common questions
What are the main advantages of peer review?
Independent expert quality control before publication, a credibility signal that readers, funders, and indexes can filter on, manuscript improvement through reviewer feedback, and serving as the baseline unit the rest of the scholarly assessment system (citation indexes, tenure review, funder reporting) is built around.
What are the main disadvantages of peer review?
Low measured agreement between independent reviewers on the same manuscript, a poor track record catching deliberate fraud (most high-profile cases surface through post-publication scrutiny, not the original review), documented bias favoring familiar authors and institutions, slow turnaround driven by unpaid reviewer labor, and no test of whether results actually replicate.
Does peer review catch fraud?
Rarely as its primary detection mechanism. Peer review evaluates plausibility and methodology based on what authors present; deliberately fabricated data is built to look plausible. Most documented high-profile fraud cases, including the 1998 Wakefield MMR-autism paper, were exposed by post-publication investigation rather than the original reviewers.
Is peer review reliable?
Measured inter-rater reliability between reviewers assessing the same manuscript is low — a 2010 meta-analysis of 48 studies found a mean Cohen’s kappa around 0.17, in the “slight agreement” range. Reliability also varies by field, journal, and review model, so “reliable” isn’t a single fixed property of the process.
What are alternatives or supplements to traditional peer review?
Open peer review, post-publication peer review, Registered Reports (design peer-reviewed before results exist), and automated screening tools for statistics, images, and paper mill patterns are the main approaches in current use — most function as additions to traditional pre-publication review rather than full replacements for it.
