How to Conduct a Peer Review

This page is the execution layer: once you’ve agreed to review a manuscript, what do you actually do with it? For the bigger picture — why peer review exists, its stages, and its limitations — see CASRAI’s Peer Review guide. For the reviewer role itself, including how to become a reviewer and the full COPE conduct obligations, see Peer Reviewer: Role, Responsibilities, and How to Become One. This page assumes you’ve already accepted an invitation and have a manuscript in front of you, and walks through reading it, assessing it, and writing a report that’s actually useful to the editor and authors.

Before you open the manuscript

Two checks belong before you read a single page, not after:

Confirm you can be unbiased. If you share an institution with an author, have been their mentor, mentee, close collaborator, or joint grant holder within roughly the past three years, or have any financial or personal interest in the outcome, decline or disclose it to the editor before proceeding. See CASRAI’s Peer Reviewer guide for the full COPE conflict-of-interest list — the short version is: when in doubt, tell the editor and let them decide whether it’s disqualifying.
Confirm you can meet the deadline and the scope. If you can only competently assess part of the manuscript (say, the statistics but not the clinical framing), say so explicitly in your report rather than silently signing off on sections outside your expertise.

Read it in passes, not once

Reviewers who read a manuscript once, start to finish, and write comments as they go tend to produce reports that are disorganized and miss the manuscript’s overall argument. A more reliable approach, consistent with guidance from publishers including Wiley and PLOS:

Skim pass. Read the abstract, introduction, figures, and conclusion first. Form a provisional view: what is this paper claiming, and does the claim seem plausible and significant? This pass also tells you roughly how long the full review will take.
Detailed pass. Read the full manuscript section by section — methods, results, discussion — taking notes tied to specific line or page numbers as you go. Most journal submission systems let you annotate a PDF directly; if not, a running notes document with line references saves rework later.
Verification pass. Go back through your notes and check anything you flagged as uncertain: does a cited statistical test actually match the data described, do the reported sample sizes add up across tables, does the discussion overstate what the results show. This pass is where most substantive review comments actually get finalized.

What to assess, in practice

CASRAI’s Peer Review guide lists the categories reviewers are generally asked to judge — originality, methodological soundness, validity of conclusions, clarity, ethical compliance, and fit. Here’s what applying each of those actually looks like at the manuscript level:

Novelty and significance

Does the introduction accurately represent what’s already established in the literature, or does it overstate the gap being filled?
If the authors claim a result is novel, can you identify closely related prior work they haven’t cited or addressed?
Would the result change how someone in the field thinks or acts, or is it a small, already-expected increment presented as more than it is?

Methodology soundness

Is the study design (experimental, observational, qualitative, computational) actually capable of answering the question the authors pose?
Are sample size, controls, and comparison groups adequate and clearly justified — not just present, but appropriate to the claim being made?
Is there enough procedural detail (reagents, instruments, survey instruments, code, parameters) for another researcher in the field to understand what was actually done?

Statistical validity

Does the chosen statistical test match the data type and study design (e.g. paired vs. unpaired, parametric vs. non-parametric assumptions actually checked, not just assumed)?
Is multiple-comparison correction applied where multiple hypotheses or subgroups are tested, and is it disclosed if not?
Do the reported effect sizes and confidence intervals, not just p-values, support the strength of the claims made in the abstract and discussion?
Are missing data, exclusions, or post-hoc subgroup analyses disclosed and justified, rather than surfacing only in the results with no explanation?

Ethical compliance

For human-subjects research, is there a clear statement of IRB/ethics committee approval (or a documented exemption) and informed consent, consistent with CASRAI’s Informed Consent in Research guide?
For animal research, is there evidence of the relevant institutional animal-care approval?
Are competing interests, funding sources, and any relevant conflicts of interest disclosed by the authors themselves, not just by you as reviewer?
Does anything in the data, figures, or text raise a concern about research misconduct (fabrication, falsification, plagiarism, image manipulation)? If so, per COPE and ICMJE guidance, flag it to the editor directly rather than investigating it yourself.

Structuring your review report

Most journals provide a submission form with defined fields, but the underlying structure is consistent across publishers, and worth using even when a journal gives you a blank box:

Summary. One short paragraph, in your own words, restating what the manuscript claims to do and find. This does two things: it shows the editor and authors you understood the paper, and it often surfaces a mismatch between what the authors think they showed and what you understood — itself a useful piece of feedback.
Major concerns. Issues that affect whether the paper’s central claims hold up — a flawed statistical approach, an inadequate control condition, an unsupported conclusion, a significant gap in prior-work coverage. Number them; editors and authors both use major-comment numbers to track responses through a revision cycle.
Minor concerns. Everything else worth flagging that doesn’t threaten the paper’s validity — unclear figure labels, missing citations, awkward phrasing, small inconsistencies between text and tables.
Recommendation. Accept, minor revision, major revision, or reject, matched to the journal’s own categories. The recommendation should follow logically from what you wrote above it — an editor should never have to guess why a report full of major concerns recommends acceptance, or vice versa.
Confidential comments to the editor (if the journal offers this field). Reserve this for things the authors genuinely shouldn’t see — a suspicion of misconduct you want the editor to look into, or a disclosed conflict you realized partway through. COPE’s guidance is explicit that this field should not be used to say something harsher to the editor than you were willing to put in the comments the authors actually see.

Writing feedback that’s constructive, not destructive

The distinction editors and authors care about isn’t tone alone — a blunt review can still be constructive, and a politely worded one can still be useless. The difference is whether a comment gives the author something they can act on.

Destructive	Constructive
“The statistics are wrong.”	“The paired t-test in Table 2 assumes normally distributed differences, which isn’t demonstrated for this sample size (n=8); a Wilcoxon signed-rank test or an explicit normality check would strengthen this.”
“This is poorly written and hard to follow.”	“Section 3.2 introduces three new variables before defining any of them; defining each on first use, or adding a symbols table, would make the derivation easier to follow.”
“The authors clearly don’t understand the prior literature.”	“The introduction doesn’t address [specific prior study], which reports a comparable effect using a different method — the manuscript should either distinguish its contribution from that work or reconcile the differing results.”

A few working principles behind that distinction, consistent with COPE’s and ICMJE’s guidance on reviewer conduct:

Substantiate every criticism with specifics — a line number, a table, a citation — rather than a general assertion. “The methodology is weak” is not actionable; “the control group in Experiment 2 wasn’t matched for age” is.
Comment on the work, not the authors. “The analysis doesn’t account for X” rather than “the authors failed to consider X” or any comment on competence, intent, or writing ability as a personal trait.
Say what would fix it, where you can. A comment that identifies a problem and a plausible path to addressing it is more useful to both authors and editor than one that only identifies the problem.
Keep your tone in the confidential editor comments consistent with what you tell the authors. If you wouldn’t say it to the authors, it shouldn’t appear only in the editor-only field either — per COPE’s guidelines, that field isn’t a place for accusations the authors never see.
Don’t pad the report to seem thorough. A short report that identifies the two or three issues that actually matter is more useful to an editor than a long one that buries them in minor stylistic notes.

Confidentiality while you review

Treat the manuscript, your review, and any communication about it as confidential from the moment you receive it — don’t discuss it, share the file, or use any of its unpublished ideas or data for your own work, even after the review closes. This applies indefinitely, not just until a decision is made. CASRAI’s Peer Reviewer guide covers the full set of COPE and ICMJE confidentiality obligations, including when it’s appropriate to involve a mentee or co-reviewer (only with the editor’s permission, and only with that person credited).

If you realize partway through that you should decline

Sometimes a conflict or capability gap only becomes clear once you’re reading the manuscript — you recognize an author’s identity despite an anonymized review, you realize the paper is outside what you can actually assess, or you find you can no longer meet the deadline. Tell the editor immediately rather than pushing through; editors would rather reassign a review early than receive a late or compromised one. Declining partway through is not a failure — agreeing to review something you can’t fairly assess, and doing it anyway, is the actual problem COPE’s guidance is trying to prevent.

Common mistakes first-time reviewers make

Reviewing as a proofreader, not an evaluator. Line-editing grammar while missing that the central statistical test doesn’t match the study design.
Writing the recommendation first and the reasoning to match it, rather than letting the recommendation follow from the evidence in the report.
Treating “reject” as requiring the harshest language available. A reject recommendation still benefits from specific, professional comments — authors often revise and resubmit elsewhere, and a report that’s all major concerns but written destructively teaches them nothing.
Skipping the summary paragraph. It’s easy to treat as filler, but it’s often what tells an editor whether reviewer and authors actually agree on what the paper claims.
Not flagging a suspected duplicate submission or plagiarism to the editor directly, and instead either ignoring it or trying to resolve it within the review itself — COPE is explicit that reviewers should raise the concern with the editor rather than investigate it themselves.

Common questions

How long should a peer review report be?

There’s no fixed length requirement across journals. A report is long enough when it covers every major concern with enough specificity that the authors can act on it, and short enough that the major concerns aren’t buried under minor ones. A handful of well-substantiated major comments is usually more useful than a long list dominated by minor stylistic notes.

Should I recommend a specific decision, or just describe the problems?

Most journals ask for both: a recommendation (accept, minor revision, major revision, reject) and the reasoning behind it. Some editors weight reviewer recommendations heavily; others use reviewer reports mainly as evidence and make the final call themselves — either way, a recommendation with no supporting reasoning, or reasoning with no recommendation, is less useful than both together.

What if two reviewers disagree?

This is normal and part of why journals typically use more than one reviewer — the editor weighs both reports, may request a third opinion, and makes the final decision. Reviewers aren’t expected to reach consensus with each other; a well-substantiated, honest assessment is the goal even if another reviewer sees it differently.

Can I use an AI tool to help write my review?

Treat this as a confidentiality question first: uploading unpublished manuscript text into a third-party AI tool can violate the confidentiality obligation covered above, since you don’t control what that tool retains or how it’s used. Check the journal’s specific policy before using any AI tool on manuscript content — policies vary and are still evolving across publishers.

Do I need to be an expert in every part of the manuscript to review it?

No, but you do need to say clearly which parts you can and can’t assess. COPE’s guidance is explicit that partial expertise is fine as long as it’s disclosed, rather than silently reviewing sections outside your competence as if you’d fully evaluated them.

Related CASRAI resources

Peer Review: What It Is, How It Works, and Why It Matters — the process-level guide this page extends
Peer Reviewer: Role, Responsibilities, and How to Become One — becoming a reviewer, COPE conduct obligations, and recognition
COPE Guidelines Explained
Research Misconduct Case Studies
Informed Consent in Research
Single-anonymized peer review and double-anonymized peer review
Types of Conflict of Interest