Peer Review Practice Certification — Professional

Lesson material opens once you are enrolled. The full syllabus and every learning outcome are shown here so you can judge the coverage before you pay.

1. Module 1

   P1

   1 lesson

   1. Locked.The remit revisited: soundness, significance and venue mandateLocked

      On completion you will be able to

      • The learner can:
      • Analyse a venue's published criteria and diagnose internal inconsistencies between the stated criteria, the reviewer form, and the decision categories available.
      • Design reviewer guidance for a venue that makes the assessment criteria operable rather than aspirational.
      • Evaluate whether a completed review answered the question the venue asked, and construct feedback to that reviewer.
      • Justify a soundness-only versus a significance-gated review policy in terms of the venue's function, and predict the failure modes of each.
      • Explain the documented limitations of peer review as a mechanism — inter-reviewer inconsistency, poor reproducibility of decisions, insensitivity to fabrication, and cost — and defend a position on what review is for given those limitations.

2. Module 2

   P2

   1 lesson

   1. Locked.Anonymisation models, bias, and the evidence baseLocked

      On completion you will be able to

      • The learner can:
      • Compare single-anonymous, double-anonymous, open-identity, open-report and post-publication models against the specific harms each is intended to mitigate.
      • Appraise the empirical evidence on whether anonymisation reduces bias, and state honestly what remains unresolved.
      • Assess the practical defeasibility of anonymisation in a named small field, and quantify the effect of preprinting on it.
      • Design a bias-mitigation approach for a venue that does not rest solely on anonymisation.
      • Distinguish bias in reviewer judgement from bias in reviewer selection, and explain why the second is often the larger effect and the less discussed.

3. Module 3

   P3

   1 lesson

   1. Locked.Advanced methodological reviewLocked

      On completion you will be able to

      • The learner can:
      • Evaluate the design-to-claim relationship across a wide range of designs and articulate the specific inferential gap in each.
      • Detect design-level failures that survive superficial reading: an active comparator that does not isolate the intervention, a validation set contaminated by the training set, a confound introduced by the sampling frame, an outcome measured after the exposure it purports to predict.
      • Assess qualitative research on its own criteria — sampling logic, saturation claims, analytic transparency, reflexivity, and the fit between evidence and interpretation — without importing quantitative criteria.
      • Review computational and modelling work: reproducibility of the pipeline, parameter sensitivity, validation strategy, and whether the model's assumptions are stated and defensible.
      • Review machine-learning and prediction studies for leakage, inappropriate cross-validation, class imbalance handling, external validation, and calibration as distinct from discrimination.
      • Judge replication and null-result submissions on criteria appropriate to them, and resist the documented tendency to hold them to a higher standard.
      • Construct a methodological critique that specifies the additional evidence or analysis that would resolve it, and estimate whether that is within the scope of a revision.

4. Module 4

   P4

   1 lesson

   1. Locked.Statistical reviewLocked

      On completion you will be able to

      • The learner can:
      • Check whether the analysis performed matches the analysis described, and whether either matches any registration or protocol.
      • Identify analysis-level errors: pseudoreplication and non-independence; multiplicity without adjustment or acknowledgement; inappropriate dichotomisation; unmodelled clustering; inappropriate handling of missing data; and covariate adjustment chosen after seeing the results.
      • Evaluate effect estimates and uncertainty, and detect the substitution of significance for magnitude.
      • Perform arithmetic plausibility checks — whether reported means, standard deviations, percentages and denominators are mutually consistent and possible for the stated sample.
      • Assess figures as statistical claims: what a bar chart of means conceals, whether error bars are defined, whether axes are truncated, and whether the figure supports the sentence citing it.
      • Write a statistical critique that a non-statistician author can act on and a statistician cannot dismiss.
      • Determine when to tell the editor that specialist statistical review is required, and specify what that reviewer should be asked.

5. Module 5

   P5

   1 lesson

   1. Locked.Reporting completeness and guideline-anchored reviewLocked

      On completion you will be able to

      • The learner can:
      • Select the correct guideline and extension for a design, including composite and hybrid designs where no single guideline fits.
      • Integrate guideline-based completeness findings into a review without producing a checklist masquerading as a review.
      • Distinguish reporting deficiency from methodological deficiency consistently across a manuscript set, and defend each classification.
      • Advise a venue on which guidelines to require, at what point in the workflow, and with what enforcement.
      • Explain to an author why a guideline is being invoked and what compliance would change.

6. Module 6

   P6

   1 lesson

   1. Locked.Data, code, materials and availabilityLocked

      On completion you will be able to

      • The learner can:
      • Verify an availability claim end to end: resolve the identifier, inspect the record, confirm it contains what the paper says, and check licensing, embargo and access conditions.
      • Judge whether deposited material is sufficient to support the specific claim, distinguishing processed summaries from the data the analysis requires.
      • Review a code deposit for correspondence to the described analysis, versioning, licensing, dependency capture and executability in principle.
      • Evaluate a restriction claim — identifiable human data, Indigenous data governance, legal or commercial restriction, species protection — for honesty and for whether a controlled-access route is genuinely described.
      • Escalate an availability failure proportionately, distinguishing the case where the claim depends on the data from the case where it does not.
      • Advise a venue on availability policy and on how to make it enforceable during review rather than aspirational at submission.

7. Module 7

   P7

   1 lesson

   1. Locked.Integrity screening in depthLocked

      On completion you will be able to

      • The learner can:
      • Detect image duplication, manipulation and inappropriate processing across a range of image types, and document each observation with figure, panel and coordinates.
      • Apply a stated threshold for escalation, and articulate the base-rate reasoning that keeps a suspicion proportionate.
      • Analyse a submission for cumulative paper-mill signatures and produce a structured, evidenced, non-conclusory report for the editor.
      • Identify citation manipulation in its forms — self-citation coercion by reviewers, cartel behaviour between journals or groups, and irrelevant citation padding — and distinguish them from legitimate dense citation of a small field.
      • Assess authorship anomalies: changes between submission and revision, additions without stated contribution, corresponding-author substitutions, and affiliation inconsistencies.
      • Determine the correct escalation route for a given finding and draft the communication at each step.
      • Evaluate the reviewer's own exposure — defamation risk, confidentiality risk, and the professional consequences of an unfounded allegation.

8. Module 8

   P8

   1 lesson

   1. Locked.Reviewer ethics: confidentiality, conflicts and generative AILocked

      On completion you will be able to

      • The learner can:
      • Construct the confidentiality argument against third-party generative-AI use from first principles, without relying on any specific policy citation.
      • Adjudicate genuinely arguable AI-use cases — a translation tool, a venue-hosted assistant, a locally run model, a grammar checker on the reviewer's own prose — and state the discriminating principle.
      • Draft venue-level reviewer guidance on AI use that is enforceable, that does not collapse into an unfollowable blanket prohibition, and that survives a change in the tooling landscape.
      • Design a conflict-of-interest declaration process for a venue, including look-back periods, the treatment of institutional conflict, and the handling of a conflict discovered mid-review.
      • Handle a reviewer who has breached confidentiality, including determination, proportionate response, and record-keeping.
      • Advise on the ghost-review and co-review problem at venue level.

9. Module 9

   P9

   1 lesson

   1. Locked.Conference reviewLocked

      On completion you will be able to

      • The learner can:
      • Contrast conference and journal review across assessment criterion, comparability, revision availability, timescale, capacity constraint, and finality.
      • Operate as a programme committee member: bid, self-assess competence, score against a rubric with a confidence rating, and meet a hard deadline.
      • Engage with an author rebuttal, classify each rebuttal point as answering, partially answering or deflecting the criticism, and revise or maintain a score with a stated reason.
      • Contribute to programme-committee discussion and write a meta-review that represents the discussion rather than the meta-reviewer's own view.
      • Manage, as an area chair, a set of divergent reviews under a quota.
      • Identify the pathologies specific to conference review: score inflation and deflation, low-confidence reviews given equal weight, reviewers who do not read the rebuttal, and reviews written in the final hours.

10. Module 10

    P10

    1 lesson

    1. Locked.Grant peer review and panel dynamicsLocked

       On completion you will be able to

       • The learner can:
       • Review a proposal for what it proposes rather than for what it has achieved, and assess feasibility, team, resources and fit against a call's published criteria.
       • Score against a published scale using its anchors, and justify the score in a written critique that will be read aloud or summarised.
       • Participate in a panel: present a proposal, respond to challenge, revise a score after discussion, and state what changed their view.
       • Recognise and counter the documented panel-dynamic effects: anchoring on the first score, order effects, deference to seniority, the loudest-reviewer effect, drift across a long session, and fatigue in the final hour.
       • Chair a panel session: manage conflicts and recusal, set and hold the discussion structure, calibrate at the start, protect the minority view, and record the panel's advice accurately.
       • Distinguish the roles of panel, chair, programme officer and funder, and explain why the panel's output is advice rather than a decision.
       • Apply the reviewer confidentiality obligation in the grant context, including the generative-AI prohibition [VERIFY] that arose there first.

11. Module 11

    P11

    1 lesson

    1. Locked.Registered reports and results-blind reviewLocked

       On completion you will be able to

       • The learner can:
       • Review a Stage 1 registered report: question importance, design adequacy, analysis pre-specification, and outcome-neutral quality-control criteria.
       • Specify the positive controls, manipulation checks and data-quality thresholds a Stage 1 protocol should commit to, and explain why they must be set before results exist.
       • Review a Stage 2 submission against protocol adherence, and evaluate deviations for whether they were unavoidable, disclosed and consequential.
       • Resist the pressure to reassess a Stage 2 submission on the attractiveness of its results, and articulate why in-principle acceptance binds.
       • Explain what the format is designed to prevent — publication bias, outcome switching, HARKing, and the reviewer's own susceptibility to a tidy result.
       • Advise a venue on operating the format, including timelines, the handling of protocol amendments, and reviewer continuity between stages.

12. Module 12

    P12

    1 lesson

    1. Locked.Running the process: handling, adjudication, appeals, recognitionLocked

       On completion you will be able to

       • The learner can:
       • Triage a submission and issue a reasoned desk rejection where review is not warranted.
       • Select a reviewer set whose competences cover the manuscript, check conflicts before inviting, and justify the composition.
       • Brief reviewers with the question the editor needs answered, the model in operation, and the confidentiality and AI expectations.
       • Manage timelines, chase, and replace non-responsive reviewers without letting a manuscript stall.
       • Adjudicate split reviews by reading rather than counting: diagnose whether the disagreement is about facts, standards or taste, weigh a specific reproducible criticism above a general endorsement, and decide whether an additional review is needed.
       • Handle a bad review — intemperate, discriminatory, empty, incompetent, self-interested, or apparently AI-generated — by withholding or editing it, notifying the reviewer, replacing it, and recording the reason.
       • Write a decision letter that states the outcome, the reasoning, and which points a revision must address.
       • Determine an appeal on process rather than re-litigating the science, and state the grounds on which an appeal can succeed.
       • Escalate integrity matters out of the editorial track without attempting investigation.
       • Design a reviewer recognition scheme and advise reviewers on holding a durable independent record of their service.