Risk of Bias Assessment: RoB 2, ROBINS-I and the Traffic-Light Plot

The RoB 2 tool (Risk of Bias 2) is Cochrane’s standard instrument for assessing risk of bias in randomized trials, organized into five bias domains answered through structured signalling questions; ROBINS-I is the parallel tool for non-randomized studies of interventions, organized into seven domains including confounding. Both produce domain-level and overall judgements that are then typically visualized as a traffic-light plot — a color-coded grid of studies (rows) by domains (columns) — most commonly generated with the free robvis tool. This guide explains which tool to use for a given study design, walks through each domain, and covers how to build and read the plot.

RoB 2 vs. ROBINS-I: which tool for which study design

	RoB 2	ROBINS-I
Study design it assesses	Randomized controlled trials (individually or cluster-randomized)	Non-randomized studies of interventions — cohort, case-control, controlled before-after, interrupted time series
Number of domains	5	7
Judgement categories per domain	Low risk of bias / Some concerns / High risk of bias	Low / Moderate / Serious / Critical risk of bias / No information
Assessment mechanism	Signalling questions (yes/probably yes/no/probably no/no information) feed a fixed algorithm that generates the domain judgement	Signalling questions feed a fixed algorithm; also requires specifying confounders and co-interventions the assessor expects a priori
Governing publication	Sterne JAC, Savović J, Page MJ, et al. “RoB 2: a revised tool for assessing risk of bias in randomised trials.” BMJ 2019;366:l4898	Sterne JA, Hernán MA, Reeves BC, et al. “ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions.” BMJ 2016;355:i4919
Maintained by	Cochrane Methods (Bias Methods Group)	Cochrane Methods, jointly with the original ROBINS-I development group
Cluster/crossover variants	Yes — separate RoB 2 versions exist for cluster-randomized and crossover trials	ROBINS-I has a companion tool, ROBINS-E, for non-randomized studies of exposures rather than interventions

Decision flow: which tool does your review need?

1. Were participants randomly allocated to comparison groups? If yes, go to RoB 2 (or its cluster-randomized / crossover variant, matched to your design).
2. If no randomization — the study is observational or quasi-experimental (cohort, case-control, controlled before-after, interrupted time series) and compares an intervention to a comparator — use ROBINS-I.
3. If the “exposure” isn’t an intervention decision (e.g. environmental or occupational exposure rather than something assigned or chosen as treatment), the relevant tool is ROBINS-E, not ROBINS-I — a distinct but related instrument outside the scope of this page.
4. Both tools are recommended by the Cochrane Handbook for use within a formal systematic review, and both are compatible with reporting the result in a PRISMA-structured synthesis.

RoB 2: the five domains

RoB 2 assesses each trial result (not just the trial as a whole — a single trial reporting multiple outcomes may need a separate RoB 2 assessment per outcome, since bias can differ by outcome) against five domains:

1. Bias arising from the randomization process. Was allocation sequence generation and concealment adequate, and were baseline imbalances between groups consistent with chance?
2. Bias due to deviations from intended interventions. Did participants or trial personnel know the assigned intervention during the trial, were there deviations from the intended intervention that arose because of the trial context, and were they balanced between groups and analyzed on an intention-to-treat basis?
3. Bias due to missing outcome data. Were outcome data available for nearly all participants, and if not, is there evidence the result wasn’t biased by the missingness?
4. Bias in measurement of the outcome. Was the method of measuring the outcome appropriate, and could knowledge of the intervention received have influenced outcome assessment (particularly relevant for subjective, non-blinded outcome measures)?
5. Bias in selection of the reported result. Was the reported result selected from among multiple outcome measurements or multiple analyses of the same outcome, based on the results?

Each domain is answered through a fixed set of signalling questions (answered Yes / Probably Yes / Probably No / No / No Information), and a built-in algorithm — not the assessor’s free judgement — maps the signalling-question answers to a domain-level rating of Low risk, Some concerns, or High risk of bias. The overall risk-of-bias judgement for that result is then also algorithmic: overall is Low only if every domain is rated Low; overall is High if any single domain is rated High, or if several domains raise Some concerns in a way that substantially lowers confidence in the result; otherwise overall is Some concerns.

ROBINS-I: the seven domains

Because non-randomized studies lack the protection randomization gives against confounding, ROBINS-I adds two domains RoB 2 doesn’t need, ahead of the domains that parallel RoB 2’s structure:

1. Bias due to confounding. Were there baseline differences between intervention groups, caused by factors other than the intervention itself, that affected the outcome? This is the domain with no RoB 2 equivalent, since randomization is designed to prevent it.
2. Bias due to selection of participants into the study. Did the way participants were selected into the analysis (including start-of-follow-up decisions) depend on both the intervention and the outcome?
3. Bias in classification of interventions. Was intervention status classified accurately, using information collected at or before the start of the intervention?
4. Bias due to deviations from intended interventions. Same concept as RoB 2’s equivalent domain, adapted for a non-randomized context.
5. Bias due to missing data. Were outcome, exposure, and confounder data reasonably complete, and is the analysis’s handling of missing data unlikely to bias the result?
6. Bias in measurement of outcomes. Could knowledge of the intervention have influenced outcome measurement?
7. Bias in selection of the reported result. Same selective-reporting concept as RoB 2’s final domain.

Before assessing confounding, ROBINS-I requires the assessor to pre-specify, from the review protocol, which confounding domains and co-interventions matter for the comparison being assessed — this is a distinguishing step RoB 2 has no equivalent of, and skipping it is a common way ROBINS-I assessments get done inconsistently across a review team.

Domain- and overall judgements use a five-level scale: Low, Moderate, Serious, Critical risk of bias, or No information. As with RoB 2, the overall judgement for a result is driven by its single worst domain rather than an average across domains — a study rated Critical on any one domain is rated Critical overall, since a Critical rating (e.g. inadequately controlled confounding severe enough to invalidate the comparison) is treated as invalidating the result rather than merely discounting it.

The traffic-light plot: what it is and how to build one

A traffic-light plot is a grid with one row per study and one column per bias domain, where each cell is a colored circle — conventionally green for Low risk, yellow for Some concerns / Moderate, and red/orange shades for High / Serious / Critical risk — showing every study’s domain-level judgements at a glance, alongside an overall-judgement column. It’s the standard way RoB 2 and ROBINS-I results are reported in a published systematic review, since a table of text judgements for a review with dozens of studies is hard to scan, and the plot makes patterns (e.g. most studies failing on the same domain) immediately visible.

The plot is almost always produced with robvis (Risk-Of-Bias VISualization), the tool purpose-built for this: McGuinness LA, Higgins JPT. “Risk-of-bias VISualization (robvis): An R package and Shiny web app for visualizing risk-of-bias assessments.” Research Synthesis Methods 2021;12(1):55–61. It’s available two ways:

• As a free web app (no coding required) — upload a spreadsheet of your domain-level judgements in the template format robvis expects, and it renders the traffic-light plot and a weighted summary bar plot directly in the browser.
• As an R package (robvis, installable from CRAN) — for reviewers who want the plot generated as part of a reproducible analysis script, or who need to customize colors, ordering, or output format beyond what the web app exposes.

Either route takes the same input: one row per study, one column per domain, populated with the judgement category (not raw signalling-question answers — those are answered first, in RevMan, the RoB 2/ROBINS-I Excel templates, or another data-collection tool, and only the resulting domain judgements go into robvis) plus a final overall-judgement column. robvis recognizes the RoB 2 and ROBINS-I domain naming and judgement-category conventions natively, so a correctly labeled spreadsheet needs no reformatting.

Common pitfalls

• Treating “some concerns” as a middle score to average out. Neither tool’s overall judgement is an average of domain scores — a single High/Serious/Critical domain can determine the overall rating regardless of how well the other domains scored, so tallying “3 green, 2 yellow” and calling it moderate is not how the algorithm actually works.
• Assessing the trial instead of the result. RoB 2 in particular is meant to be applied per outcome-and-analysis, not once per trial — a trial can be well-conducted for its primary outcome and higher risk for a secondary one measured differently.
• Skipping ROBINS-I’s pre-specification step. Jumping straight to the confounding domain without having the review team agree in advance on the confounders and co-interventions that matter is a documented source of inter-rater disagreement.
• Confusing risk-of-bias assessment with GRADE certainty rating. Risk of bias is one of five domains GRADE downgrades certainty for, alongside inconsistency, indirectness, imprecision, and publication bias — RoB 2/ROBINS-I feed into that GRADE domain, but a GRADE certainty rating is a separate, review-level judgement made after risk-of-bias assessment across all included studies, not a synonym for it.

Frequently asked questions

Is RoB 2 the same as the original Cochrane risk-of-bias tool?

No. RoB 2 is a full revision of the original Cochrane risk-of-bias tool for randomized trials (sometimes called “RoB 1” in retrospect), restructured around the current five domains and signalling-question mechanism. Cochrane Reviews are now expected to use RoB 2 rather than the original tool.

Can RoB 2 or ROBINS-I be used outside a Cochrane review?

Yes. Both are general-purpose tools recommended in the Cochrane Handbook but not restricted to Cochrane-branded reviews — they’re widely used in systematic reviews published in any journal, and referenced directly by PRISMA-family reporting guidance for the quality-appraisal step.

What tool assesses risk of bias in a scoping review?

Scoping reviews typically don’t include a formal risk-of-bias assessment at all, since their purpose is mapping the scope and nature of existing literature rather than synthesizing an effect estimate — see our step-by-step scoping-review guide for how that differs from a full systematic review.

Do I need both RoB 2 and ROBINS-I in the same review?

If a review includes both randomized and non-randomized studies of the same intervention, yes — each study is assessed with the tool matching its own design, and the two sets of judgements are typically reported separately (often in separate traffic-light plots) rather than merged into one scale, since the judgement categories aren’t numerically equivalent across tools.

Related guides

• How to Conduct a Scoping Review — the JBI/PRISMA-ScR process, and why scoping reviews skip formal risk-of-bias assessment.
• Rapid Review vs. Systematic Review — how a rapid review’s methodology, including its quality-appraisal step, differs from a full systematic review.
• How to Detect and Assess Publication Bias — the related GRADE domain covering whether a study enters the literature at all.
• How to Register a Systematic Review on PROSPERO — where risk-of-bias tool choice is typically specified in the protocol.
• How to Peer Review a Systematic Review or Meta-Analysis — what a reviewer checks in a submitted review’s risk-of-bias assessment.
• Fixed-Effect vs. Random-Effects Meta-Analysis — the pooling-model decision that follows risk-of-bias assessment when studies proceed to meta-analysis.
• How to Read a Forest Plot — interpreting the pooled-estimate output that risk-of-bias assessment feeds into.

Last verified 2026-08-16 against the Cochrane Methods risk-of-bias resource pages, the RoB 2 and ROBINS-I primary publications in BMJ, and the robvis primary publication in Research Synthesis Methods. Domain names, judgement-category labels, and tool-selection criteria reflect the current published versions of RoB 2 and ROBINS-I as of this date; consult the Cochrane Methods pages directly before an assessment, since both tools receive periodic implementation guidance updates.