Skip to main content
v2026.11,986 entries · CC-BY 4.0

AMSTAR 2: The 16-Item Tool for Appraising Systematic Reviews

AMSTAR 2 is the standard tool for appraising the methodological quality of a systematic review. Here is what its 16 items assess and how the confidence rating is calculated.

Written and maintained by CASRAI Editorial Board

Published · Last updated

On this page

AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews, version 2) is a 16-item checklist for appraising the methodological quality of a systematic review of randomized and/or non-randomized studies of healthcare interventions. Seven items are critical domains; the number of weaknesses in them, not a summed score, sets the overall confidence rating: High, Moderate, Low or Critically Low.

It does not appraise the individual studies inside a review — that is what tools like the Risk of Bias 2 (RoB 2) or Newcastle-Ottawa Scale are for. AMSTAR 2 instead appraises the review itself: whether its methods — the search, study selection, data extraction, risk-of-bias assessment, and synthesis — were rigorous enough to trust its conclusions. It was developed by Beverley Shea and colleagues and published in BMJ on 21 September 2017 (358:j4008, PMID 28935701; open access at PMC5833365). It updates the original 2007 AMSTAR (11 items, built around reviews of randomized trials) so that it also covers reviews that include non-randomized studies. For appraising the individual studies inside a review across many designs, see JBI critical appraisal checklists.

The data table below this guide lists which items are critical and which response options each offers.

Ask CASRAI · free to tryQuestion about this guide? Ask CASRAI answers with linked sources. 2 free questions a day, no account.

Ask your first question freeSubscribe

Who developed AMSTAR 2 and what it is for

The 2017 paper was written by Beverley J. Shea, Barnaby C. Reeves, George Wells, Micere Thuku, Candyce Hamel, Julian Moran, David Moher, Peter Tugwell, Vivian Welch, Elizabeth Kristjansson and David A. Henry. The authors describe the instrument as a tool for critically appraising systematic reviews of healthcare interventions, whether the review includes randomized trials, non-randomized studies of interventions, or both. The authors also state that it is not intended to generate an overall numeric score; the output is a judgement about confidence in the review’s results.

The 16 AMSTAR 2 items

The items below follow the wording of Shea et al. (2017). Critical domains are marked, and the response options for each item are in the data table below.

  1. Item 1. Did the research questions and inclusion criteria include the components of PICO (population, intervention, comparator, outcome)?
  2. Item 2 (critical). Did the report contain an explicit statement that the review methods were established prior to conduct of the review, and did it justify any significant deviations from the protocol?
  3. Item 3. Did the review authors explain their selection of the study designs for inclusion?
  4. Item 4 (critical). Did the review authors use a comprehensive literature search strategy?
  5. Item 5. Did the review authors perform study selection in duplicate?
  6. Item 6. Did the review authors perform data extraction in duplicate?
  7. Item 7 (critical). Did the review authors provide a list of excluded studies and justify the exclusions?
  8. Item 8. Did the review authors describe the included studies in adequate detail?
  9. Item 9 (critical). Did the review authors use a satisfactory technique for assessing the risk of bias in the individual studies included in the review?
  10. Item 10. Did the review authors report on the sources of funding for the studies included in the review?
  11. Item 11 (critical). If a meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results?
  12. Item 12. If a meta-analysis was performed, did the review authors assess the potential impact of risk of bias in individual studies on the results of the meta-analysis or other evidence synthesis?
  13. Item 13 (critical). Did the review authors account for risk of bias in individual studies when interpreting or discussing the results of the review?
  14. Item 14. Did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review?
  15. Item 15 (critical). If they performed quantitative synthesis, did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?
  16. Item 16. Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?

Items 11, 12 and 15 depend on a meta-analysis or quantitative synthesis. On the published checklist form, the meta-analysis items that are critical (items 11 and 15) offer a “No meta-analysis conducted” response so that a narrative-synthesis review is not marked down for a step it never performed.

Response options and what counts as Partial Yes

Each item is answered Yes, Partial Yes or No. A published form copy of the checklist (the Nordic Nutrition Recommendations version) shows which items have a Partial Yes option: items 2, 4, 7, 8 and 9. The remaining tabulated items are Yes/No only, and items 11 and 15 add “No meta-analysis conducted”. The data table below sets these out item by item. Criteria stated on that form for several items:

  • Item 2. Partial Yes requires a written protocol or guide that includes the review question(s), a search strategy, inclusion and exclusion criteria, and a risk-of-bias assessment. Yes additionally requires a registered protocol, a meta-analysis or synthesis plan where appropriate, a plan for investigating causes of heterogeneity, and justification for any deviations from the protocol.
  • Item 4. Partial Yes requires searching at least two databases relevant to the research question, giving keywords and/or a search strategy, and justifying any publication restrictions (for example, language).
  • Items 5 and 6 (duplicate work). The standard is that two reviewers independently agreed on selection (item 5) or extraction (item 6), or that two reviewers worked on a sample with acceptable agreement (the form gives 80 percent) and one reviewer completed the remainder.
  • Item 7. Partial Yes means the authors provided a list of all potentially relevant studies read in full text but excluded from the review; Yes means each exclusion is also justified.
  • Item 9 (randomized trials). Partial Yes requires that risk of bias was assessed from unconcealed allocation and from lack of blinding of patients and assessors when outcomes were subjective. Blinding is not required for objective outcomes such as all-cause mortality. Yes adds that bias from the allocation sequence not being truly random, and from selective reporting of outcomes, was also assessed.
  • Item 15. Yes requires graphical or statistical tests for publication bias and a discussion of its likely impact.

The form copy is a secondary reproduction; check the criteria against the AMSTAR website (amstar.ca) before using a stricter reading in a published appraisal.

The AMSTAR 2 Confidence Rating

Seven of the 16 items — items 2, 4, 7, 9, 11, 13, and 15 — are designated critical domains, because a weakness in any one of them can substantially undermine confidence in the review’s conclusions even if every other item is answered well. The overall rating is driven by how many critical and non-critical weaknesses exist, not by a simple item count of the whole checklist:

Rating Criteria
High No or one non-critical weakness: the review provides an accurate and comprehensive summary of the available studies.
Moderate More than one non-critical weakness, and no critical flaws.
Low One critical flaw, with or without non-critical weaknesses.
Critically Low More than one critical flaw, with or without non-critical weaknesses.

Because the rule counts weaknesses rather than adding points, an appraisal reported as, say, “11 of 16” is not an AMSTAR 2 result. The authors advise against computing an overall numeric score.

The 7 critical items

The seven critical domains are: item 2 (review methods established before the review, with deviations justified), item 4 (adequacy of the literature search), item 7 (a list of excluded studies with justification), item 9 (adequacy of the technique for assessing risk of bias in individual studies), item 11 (appropriateness of the meta-analytical methods, if a meta-analysis was performed), item 13 (whether risk of bias in individual studies was accounted for when interpreting the review’s results), and item 15 (adequacy of the investigation of publication bias). The remaining nine items — items 1, 3, 5, 6, 8, 10, 12, 14 and 16 — cover things like whether the review question and inclusion criteria were specified in advance, whether study selection and data extraction were done in duplicate, and whether conflicts of interest were reported. A weakness in one of these alone lowers confidence but does not, by itself, cap the rating at Low.

Shea and colleagues present the critical-domain list as a default. Appraisers may add or substitute domains when a review’s context warrants it; for instance, a review that includes only high-quality randomized trials may make some critical items less decisive. If you change the list, state the change in your methods so readers can see how the rating was reached.

Worked example of the rating logic

The following is an illustrative scenario, not a real review. An appraiser reads a systematic review of an intervention and finds: the search covered several databases with a stated strategy (item 4 Yes); there was a protocol with a registration (item 2 Yes); excluded full-text studies were listed with reasons (item 7 Yes); risk of bias was assessed with a suitable instrument (item 9 Yes); there was a meta-analysis with appropriate methods (item 11 Yes), but results were interpreted without regard to the risk of bias in the included trials (item 13 No) and no investigation of publication bias was reported (item 15 No). That is two critical weaknesses, so the confidence rating is Critically Low, regardless of how the non-critical items were answered. If the same review had investigated publication bias but still ignored risk of bias in its discussion, it would have one critical flaw and be rated Low. A review with no critical flaws and a single non-critical weakness (for example, funding of included studies not reported) would be rated High.

AMSTAR vs. AMSTAR 2

The original AMSTAR (2007, 11 items) was built for and validated against systematic reviews of randomized controlled trials. AMSTAR 2 (2017, 16 items) keeps the same underlying logic — appraising the review’s methods, not its findings — but restructures the items, adds explicit critical domains and the four-tier confidence rating, and extends coverage to systematic reviews that include non-randomized studies of interventions, which the original tool did not adequately handle. AMSTAR 2 is the current, actively used version; AMSTAR 1 is largely of historical interest.

Scope and limitations

Shea et al. designed AMSTAR 2 for reviews of healthcare interventions. They state that it is not designed for diagnostic test accuracy reviews, individual participant data meta-analyses, network meta-analyses, scoping reviews or realist reviews, and that it does not prescribe which risk-of-bias instrument a review should use; item 9 asks whether the technique used was satisfactory. The authors also note that they did not perform an extensive validation of the new tool.

Later methodological work has pointed to practical issues. A 2022 study in Systematic Reviews (11:147, PMC9308914) examined inter-rater reliability and usability. It found that strict yes/no responses can force a “No” for items the review simply under-reports, which may push many reviews toward the lowest ratings. It reported agreement ranging from fair (items 1, 12 and 13) to almost perfect (items 2, 4, 6, 7, 15 and 16), so a single rater’s judgement on the lower-agreement items deserves a second opinion.

How AMSTAR 2 ratings look in practice

A 2025 methodological study by Lunny and colleagues (Research Synthesis Methods 17(1):63–92, doi 10.1017/rsm.2025.10032) applied AMSTAR 2 to 200 systematic reviews. The overall confidence ratings were High for 39 reviews (19.5%), Moderate for 15 (7.5%), Low for 21 (10.5%) and Critically Low for 125 (62.5%). In other words 146 of 200 reviews (73%) were rated Low or Critically Low. Appraisal took a median of 51 minutes (interquartile range 26–67) per review when AMSTAR 2 was the first tool applied. Budget for that time, and for a second rater, when you plan an overview of reviews.

AMSTAR 2 vs ROBIS

AMSTAR 2 and ROBIS (Risk Of Bias In Systematic reviews) both appraise systematic reviews but ask different questions: AMSTAR 2 assesses methodological quality and the confidence one can place in the review’s results, whereas ROBIS assesses risk of bias in the review’s conclusions. In the same 200-review study, ROBIS rated 162 of 200 reviews (81%) high risk of bias, and the two tools gave discordant overall judgements for 18 reviews (9%). The authors concluded that the tools are complementary rather than interchangeable, so the choice should follow the purpose of the appraisal. The median ROBIS appraisal time in that study was 64 minutes.

Reporting an AMSTAR 2 appraisal

This is editorial advice rather than part of the tool. When AMSTAR 2 appears in an overview of reviews or a guideline’s evidence table, it helps readers if you (1) use at least two independent raters and say how disagreements were resolved, (2) record a short reason for every Partial Yes and No, (3) show the per-item judgements in a table beside the overall rating, and (4) state any change to the default critical-domain list. Report the overall rating as one of the four categories, not as a score.

For choosing between AMSTAR 2 and the other major appraisal tools — CASP, Newcastle-Ottawa, and RoB 2 — by study design, see CASRAI’s Choosing a Critical Appraisal Tool guide, which places AMSTAR 2 as the recommended tool specifically for appraising a systematic review of interventions. For a design-specific checklist walkthrough, see CASP checklists and how to apply them.

Frequently Asked Questions

What does AMSTAR 2 stand for?

A MeaSurement Tool to Assess systematic Reviews, version 2. It appraises the methodological quality of a systematic review itself, not the individual studies inside it.

How many items are in the AMSTAR 2 checklist?

16 items, seven of which are designated critical domains (items 2, 4, 7, 9, 11, 13, and 15) that drive the overall confidence rating.

Who developed AMSTAR 2?

Beverley Shea and colleagues, published in BMJ in 2017 (358:j4008).

What is the difference between AMSTAR and AMSTAR 2?

The original 2007 AMSTAR had 11 items and was built for reviews of randomized controlled trials only. AMSTAR 2, published in 2017, has 16 items, adds explicit critical domains and a four-tier confidence rating, and extends coverage to reviews that include non-randomized studies of healthcare interventions.

What are the four AMSTAR 2 confidence ratings?

High, Moderate, Low, and Critically Low. One critical flaw gives Low and more than one gives Critically Low, regardless of how the non-critical items were rated; with no critical flaw the rating is High (no or one non-critical weakness) or Moderate (more than one).

Can I calculate an AMSTAR 2 score?

No. The authors advise against an overall numeric score; report the confidence category and the per-item answers.

Does AMSTAR 2 appraise individual studies or the review as a whole?

The review as a whole — its search strategy, study selection process, data extraction, risk-of-bias assessment methods, and synthesis. Appraising the individual studies included in the review requires a separate tool matched to each study’s design, such as RoB 2 for randomized trials or Newcastle-Ottawa for observational studies.

Can AMSTAR 2 be used for network meta-analyses or scoping reviews?

It was not designed for them. The authors name diagnostic test accuracy reviews, individual participant data meta-analyses, network meta-analyses, scoping reviews and realist reviews as outside its intended scope.

How is AMSTAR 2 different from PRISMA?

PRISMA is a reporting guideline that tells review authors what to report; AMSTAR 2 is an appraisal tool used by readers to judge the quality of a completed review. A review can follow PRISMA reporting and still receive a low AMSTAR 2 rating, for example if it never registered its protocol.

How long does an AMSTAR 2 appraisal take?

In one study of 200 reviews, the median was 51 minutes per review when AMSTAR 2 was the first tool applied.

Related reading: Choosing a Critical Appraisal Tool for Your Study Design — a side-by-side comparison of CASP, AMSTAR 2, Newcastle-Ottawa, and RoB 2, and which one to use for which study design.

See also: Risk of Bias Assessment: RoB 2, ROBINS-I and the Traffic-Light Plot — the tools for appraising the individual studies an AMSTAR 2-rated review is built from.

Sources

Sources (accessed 2026-11-15):

Data: AMSTAR 2 items: critical status and response options

Item wording and critical-domain status from Shea et al. 2017 (BMJ); response options from the published AMSTAR 2 checklist form.

Period covered: 2017.

AMSTAR 2 items: critical status and response options
Item numberDomainCritical domainPartial Yes option on formResponse options on form
1PICO in research questions and inclusion criteriaNon-criticalNoYes / No
2Methods established before the review; deviations justifiedCriticalYesYes / Partial Yes / No
4Comprehensive literature search strategyCriticalYesYes / Partial Yes / No
5Study selection in duplicateNon-criticalNoYes / No
6Data extraction in duplicateNon-criticalNoYes / No
7List of excluded studies with justificationCriticalYesYes / Partial Yes / No
8Included studies described in adequate detailNon-criticalYesYes / Partial Yes / No
9Satisfactory risk-of-bias technique for individual studiesCriticalYesYes / Partial Yes / No
10Funding sources of included studies reportedNon-criticalNoYes / No
11Appropriate meta-analytic methods (if meta-analysis done)CriticalNoYes / No / No meta-analysis conducted
13Risk of bias accounted for when interpreting resultsCriticalNoYes / No
14Heterogeneity explained and discussedNon-criticalNoYes / No
15Adequate investigation of publication biasCriticalNoYes / No / No meta-analysis conducted
16Review authors' conflicts of interest and funding reportedNon-criticalNoYes / No

Sources

Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about AMSTAR 2: The 16-Item Tool for Appraising Systematic Reviews

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →