CASRAI Dictionary

Tag: meta-analysis

  • Systematic Review vs Meta-Analysis: The Difference Explained

    A systematic review is a structured, protocol-driven synthesis that identifies, appraises and summarises all studies meeting pre-specified criteria. A meta-analysis is an optional statistical step within or after such a review that pools the numerical results into a single combined estimate. Every meta-analysis should rest on a systematic review, but not every systematic review contains a meta-analysis.

    Two related but distinct things

    The terms are often used interchangeably, which causes real confusion. The systematic review is the method: a comprehensive search, transparent selection, risk-of-bias assessment and synthesis. The meta-analysis is one possible synthesis technique — combining effect estimates statistically to gain precision and to quantify how consistent the studies are. A review may instead use a narrative or structured qualitative synthesis when pooling is not appropriate.

    Feature Systematic review Meta-analysis
    What it is Structured synthesis of all eligible studies Statistical pooling of study results
    Output Narrative or quantitative summary, evidence tables Combined effect estimate with confidence interval
    Always needs the other? No — can stand alone Yes — should rest on a systematic review
    Key risk Incomplete or biased search Pooling heterogeneous or incomparable studies
    Visual artefact PRISMA flow diagram, evidence tables Forest plot, funnel plot

    PRISMA reporting underpins both

    Whether or not pooling occurs, the review should be reported to the PRISMA 2020 standard. PRISMA’s checklist and flow diagram make the search, selection and synthesis auditable. When a meta-analysis is performed, PRISMA additionally expects the synthesis methods, the model used, and the handling of heterogeneity and certainty to be reported.

    Heterogeneity: the decisive question

    The central judgement in any meta-analysis is whether the studies are similar enough to combine. Heterogeneity describes the variability in true effects across studies, beyond what chance alone would produce. Reviewers assess it visually and with statistics such as the I² and the χ² test for heterogeneity. High heterogeneity warns that a single pooled number may be misleading — combining apples and oranges produces a fruit salad, not an average. Where studies differ in populations, interventions or outcomes, a random-effects model, subgroup analysis or a decision not to pool at all may be the honest choice.

    Forest plots and reading the result

    The signature output of a meta-analysis is the forest plot. Each study appears as a point estimate with a confidence interval, sized by its weight, and the pooled estimate sits at the bottom, often as a diamond whose width is its confidence interval. A funnel plot, meanwhile, is used to inspect for small-study effects and possible publication bias. These plots are how readers see, at a glance, both the central estimate and the spread of the evidence behind it.

    When meta-analysis is — and isn’t — appropriate

    Pooling is appropriate when studies ask a comparable question, measure comparable outcomes and are methodologically sound enough that a combined estimate is meaningful. It is inappropriate when heterogeneity is high and unexplained, when studies are at high risk of bias, or when outcomes are not commensurable. In those cases a rigorous systematic review with a narrative synthesis is the stronger contribution. For more, see our research-lifecycle coverage, the CASRAI dictionary, and how reviews fit the hierarchy of evidence.

    Frequently asked questions

    Is a meta-analysis always better than a systematic review?

    No. A meta-analysis adds statistical precision only when the underlying studies are comparable and sound. Pooling heterogeneous or biased studies produces a precise but misleading number. A careful systematic review without pooling is often the more honest result.

    What does heterogeneity tell me?

    It tells you how much the true effects vary across studies beyond chance. High, unexplained heterogeneity is a signal to investigate sources of variation and to question whether a single pooled estimate is meaningful.

    What is a forest plot?

    A forest plot displays each study’s effect estimate and confidence interval alongside the pooled result, letting readers see both the combined estimate and the consistency of the evidence at a glance.

    Do both follow the same reporting standard?

    Yes. Both follow PRISMA 2020, with extra synthesis items reported when a meta-analysis is conducted. See our author guidance for preparing a compliant manuscript.

  • Crediting contributions in systematic reviews and meta-analyses

    A systematic review looks, from the outside, like a single coherent document with a tidy list of authors. From the inside it is a small project with a remarkable division of labour: a protocol to register, a search strategy to design and run across multiple databases, thousands of records to screen against eligibility criteria, full texts to retrieve and assess, data to extract twice over, risk-of-bias judgements to make, a synthesis or meta-analysis to compute, and a report to write to an exacting standard. Each of those tasks is a distinct skill, and each is usually done by a different person or pair of people. The conventional author byline flattens all of it. This article looks at how structured reporting through PRISMA and structured contributorship through the CRediT taxonomy together make the real shape of this work visible, and where the vocabulary for it sits in the credit extensions domain of the CASRAI Dictionary.

    Why a review is hard to credit fairly

    The difficulty is that the most laborious and methodologically critical parts of a review are precisely the ones that leave no trace in a traditional byline. Screening twenty thousand abstracts in duplicate is exacting, consequential work — get the eligibility judgements wrong and the whole review is compromised — yet it is invisible in author order. The same is true of designing a reproducible search, performing duplicate data extraction, or making risk-of-bias assessments. Meanwhile, the person who conceived the question and the person who drafted the manuscript are easy to recognise. A fair account of a review has to name the unglamorous, high-stakes tasks as clearly as the visible ones.

    PRISMA: reporting the process transparently

    The first half of the answer is methodological transparency. PRISMA — Preferred Reporting Items for Systematic Reviews and Meta-Analyses — is the reporting guideline that tells readers what a review actually did: how the search was constructed, how records moved from identification through screening to inclusion (the familiar flow diagram), how data were extracted, and how studies were appraised and synthesised. PRISMA does not assign credit, but it makes the work auditable. When a review reports its process to the PRISMA standard, the existence and scale of each task — the searching, the screening, the extraction, the appraisal — becomes explicit rather than implied. That visibility is the precondition for crediting it: you cannot recognise a contribution that the reporting has hidden.

    CRediT: naming who did what

    The second half is contributorship. The Contributor Roles Taxonomy provides a controlled vocabulary of contribution types that maps unusually well onto the anatomy of a review. The full set is set out in our overview of the CRediT roles, but several are worth singling out for evidence synthesis:

    • Conceptualization — formulating the review question and eligibility criteria.
    • Methodology — designing the search strategy and the synthesis approach, often the work of an information specialist.
    • Investigation — running the searches, screening records and retrieving full texts.
    • Data curation — managing the extracted data, de-duplication and the records that underpin the flow diagram.
    • Formal analysis — the meta-analysis itself, including heterogeneity assessment and any sensitivity analyses.
    • Writing – original draft and Writing – review & editing — producing and refining the manuscript.

    Used together, these roles let a review record that the information specialist designed the search, that two named reviewers screened and extracted in duplicate, and that the statistician ran the synthesis — rather than leaving all of it to be guessed from author order. The wider CRediT taxonomy turns the division of labour into a machine-readable statement attached to the output.

    The role information specialists deserve

    One contribution that systematic reviews chronically under-credit is that of the information specialist or research librarian who designs and validates the search. A poorly constructed search undermines a review more surely than almost any other flaw, and a well-constructed one is a genuine methodological achievement. Recording this work explicitly under Methodology and Investigation — rather than relegating it to an acknowledgement — is one of the clearest practical gains from applying contributorship to evidence synthesis. It names a contribution that is both critical and routinely invisible.

    Crediting duplicate work without double-counting

    Reviews rely on tasks done independently by two people — duplicate screening, duplicate extraction — precisely to reduce error. Contributorship should reflect that both reviewers did the work, which CRediT handles naturally by allowing a role to be assigned to more than one contributor. The honest principle, as ever, is that a role records what a person actually did: both screeners earn the Investigation role because both genuinely screened, not as a courtesy. This is the same standard that applies across all contribution recording — credit follows real work, and is neither inflated for visibility nor withheld for convenience.

    A consistent record across systems

    Systematic reviews increasingly register protocols, deposit search strategies and data, and publish in journals that require both PRISMA reporting and a contributorship statement. For that ecosystem to work, the way a contribution is described has to mean the same thing wherever it appears. That consistency is what the CASRAI Dictionary exists to provide: a stable vocabulary so that a Methodology contribution declared in a protocol registry, a manuscript and an institutional record can be recognised as the same claim. Combined with PRISMA’s transparency about process, structured contribution makes the substantial, distributed work of evidence synthesis legible — crediting the screeners, extractors and search designers whose labour holds a review together, not only the names at the top of the list.