Heterogeneity in meta-analysis is variation among the true effects being estimated by the individual studies included in a synthesis — variation that goes beyond what would be expected from chance (sampling error) alone. Every meta-analysis pools studies that differ in some ways; the question heterogeneity forces is whether they differ in ways that make averaging them together misleading. This guide covers how heterogeneity is defined, how it’s quantified (Q, I², τ², and the prediction interval), what fixed-effect and random-effects models actually assume about it, how to investigate its causes without manufacturing false findings, when not to pool at all, and what PRISMA 2020 expects reviewers to report. It assumes you’re already familiar with the basics of systematic review methodology.
Three kinds of heterogeneity, not one
The Cochrane Handbook distinguishes three related but distinct concepts, and conflating them is one of the most common errors in how heterogeneity gets discussed:
- Clinical heterogeneity (clinical diversity) — variability in the participants, interventions, and outcomes studied. A trial of a drug in adults and a trial of the same drug in children are clinically diverse even if both are well-designed.
- Methodological heterogeneity (methodological diversity) — variability in study design, outcome measurement instruments, and risk of bias across the included studies.
- Statistical heterogeneity — variability in the intervention effect sizes actually observed across studies, over and above what random error would produce.
The relationship between the three matters as much as the definitions themselves: statistical heterogeneity is a symptom, not a root cause. It is, in the Cochrane Handbook’s framing, a consequence of clinical or methodological diversity, or both — and it can be very difficult to tell which is driving it, because in most real reviews it’s some mix of both. A meta-analyst who reports an I² value without asking what clinical or methodological difference produced it has described the symptom and stopped short of the diagnosis.
Quantifying statistical heterogeneity
Three statistics and one interval are used, in practice, alongside each other — none of them is sufficient on its own, and all four are typically reported directly on or beside a forest plot.
Cochran’s Q (the chi-squared test)
Q tests whether the differences in observed results across studies are compatible with chance alone: a large Q relative to its degrees of freedom (equivalently, a low p-value) is evidence of heterogeneity. But Q is a poor primary decision tool for two opposite reasons:
- Underpowered with few studies. Most meta-analyses include a small number of studies, and Q has low power to detect real heterogeneity in that situation — a non-significant Q must not be read as evidence of no heterogeneity.
- Over-sensitive with many studies. With a large number of included studies, Q has high power to detect even a small, clinically unimportant amount of heterogeneity, flagging it as “significant” regardless of whether it matters practically.
Because of this, some methodologists use a more lenient p < 0.10 threshold for Q rather than the conventional 0.05, and few treat Q’s p-value as a standalone verdict either way.
I²: a proportion, not an amount
I² describes the percentage of total variability in effect estimates that is due to heterogeneity rather than sampling error (chance). That distinction is the single most common misreading of I² in published papers: I² is not a measure of how much the true effects vary in absolute terms — it’s the proportion of the observed spread that isn’t explained by chance. A high I² can occur with a trivially small absolute difference in true effects if the individual studies are very precise (large sample sizes), and a low I² can occur alongside meaningful absolute variation if the studies are individually imprecise.
The Cochrane Handbook offers rough interpretation bands, with an explicit warning attached to them:
- 0–40%: might not be important
- 30–60%: may represent moderate heterogeneity
- 50–90%: may represent substantial heterogeneity
- 75–100%: considerable heterogeneity
Note that these bands deliberately overlap — they are not a lookup table with clean cutoffs. The Handbook is explicit that the importance of any given I² value depends on the magnitude and direction of the effects involved and the strength of evidence for heterogeneity (for example, a confidence interval for I² itself), and that I² can be unreliable when the number of included studies is small. Treat these bands as a rough vocabulary for describing a result, not a decision rule for whether to pool.
τ² (tau-squared): the between-study variance
τ² is the estimated variance of the true effects across studies — the parameter a random-effects model is actually estimating when it assumes studies don’t share one identical true effect but instead have effects drawn from a distribution. Its square root, τ, is the estimated standard deviation of that distribution, expressed on the same scale as the effect measure itself (e.g. log odds ratio, mean difference). Unlike I², τ² is not scale-free, which is precisely why it’s harder to interpret in isolation — a τ² of 0.02 means something different for a mean difference in millimetres than for a mean difference measured in years. This is the main reason methodologists increasingly report a prediction interval alongside it instead of leaving τ² to speak for itself.
The prediction interval
Where a confidence interval around a pooled random-effects estimate expresses uncertainty about the average true effect, a prediction interval expresses the range within which the true effect of a single new, similar study would be expected to fall. It’s built from both the pooled estimate’s uncertainty and τ², so it directly communicates the practical consequence of heterogeneity: how much the effect might vary if you tried this intervention in one more comparable setting. Many methodologists now consider the prediction interval the single most informative summary of heterogeneity precisely because it answers the question a reader actually has — “how much could this differ if applied to my population?” — in a way neither Q, I², nor τ² does on its own. It is only reliable as a summary when a reasonable number of studies are included (commonly cited as five or more); with very few studies, the interval itself becomes unstable and should be interpreted cautiously or omitted.
Fixed-effect vs. random-effects models
The choice of model is not a statistical afterthought — it’s an assumption about the world the heterogeneity statistics above are meant to test.
- Fixed-effect model: assumes every included study is estimating the exact same single true effect, and that any observed differences between studies are due entirely to sampling error. This is a strong assumption that is rarely defensible once real clinical or methodological diversity exists across studies.
- Random-effects model: assumes the true effect varies from study to study, drawn from a distribution of effects, and estimates the mean of that distribution along with τ². It produces a wider confidence interval than a fixed-effect analysis of the same data whenever heterogeneity is present, and gives relatively more weight to smaller studies than a fixed-effect model would.
Given that most real-world sets of studies differ at least somewhat in populations, interventions, or methods, random-effects is usually the more honest default once heterogeneity is present or plausible — but it is important to be precise about what it does. A random-effects model does not remove, correct for, or “fix” heterogeneity. It changes how heterogeneity is accounted for in the pooled estimate’s uncertainty (wider intervals, different weighting), but the underlying clinical or methodological diversity that’s driving the heterogeneity is still there, unexplained, in the data. Switching models is not a substitute for investigating why the studies disagree.
Investigating the causes of heterogeneity
Once heterogeneity is identified, the substantive question is what’s causing it. Four tools are used, and the way they’re used matters as much as which one is chosen:
- Subgroup analysis — splitting included studies into groups by a study-level characteristic (e.g. dose, population age band, study design) and comparing pooled effects across groups.
- Meta-regression — modeling the study-level effect estimate as a function of one or more study-level covariates, useful when there are enough studies to support it and the covariate of interest is continuous or there are several candidate moderators.
- Sensitivity analysis — re-running the pooled analysis under different assumptions or with different subsets of studies included (e.g. excluding studies at high risk of bias) to see whether the result is robust.
- Outlier and influence diagnostics — identifying individual studies whose removal would materially change the pooled estimate or the heterogeneity statistics, distinct from simply excluding studies that look inconvenient.
The hard rule governing all four: these investigations must be pre-specified in the review protocol — registered, for example, in PROSPERO — or explicitly reported as exploratory, post-hoc analyses. Post-hoc subgroup analysis, run after seeing which split produces a “significant” or interesting-looking result, is a well-recognised source of false and non-replicable findings; testing enough subgroups eventually finds one that looks meaningful by chance alone. This is the same multiplicity problem that undermines unregistered subgroup analysis in individual clinical trials, and it applies with at least as much force when the “subgroups” are entire studies. A meta-analysis that reports several subgroup or meta-regression results without having pre-specified them, or without flagging them as exploratory, is not offering a finding — it’s offering a hypothesis that happens to be dressed as a result.
When not to pool at all
Quantifying and investigating heterogeneity is not a guarantee that pooling is the right call. If heterogeneity is severe, and the causes cannot be adequately explained by the subgroup, meta-regression, or sensitivity analyses available, forcing a single pooled effect estimate can actively mislead readers — it reports a number that doesn’t correspond to any real, coherent population or intervention. In that situation, a narrative synthesis, or a structured approach such as Synthesis Without Meta-analysis (SWiM), is a more honest way to report the evidence than an average that papers over incompatible studies. The decision not to pool is a legitimate methodological outcome, not a failure to complete the analysis — and reviewers assessing a submitted meta-analysis should treat unexplained, severe heterogeneity combined with a pooled estimate and no discussion of why pooling was still appropriate as a red flag, not a formatting detail.
PRISMA 2020 reporting expectations
PRISMA 2020 requires that a systematic review’s methods section state which statistical methods were used to synthesize results and how heterogeneity was assessed, and that the results section report the heterogeneity findings themselves — not just a pooled effect estimate in isolation. Where subgroup, meta-regression, or sensitivity analyses were used to explore possible causes of heterogeneity, PRISMA 2020 expects those investigations and their results to be reported explicitly, including whether they were pre-specified. A PRISMA-compliant synthesis methods section is, in practice, the place where a reader should be able to tell whether heterogeneity was measured, interpreted correctly, investigated appropriately, and — where it mattered — allowed to change the conclusions drawn. Peer reviewers evaluating a submitted meta-analysis should check heterogeneity handling as a distinct item — see CASRAI’s guide on how to peer review a systematic review or meta-analysis for the full checklist.
Software: where these statistics come from in practice
- R — the
metaforpackage (Viechtbauer) computes Q, I², τ², and prediction intervals as part of itsrma()fitting functions and supports meta-regression and subgroup models directly; themetapackage offers a more guided workflow for standard pairwise meta-analysis with the same statistics reported by default. - RevMan (Cochrane’s own review-authoring software) reports Q, I², and τ² on every forest plot it generates and is the tool most Cochrane reviews are produced in.
- Stata — the
metasuite of commands (built into modern Stata) and the long-standing user-writtenmetancommand both report the standard heterogeneity statistics and support subgroup and meta-regression analysis.
Frequently asked questions
Is a high I² always a problem?
Not automatically. A high I² means a large share of the observed variability isn’t due to chance, but whether that matters depends on the absolute size of the variation, whether it can be explained by pre-specified clinical or methodological differences, and how the resulting prediction interval looks. A well-explained, high-I² result with a pre-specified subgroup structure that resolves most of the variation is a stronger paper than an unexplained low-I² result that got there by chance with few studies.
Can you have heterogeneity even if I² is 0%?
Yes. I² can be low or zero simply because there are too few studies for it to be estimated reliably, not because the true effects are actually consistent. This is exactly why the Cochrane Handbook cautions against a mechanical cutoff and why a non-significant Cochran’s Q should never be read as proof of homogeneity.
Does random-effects modeling let you ignore heterogeneity?
No. Choosing a random-effects model changes how uncertainty is estimated and reported (wider intervals, different study weighting) — it does not explain, resolve, or reduce the underlying clinical or methodological diversity producing the heterogeneity. Investigating the causes remains a separate, necessary step.
What’s the difference between a confidence interval and a prediction interval around a pooled estimate?
A confidence interval expresses uncertainty about the average true effect across the population of studies. A prediction interval expresses the range within which the true effect of one new, comparable study would plausibly fall — it is almost always wider, and it’s the more relevant number for a reader deciding whether a pooled result applies to their own setting.
When should subgroup analysis results be treated with caution?
Whenever they weren’t pre-specified in the protocol. A subgroup finding discovered by testing several possible splits after seeing the data carries a real risk of being a chance finding dressed up as a discovery, regardless of how plausible the biological or clinical story sounds afterward.
