How to Detect Publication Bias

Publication bias is not something a systematic review or meta-analysis can simply define its way around — it has to be actively checked for, using a specific set of statistical and visual methods, and the result of that check has to be reported (PRISMA 2020 and GRADE both require it). This guide covers how reviewers actually detect and assess publication bias in practice: reading a funnel plot, running Egger’s test, applying trim-and-fill, and correctly interpreting what a positive result does — and does not — prove. For the underlying concept itself (what publication bias is and why it happens), see the Publication Bias dictionary entry; this page picks up where that one leaves off.

Why detection is a required step, not an optional check

Publication bias is one of the five domains GRADE explicitly downgrades certainty of evidence for, alongside risk of bias, inconsistency, indirectness, and imprecision (Cochrane Handbook, Chapter 14, “Completing ‘Summary of findings’ tables”). PRISMA 2020 (Page MJ et al., BMJ. 2021;372:n71) requires reviewers to report whether and how they assessed the risk of bias due to missing results across studies. In practice, that means every meta-analysis needs a stated method — not just an assurance that a comprehensive search was run.

The Cochrane Handbook (Chapter 13, “Assessing risk of bias due to missing evidence in a meta-analysis”) is explicit that publication bias is one specific mechanism inside a broader “reporting bias” family that also includes outcome reporting bias, time-lag bias, language bias, and citation bias. The methods below detect the visible symptom — asymmetry or a gap in the distribution of observed effect sizes — without being able to distinguish which underlying mechanism produced it. That limitation runs through every method on this page and is the single most common way funnel-based detection gets over-interpreted.

Detection methods compared

Method	What it does	Minimum studies	What it tells you	What it does not tell you
Funnel plot (visual)	Scatter plot of each study’s effect estimate (x-axis) against a measure of its precision — typically standard error or sample size (y-axis, usually inverted so precise studies sit near the top)	No formal minimum, but fewer than ~10 studies makes any shape judgment unreliable	A quick, informal look at whether small, imprecise studies are distributed symmetrically around the pooled effect	Statistical significance of any asymmetry; visual judgment is subjective and prone to disagreement between reviewers
Egger’s test	Regresses each study’s standardized effect estimate on its precision; a statistically significant intercept indicates funnel-plot asymmetry	Cochrane guidance and the original methods literature recommend against using it below roughly 10 studies — power is very low at small k	A formal, reproducible test for the same asymmetry a funnel plot shows visually, replacing eyeballing with a p-value	The cause of asymmetry — a significant result is equally consistent with publication bias, genuine clinical/methodological heterogeneity between small and large trials, or chance
Trim-and-fill	A rank-based method (Duval & Tweedie, 2000) that estimates how many studies are “missing” from one side of an asymmetric funnel plot, imputes them, and recalculates a bias-adjusted pooled effect	Same small-study caveats as Egger’s test; performs poorly under substantial between-study heterogeneity	An adjusted effect estimate that shows how sensitive the pooled result is to assumed missing studies — useful as a sensitivity analysis	A “true,” corrected effect size. It assumes symmetry is the only expected pattern and that asymmetry has one cause (suppression), which is frequently not the case; Cochrane guidance is explicit that trim-and-fill results should be interpreted cautiously and are best reported alongside, not instead of, the unadjusted estimate

Reading a funnel plot: a worked example

Picture a funnel plot for a meta-analysis of 24 trials of some intervention, effect size (e.g. standardized mean difference) on the x-axis, standard error on the y-axis (inverted, so the largest, most precise trials sit near the top and the smallest, least precise trials spread out toward the bottom).

Symmetric funnel: small studies scatter roughly evenly on both sides of the pooled estimate as precision decreases, forming the inverted-funnel shape the method is named for. This is consistent with (though not proof of) an absence of small-study effects.
Asymmetric funnel: small, imprecise studies cluster preferentially on one side — for example, small studies showing a large positive effect are present, but small studies showing a null or negative effect are conspicuously sparse or absent, while large studies cluster tightly near the pooled estimate. This gap in the lower corner of the plot is the visual signature reviewers are looking for.

What an asymmetric funnel plot on its own does support: a signal that something correlated with study size is influencing the observed effect — worth investigating further with a formal test and worth reporting as a limitation regardless of cause.

What it does not prove by itself: that unpublished null studies exist, that they were suppressed deliberately, or that the pooled effect is wrong. The Cochrane Handbook lists several alternative explanations for the same visual pattern: genuine heterogeneity where smaller trials were conducted in different populations or with different protocols than larger ones, methodological differences correlated with study size (e.g. smaller trials having less rigorous methods and inflated effects for reasons unrelated to publication), or simply chance, especially with a modest number of studies. A reviewer who reports “the funnel plot was asymmetric, and Egger’s test was significant (p = 0.02)” without discussing these alternatives is over-claiming what the evidence supports.

Small-study effects: the more defensible framing

Because the mechanism behind funnel-plot asymmetry can’t be determined from the plot itself, methodologists increasingly favor the more neutral term small-study effects — the observed empirical pattern that smaller studies in a meta-analysis tend to show larger, more favorable effect sizes than larger studies — over asserting “publication bias” as the cause. Reporting asymmetry as a small-study effect and then discussing publication bias as one of several candidate explanations (alongside heterogeneity and methodological quality) is the more defensible way to write this section of a review, and is the framing GRADE’s own guidance points toward when downgrading certainty for suspected publication bias.

A practical assessment workflow

Check whether you have enough studies to test at all. With fewer than roughly 10 studies in the meta-analysis, Cochrane guidance recommends against relying on Egger’s test or trim-and-fill; state this limitation directly rather than running a statistically underpowered test and reporting its (likely null) result as reassurance.
Generate the funnel plot and inspect it visually for asymmetry, ideally with more than one reviewer independently forming a judgment before comparing notes.
Run Egger’s test (or an equivalent regression-based test) where the study count supports it, to move from a subjective visual read to a reproducible statistic.
If asymmetry is detected, run trim-and-fill as a sensitivity analysis, not a correction to be substituted for the primary result. Report both the original and the trim-and-fill-adjusted pooled estimate side by side.
Consider non-funnel-based evidence alongside the statistical tests: whether the review’s search included trial registries and grey literature (PRISMA 2020 requires this specifically to reduce the impact of unpublished studies), and whether registered-but-unpublished trials matching the review’s eligibility criteria were identified during screening.
Report the assessment explicitly in the GRADE Summary of Findings table, downgrading certainty for publication bias only where the totality of evidence — funnel asymmetry, a significant Egger’s test, and/or known unpublished eligible trials — supports it, and stating the reasoning rather than a bare downgrade.

Common mistakes

Treating a symmetric funnel plot as proof of no publication bias. A funnel plot with fewer than ~10 studies can look symmetric by chance; absence of visual asymmetry is weak evidence at best at low study counts.
Reporting Egger’s test on a meta-analysis with substantial heterogeneity without noting that heterogeneity itself is a documented cause of a false-positive asymmetry signal.
Presenting the trim-and-fill-adjusted estimate as the review’s headline result. Cochrane guidance treats it as a sensitivity check on the robustness of the primary finding, not a replacement for it.
Skipping the assessment entirely in a review with few included studies, rather than stating plainly that publication bias could not be formally assessed given the small number of studies — PRISMA 2020 expects the attempt and the limitation to be documented either way.

Frequently asked questions

What is the difference between publication bias and a funnel plot?

Publication bias is the underlying phenomenon — studies with certain results being more or less likely to be published. A funnel plot is one diagnostic tool used to look for a visual symptom (asymmetry) that is consistent with publication bias, among other possible causes. See the Publication Bias entry for the underlying concept.

How many studies do I need before I can test for publication bias?

There’s no universal cutoff, but both Egger’s test and trim-and-fill lose statistical power sharply below roughly 10 studies, and Cochrane guidance recommends against relying on either at that point. A funnel plot can still be generated and reported at lower counts, with the caveat that any shape judgment is unreliable.

Does an asymmetric funnel plot always mean publication bias?

No. Asymmetry is a generic “small-study effects” signal that can also result from genuine clinical or methodological heterogeneity between smaller and larger trials, or from chance. It should prompt further investigation and disclosure, not an automatic conclusion.

What is trim-and-fill actually correcting?

It doesn’t correct the underlying data — it imputes hypothetical “missing” studies to restore funnel symmetry and recalculates the pooled effect under that assumption, as a way of testing how sensitive the result is to a specific bias scenario. Duval and Tweedie’s original method (Biometrics. 2000;56(2):455–463) and subsequent guidance both caution against treating the adjusted estimate as more “true” than the original.

Is publication bias assessment required by PRISMA?

PRISMA 2020 requires reporting on methods used to assess risk of bias due to missing results, and on the results of that assessment, for any synthesis where it was performed. It does not mandate a specific statistical test, but it does require reviewers to state what they did.

How to Detect and Assess Publication Bias

Ask about How to Detect and Assess Publication Bias