Q-Values and the Positive False Discovery Rate: The FDR Analog of a P-Value

A q-value is the false-discovery-rate analog of a p-value: where a p-value answers "how surprising is this one result under the null," a q-value answers "if I called every test with a q-value this small or smaller ‘significant,’ what fraction of that whole list would be false positives?" It was introduced by John Storey in 2002 for exactly the setting where a single p-value threshold stops being useful — genome-wide screens, multi-outcome trials, or any study running hundreds or thousands of simultaneous tests, where controlling the chance of even one false positive (the p-value/Bonferroni approach) throws away too much power to be usable.

What a q-value actually measures

Formally, the q-value for a given test is the minimum false discovery rate (FDR) at which that test would be included in the list of "significant" results. Rank every p-value from smallest to largest. As you move down the ranked list including more and more tests, the expected proportion of false positives among the included tests rises. The q-value of test i is the smallest FDR you would have to accept, at the threshold set at test i‘s own p-value, to include it. A test with q = 0.02 means: if you drew the significance line right at this test’s p-value, an estimated 2% of everything above the line would be a false positive.

This is a different quantity from what the p-value alone tells you. A p-value is about one test in isolation, computed as if it were the only test run. A q-value is a property of the whole batch of tests plus this test’s rank inside that batch — the same p-value can produce a smaller or larger q-value depending on how many other tests were run alongside it and how many of them also looked significant.

"Positive FDR" — the technical distinction in the name

The false discovery rate, as originally defined by Yoav Benjamini and Yosef Hochberg in 1995, is E[V/R · 1{R>0}] — the expected proportion of false discoveries among all rejections, with the convention that this proportion is counted as zero on the (possible) occasions where no test is rejected at all. Storey’s 2002 paper defined a close relative, the positive false discovery rate (pFDR), as E[V/R | R>0] — the same ratio, but conditioned on at least one rejection actually happening, rather than averaged over runs that reject nothing. The q-value is defined in terms of the pFDR, not the plain FDR. In practice, for any batch of tests where at least one result clears the threshold (the only case where the question "what’s my false discovery rate?" is even being asked), the two coincide numerically — the distinction is about how the quantity is defined and estimated at the boundary, not a different number you’ll see printed in a results table.

Q-value vs. BH-adjusted p-value: the mix-up this term causes

Most statistical software does not implement Storey’s original q-value by default. R’s p.adjust(method = "BH") and Python’s statsmodels.stats.multitest.multipletests(method='fdr_bh') compute the Benjamini-Hochberg adjusted p-value — for the test ranked i out of m, the smallest FDR threshold at which it would be rejected under the BH step-up procedure, assuming every null hypothesis is exactly true (formally, assuming π₀, the true proportion of null tests, equals 1). Software and papers routinely call this output "the q-value" — and for reporting purposes, treating it as one is standard, defensible practice. See CASRAI’s Bonferroni correction and multiple comparisons guide for the BH step-up procedure worked in full.

Storey’s original q-value differs in one specific way: instead of assuming π₀ = 1 (that every test is a true null until proven otherwise), it estimates π₀ from the shape of the p-value distribution itself — roughly, from how many p-values land in the "flat," uninformative region well above zero — and scales the adjustment by that estimate. When most of your tests really are null (π₀ close to 1, the sparse-signal case typical of an exploratory screen), the two methods produce nearly identical numbers, because the assumption BH makes is nearly true. When a large share of your tests carry a real effect (π₀ well below 1), Storey’s q-value is systematically smaller than the BH-adjusted p-value for the same test, because it is no longer paying the tax of assuming nulls that the data show aren’t there. The simulation below shows both regimes side by side, with the actual numbers.

A reproducible simulation: Bonferroni vs. BH-FDR vs. Storey’s q-value

The results in this section come from a synthetic, seeded simulation run for this page — not from a real study or dataset. The generating code (a mulberry32 pseudo-random generator, seeded, with a Box-Muller normal draw and an Abramowitz-Stegun erf approximation for exact two-sided p-values) is reproducible from the seed given for each scenario; re-running it with the same seed returns exactly these numbers.

Setup: 1,000 simultaneous z-tests. A known subset are "true effects" (drawn from a normal shifted by the stated amount); the rest are true nulls (standard normal). All three procedures are run at a nominal 5% error rate: Bonferroni at family-wise α = 0.05 (threshold 0.05/1,000), BH step-up targeting FDR = 0.05, and Storey’s q-value (with λ = 0.5 for the π₀ estimate, a simplified single-point version of the method — the reference qvalue Bioconductor package uses a smoothed estimate over a range of λ values, which is more stable but not reproducible in a few lines here).

Read honestly, not selectively: Scenario A is the clean story — at a realistic 5% true-effect rate, BH-FDR more than doubles Bonferroni’s power (0.360 vs. 0.160) at the cost of one false positive out of nineteen calls. But Scenario B shows the limit of that story: when the true effect size is too weak relative to the noise, neither method finds anything, and FDR control buys zero power advantage over Bonferroni — being less conservative does not manufacture signal that isn’t there. Scenario C, a re-seed of the exact same setup as A, landed on zero realized false positives for both FDR methods purely by chance — a reminder that the FDR is a long-run expectation, not a per-run guarantee; one run controlling it perfectly (or one run running hot) doesn’t confirm or refute the method. Scenario D is where Storey’s q-value actually earns its distinction from the plain BH-adjusted p-value: with 40% of tests carrying a real effect (π₀ ≈ 0.6, correctly estimated by the simulation at 0.612), the q-value’s π₀-adjustment recovered 323 true positives against BH’s 304 — a real, if modest, power gain from not assuming every test is null when the data say otherwise, purchased with a slightly higher false-positive count (16 vs. 13) and a realized FDP (0.047) still under the 0.05 target.

How to compute and report q-values

In R, the qvalue Bioconductor package (by Storey and colleagues) computes the full method, including the smoothed π₀ estimate; base R’s p.adjust(pvalues, method = "BH") gives the simpler BH-adjusted p-value most papers actually report and call a q-value. In Python, statsmodels.stats.multitest.multipletests(pvals, method='fdr_bh') is the equivalent BH computation; a standalone qvalue package exists for the full Storey method.

When reporting, state explicitly: which method produced the number ("BH-adjusted p-values" vs. "q-values via the Storey method" are not interchangeable phrases, even though the numbers are often close), the software and package/function used, and the target FDR threshold applied (0.05 and 0.10 are both common in genomics; there is no single conventional default the way 0.05 functions for a p-value). If π₀ was estimated (true Storey q-values), note the estimation approach, since a poorly chosen λ or an unstable spline fit can shift the reported q-values meaningfully in a small or unusually shaped dataset.

Common misreadings of a q-value

A q-value inherits the interpretation traps of a p-value and adds one of its own. It is not the probability that this specific finding is a false positive, full stop — it is an estimate of the proportion of false positives among all tests at least this extreme, a batch-level quantity being reported at the level of one test. Two papers can report the identical q-value for the identical effect and mean different things by it if one ran 500 tests and the other ran 50,000 — the q-value depends on the whole distribution of p-values in the batch, not on this test’s effect size or precision alone. And a small q-value says nothing about effect size or practical importance; with a large enough m or a large enough sample size per test, trivially small effects can clear a stringent FDR threshold the same way they can clear a p < 0.05 threshold. See CASRAI’s statistical significance guide for the broader version of that problem, and the data dredging entry for what happens when the set of tests being FDR-corrected wasn’t actually pre-specified.

When FDR/q-value control beats Bonferroni — and when it doesn’t

The decision rule itself, worked in more depth with its own examples, lives on CASRAI’s Bonferroni correction and multiple comparisons guide. In short: q-value/FDR control is the right tool for large-scale exploratory screening — genome-wide association studies, differential-expression panels (see CASRAI’s RNA-seq experimental design guide for where q-values sit in that pipeline), or any multi-outcome study generating a shortlist for follow-up rather than certifying each result individually. Bonferroni (or Holm-Bonferroni) remains the right tool for a small, pre-specified set of confirmatory comparisons — primary and key secondary endpoints in a trial, a handful of planned pairwise tests — where a single false positive is the failure mode that matters and family-wise error control, not FDR control, is what the audience (a regulator, a journal, a pre-registration) actually expects.

Assumptions and limits

The BH procedure and the q-value both assume the p-values behave well under independence or a specific positive-dependence structure (PRDS); strong negative correlation among tests can make the nominal FDR control unreliable, though BH tends to be robust in practice for the kinds of correlation structures common in genomics and multi-outcome trials. π₀ estimation additionally assumes the p-values from true nulls are uniformly distributed on [0,1] — a assumption that fails if the underlying test statistic’s null distribution is misspecified (a common issue with poorly calibrated test statistics from complex pipelines), which will bias the π₀ estimate and, downstream, every q-value computed from it. Neither method changes the underlying data or effect sizes; both only change which threshold a given result has to clear to be called significant.

Frequently asked questions

What is a q-value in statistics?

A q-value is the minimum false discovery rate at which a given test’s result would be called significant — the FDR analog of a p-value, used when many tests are run simultaneously.

What’s the difference between a q-value and an adjusted p-value?

In common usage, none — the Benjamini-Hochberg adjusted p-value most software produces is routinely called a q-value. Strictly, Storey’s original q-value additionally estimates the proportion of true nulls (π₀) rather than assuming it’s 1, which can produce a smaller, more powerful number when a substantial share of tests carry a real effect.

What does "positive false discovery rate" mean?

It’s the false discovery rate conditioned on at least one test being called significant — Storey’s 2002 refinement of Benjamini and Hochberg’s original 1995 FDR definition. The two coincide numerically whenever at least one rejection actually occurs, which is the only case where reporting a false discovery rate is meaningful in practice.

Can a q-value be smaller than the raw p-value?

No — a q-value (or BH-adjusted p-value) is never smaller than the raw p-value for the same test; the correction only ever makes significance harder to achieve, never easier.

Is a q-value of 0.05 the same as a p-value of 0.05?

No. A p-value of 0.05 describes one test in isolation. A q-value of 0.05 describes an estimated 5% false-discovery rate among every test at least as extreme as this one in the current batch — it depends on how many tests were run and how many looked significant, not on this test’s evidence alone.

How do I calculate q-values in R or Python?

In R, p.adjust(p, method = "BH") for the standard BH-adjusted value, or the Bioconductor qvalue package for Storey’s full method with π₀ estimation. In Python, statsmodels.stats.multitest.multipletests(pvals, method='fdr_bh') computes the BH equivalent.