Direct comparison
Bonferroni vs Tukey HSD: Which to Use
Bonferroni corrects any comparison set but costs power; Tukey HSD is exact for all-pairwise ANOVA. Formulas, assumptions, and a worked power comparison.
Written and maintained by CASRAI Editorial Board
Last updated
Ask CASRAI · free to try
Ask about Bonferroni vs Tukey HSD: Which to Use
Ask your first 2 questions free below. Subscribers get 150 a day for $29 a month.
An AI assistant specialized in research administration. It cites the sources behind every answer, labels web answers and says when it can't answer.
Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.
Works on this site and inside Claude, Cursor and the AI tools you already use.
Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays free to read, with no account and no card.
How do Bonferroni Correction, Tukey HSD compare side by side?
The table below compares Bonferroni Correction, Tukey HSD across 9 procurement-relevant dimensions, from what it controls through common software.
Side-by-side comparison
| Dimension | Bonferroni Correction | Tukey HSD |
|---|---|---|
| What it controls | Family-wise error rate across any set of m comparisons, via per-test alpha/m | Family-wise error rate specifically for all pairwise comparisons among k ANOVA group means |
| Comparison set it is valid for | Any set -- pairwise, planned contrasts, comparisons across different analyses | Only the full set of C(k,2) pairwise comparisons from one ANOVA |
| Distribution used | Student's t (or z), with alpha divided by m | Studentized range distribution (q) |
| Key assumptions | Whatever the underlying test assumes -- no equal-n or equal-variance requirement of its own | Normality, homogeneity of variance; classic form assumes equal n per group (Tukey-Kramer extends to unequal n) |
| Conservative for all-pairwise ANOVA? | Yes -- doesn’t exploit the correlation among the comparisons | No -- exact for that comparison set |
| Power (worked simulation, k=6, n=15/group) | 35.1% avg power across true pairwise differences (5% FWER target, actual 2.0%) | 43.0% avg power, same data (5% FWER target, actual 5.0%) |
| Typical use case | A small, pre-specified subset of comparisons, or comparisons mixing different test types | Post hoc test after a significant one-way ANOVA, comparing every group to every other group |
| Where it breaks down | Overly conservative (loses real power) when applied to a large all-pairwise set | Invalid outside the all-pairwise ANOVA context -- do not use it to correct an arbitrary subset of comparisons |
| Common software | p.adjust(method="bonferroni") in R; "Bonferroni" post hoc option in SPSS/Stata | TukeyHSD() in R; "Tukey" post hoc option in SPSS/Stata; PROC GLM ADJUST=TUKEY in SAS |
Common questions
Common questions about Bonferroni Correction vs Tukey HSD
Is Bonferroni ever more powerful than Tukey HSD?
+
Yes -- if you're only testing a handful of pre-specified comparisons rather than the full all-pairwise set, Bonferroni's threshold (alpha divided by that smaller number of tests) can end up less conservative than Tukey's, since Tukey's critical value is fixed for the entire k-group family regardless of how many comparisons you actually report. For a small planned subset, Bonferroni is usually the better choice.
What do I use if group sizes are unequal?
+
The classic Tukey HSD formula assumes equal n per group. The Tukey-Kramer procedure extends it to unequal group sizes and is what most statistical software (R’s TukeyHSD(), SPSS’s Tukey post hoc option) implements by default.
What if group variances are unequal?
+
Neither Bonferroni-adjusted t-tests nor Tukey HSD in their standard form assume unequal variances. When Levene's test flags unequal variances, use Games-Howell for all-pairwise comparisons, or Bonferroni-adjusted Welch t-tests for a smaller pre-specified set.
