The Breslow-Day Test: Checking Homogeneity Before You Pool Odds Ratios

The Breslow-Day test checks whether the odds ratio linking an exposure to an outcome is the same across the strata of a confounding or effect-modifying variable — that is, whether a single pooled odds ratio is even a valid summary of the data. It answers a narrower, earlier question than the Mantel-Haenszel method does: before you compute OR_MH and report one confounder-adjusted number, is “one number” the right thing to report at all, or do the strata actually disagree with each other about the size (or even the direction) of the effect? Run the Breslow-Day test first. Pool with Mantel-Haenszel only once it doesn’t reject.

What the Breslow-Day test actually tests

The null hypothesis is homogeneity: OR₁ = OR₂ = … = OR_k, one true common odds ratio underlying all k strata, with the observed stratum-specific differences attributable to sampling variation alone. The alternative is that at least one stratum’s true odds ratio differs from the others — in epidemiological language, that the stratifying variable is an effect modifier (sometimes called a source of statistical interaction) rather than purely a confounder to adjust away. Breslow & Day (1980, IARC Scientific Publications No. 32) built the test around a specific comparison: for each stratum, how far is the observed exposed-outcome-positive cell count from the count you’d expect in that stratum if one common odds ratio really did hold everywhere? Summing a standardized version of that gap across strata gives a chi-square statistic. A large statistic is evidence against homogeneity — evidence the strata don’t share one true effect.

Why this has to come before the pooled estimate, not after

The Mantel-Haenszel formula computes OR_MH = Σ(a_id_i/n_i) ÷ Σ(b_ic_i/n_i) regardless of whether the underlying stratum odds ratios agree with each other — the arithmetic runs and produces a number no matter what the data actually look like. That number is only interpretable as “the common odds ratio, adjusted for the confounder” if a common odds ratio genuinely exists. If the strata disagree — if the exposure’s effect is real but different sizes in different strata, or points in different directions — the Mantel-Haenszel weighted average is a blend of unlike quantities, similar in spirit to reporting a single average temperature for a day that was freezing in the morning and hot by noon: arithmetically valid, substantively misleading. Reporting one adjusted OR in that situation doesn’t just lose precision, it actively hides the finding that the stratifying variable changes the exposure’s effect — often the more interesting, more actionable result. The Breslow-Day test is the formal check for exactly this failure mode, run on the same stratified 2×2 tables before the pooled number is trusted, not as an afterthought once it’s already been reported. See confounding variable for the underlying distinction between a variable to adjust for and one that modifies the effect itself, and how to interpret an odds ratio for what the pooled number means once homogeneity actually holds.

The test statistic, step by step

For each stratum i, label the 2×2 table exposed/outcome+ = a_i, exposed/outcome− = b_i, unexposed/outcome+ = c_i, unexposed/outcome− = d_i, with row total R_1i = a_i+b_i, column total C_1i = a_i+c_i, and stratum total n_i. Using ψ = OR_MH (the pooled estimate) as the working value of the common odds ratio, the expected count of the a-cell, A_i, is the root of a quadratic that falls inside the table’s valid range:

(1−ψ)A_i² + [(n_i−R_1i−C_1i) + ψ(R_1i+C_1i)]A_i − ψ·R_1i·C_1i = 0

The other three expected cells follow from the same margins: B_i = R_1i−A_i, C_i = C_1i−A_i, D_i = n_i−R_1i−C_1i+A_i. The variance of A_i under that common-OR model is the reciprocal of the sum of reciprocals of the four expected cells: Var(A_i) = [1/A_i + 1/B_i + 1/C_i + 1/D_i]⁻¹. The Breslow-Day statistic sums the squared, standardized gap between what was actually observed and what homogeneity would predict:

X²_BD = Σ_i (a_i − A_i)² / Var(A_i)

Under the null, X²_BD is approximately chi-square distributed with k−1 degrees of freedom (one fewer than the number of strata — the same df convention already used on the Mantel-Haenszel page’s own homogeneity check). Standard 5% critical values: 3.84 at 1 df, 5.99 at 2 df, 7.81 at 3 df.

The Tarone correction

Tarone (1985, “On heterogeneity tests based on efficient scores,” Biometrika 72(1), 91–95) showed the plain Breslow-Day statistic above is a slightly biased estimate of its own asymptotic chi-square reference distribution, because ψ was itself estimated from the same data (as OR_MH) rather than known in advance. The correction subtracts a term built from the same quantities already computed:

X²_Tarone = X²_BD − [Σ_i(a_i−A_i)]² / Σ_iVar(A_i)

still referred to the same k−1 df chi-square. The two statistics are usually close and rarely disagree on the conclusion at conventional significance levels, but the Tarone form is the more defensible one to report, particularly with sparse per-stratum counts — it’s the default most software actually implements (R’s DescTools::BreslowDayTest() takes it as an option; SAS’s PROC FREQ CMH statement with the BRESLOWDAY option reports both the plain and Tarone-adjusted statistics side by side).

Worked example: a case where the pooled number should not be trusted

Illustrative example — a fixed-seed simulated dataset, not drawn from a real study, generated so the true stratum-specific odds ratios are known in advance rather than estimated. Every figure below is independently computed and reproducible from that script (mulberry32 PRNG, seed 290829901), not chosen to look plausible.

Four strata were constructed with deliberately unequal true odds ratios — 1.0, 1.8, 3.2, and 6.0 — simulating a case where an exposure’s effect genuinely strengthens across the stratifying variable (a case-control design, cases and controls sampled per stratum, exposure status drawn from stratum-specific probabilities implied by each true OR):

Stratum	Exposed, case (a)	Unexposed, case (b)	Exposed, control (c)	Unexposed, control (d)	Stratum n	Observed OR
1	63	157	65	155	440	0.96
2	116	124	73	167	480	2.14
3	110	90	48	152	400	3.87
4	120	60	43	137	360	6.37

The Mantel-Haenszel pooled estimate runs without complaint: OR_MH = 2.50, 95% CI [2.04, 3.06] (Robins-Breslow-Greenland SE(ln OR) = 0.103) — a plausible-looking number that a reader with no further check would report as “the confounder-adjusted odds ratio.” Using ψ = 2.50 to compute each stratum’s expected a-cell under homogeneity:

Stratum	Observed a	Expected A (ψ=2.50)	Var(A)	(a−A)²/Var(A)
1	63	84.24	20.94	21.6
2	116	120.23	26.99	0.7
3	110	100.47	22.52	4.0
4	120	101.57	21.14	16.1

X²_BD = 42.3 on 3 df, p < 0.0001 (Tarone-corrected: X²_Tarone = 42.3, same conclusion) — nowhere close to the 7.81 critical value, decisively rejecting homogeneity. Stratum 1’s observed count of 63 sits 21 events below its expected 84 under the pooled model; stratum 4’s sits 18 above. The Breslow-Day test is correctly flagging exactly the pattern the data were built to contain: this is not one confounder-adjusted effect, it’s four different effects. The honest next step is not to report OR_MH = 2.50 as “the” adjusted odds ratio — it’s to report the stratum-specific ORs (0.96, 2.14, 3.87, 6.37) on their own terms, describe the stratifying variable as an effect modifier rather than a pure confounder, and, if a formal test of the interaction itself is wanted, fit it directly with an exposure×stratum interaction term in logistic regression rather than leaning on Breslow-Day’s yes/no signal alone.

A second case: few strata, and what a non-significant result does and doesn’t mean

Illustrative example, same method, independent fixed-seed simulation (seed 290829902) with only two strata and a genuine common true OR of 2.0 built into the data-generating process.

Stratum	Exposed, case (a)	Unexposed, case (b)	Exposed, control (c)	Unexposed, control (d)	Observed OR
1	63	87	42	108	1.86
2	80	70	45	105	2.67

OR_MH = 2.24, 95% CI [1.60, 3.13]. With only two strata, the Breslow-Day test has just 1 df: X²_BD = 1.09, p = 0.30 — correctly non-significant here, since the data really were built to share one common OR. But report this kind of result carefully: with 1 df the test has limited power to detect real heterogeneity when only two or three strata are available, so a non-significant Breslow-Day result with few strata is evidence consistent with homogeneity, not proof of it. Don’t state “the Breslow-Day test confirms homogeneity” from a small number of strata; state that it found no evidence against it, which is a weaker and more accurate claim. This same caveat is echoed on the Mantel-Haenszel guide’s own FAQ, from the pooling side of the same question.

Running the test in practice

• R: DescTools::BreslowDayTest(x, correct = TRUE) on a 2×2×k array reports both the plain and Tarone-corrected statistics; correct = TRUE applies the Tarone adjustment. The epitools and epiDisplay packages implement equivalent versions.
• SAS: PROC FREQ with a three-way table and TABLES stratum*exposure*outcome / CMH BRESLOWDAY; reports the Breslow-Day and Tarone-adjusted chi-squares alongside the Cochran-Mantel-Haenszel statistic in the same output block.
• Sparse strata: the chi-square approximation assumes reasonably large expected cell counts. With many thin strata (small n per stratum, expected cells under 5), the asymptotic p-value becomes unreliable in the same way a standard chi-square test does with sparse tables — prefer an exact conditional test (available in StatXact-style exact procedures, or via simulation) over trusting the chi-square approximation blindly, or coarsen the stratification if the strata were cut finer than the confounder genuinely needs.

Common pitfalls

• Skipping straight to Mantel-Haenszel. The formula computes a number regardless of whether pooling is valid — nothing about the arithmetic itself will warn you that the strata disagree. Run Breslow-Day (or at minimum inspect the stratum-specific ORs and their confidence intervals for obvious non-overlap) before reporting OR_MH as a single adjusted effect.
• Reading “not significant” as “proved homogeneous,” especially with few strata. See the second worked example above — a non-rejection with 1 or 2 df is weak evidence, not confirmation.
• Treating heterogeneity as a nuisance to test away rather than a finding. When Breslow-Day rejects, the substantively interesting result is often the effect modification itself (the exposure works differently across the stratifying variable), not just a disqualified pooled number. Report the stratum-specific effects, don’t just note that pooling “wasn’t appropriate” and move on.
• Using the asymptotic chi-square with genuinely sparse per-stratum counts. Small expected cells (any A_i, B_i, C_i, or D_i under roughly 5) make the chi-square approximation unreliable; an exact test is the safer choice in that regime.

Frequently asked questions

What does it mean if the Breslow-Day test is significant?

It means the data provide statistical evidence against a single common odds ratio across strata — the stratifying variable behaves as an effect modifier for this exposure-outcome relationship, not (only) a confounder. It does not, by itself, say which stratum differs from which, or by how much; that requires looking at the stratum-specific odds ratios and their confidence intervals directly.

Can I still report a Mantel-Haenszel pooled OR if Breslow-Day rejects?

Not as a single “the adjusted effect” number without a strong caveat. The statistically honest options are to report the stratum-specific odds ratios instead, to fit a model with an explicit interaction term so the effect modification itself is estimated and tested, or to report the pooled OR alongside the stratum-specific estimates with an explicit statement that the strata differ — never to report OR_MH alone as if homogeneity had been confirmed.

Is Breslow-Day the same as testing for interaction in a regression model?

They test a closely related question with different machinery. Breslow-Day is a stratified, non-model-based chi-square test built directly from the 2×2 tables; a likelihood-ratio or Wald test on an exposure×covariate interaction term in logistic regression tests essentially the same substantive hypothesis but within a regression framework that can include continuous covariates and multiple confounders simultaneously. For a single categorical stratifying variable and hand-checkable stratified data, they typically agree closely.

What’s the smallest number of strata the test works with?

Two, with 1 df — but see the power caveat above. There’s no upper limit on the number of strata other than the usual sparse-data caveat once individual strata get thin.

See also: the Mantel-Haenszel method for the pooling step this test gates, how to interpret an odds ratio, confidence interval for an odds ratio, confounding variable, case-control study, and the Research Methods & Statistics pillar.