The chi-square test (χ² test) is a statistical test used to analyze categorical data — data sorted into named groups or categories rather than measured on a numeric scale. It asks whether the pattern of counts observed across those categories differs from what would be expected under a specific null hypothesis. It is one of the most widely used tests in research because so much data collected in surveys, clinical records, and experiments is categorical: yes/no, treatment group, disease status, education level, and similar variables.
The counts and percentages a chi-square test analyzes are the same categorical frequency distributions described in CASRAI’s guide to descriptive statistics.
This is a fundamentally different job from a t-test or ANOVA, which compare means of a continuous outcome across groups. The chi-square test compares frequencies — how many observations fall into each category — not averages. If your outcome is a number (blood pressure, reaction time, test score), you generally want a t-test, ANOVA, or regression. If your outcome is a category (recovered/not recovered, pass/fail, party affiliation), the chi-square family is usually the starting point. See CASRAI’s overview of statistical tests and types of variables for how this choice fits into the broader landscape of test selection.
The two main forms of the chi-square test
“Chi-square test” is often used loosely to mean either of two related but distinct procedures. Both use the same underlying chi-square distribution and the same core comparison of observed counts to expected counts, but they answer different questions.
1. Chi-square test of independence
The test of independence asks whether two categorical variables are associated with each other, using data laid out in a contingency table (also called a cross-tabulation). The null hypothesis is that the two variables are independent — knowing a case’s category on one variable tells you nothing about its category on the other.
Illustrative example (not real data): Suppose a researcher records smoking status (smoker / non-smoker) and diagnosis of a respiratory condition (present / absent) for 200 participants in a hypothetical study:
| Condition present | Condition absent | Row total | |
|---|---|---|---|
| Smoker | 45 | 35 | 80 |
| Non-smoker | 30 | 90 | 120 |
| Column total | 75 | 125 | 200 |
The chi-square test of independence would compare these observed counts to the counts expected if smoking status and diagnosis were unrelated, and produce a test statistic and p-value indicating whether the association seen in this sample is stronger than would be expected by chance alone.
2. Chi-square goodness-of-fit test
The goodness-of-fit test asks whether the distribution of a single categorical variable matches a hypothesized or expected distribution. There is no second variable and no contingency table — just one set of categories and a claim about what proportion of observations should fall into each.
Illustrative example (not real data): A hypothetical study checks whether survey respondents were recruited evenly across four recruitment channels, where the researchers expected an equal 25% split across each channel among 160 respondents:
| Channel | Observed (O) | Expected (E) |
|---|---|---|
| 50 | 40 | |
| Social media | 28 | 40 |
| Flyers | 44 | 40 |
| Referral | 38 | 40 |
The goodness-of-fit test compares each observed count to its expected count and produces a single statistic summarizing how far the whole distribution departs from the expected one.
Observed vs. expected frequencies, and degrees of freedom
Both forms of the test rest on the same comparison: for every cell in the table, compare the observed frequency (O) — the actual count — to the expected frequency (E) — the count predicted under the null hypothesis. The test statistic is:
χ² = Σ (O − E)² / E
summed across every cell in the table. Larger deviations between observed and expected counts push χ² higher; a χ² near zero means the data look almost exactly like what the null hypothesis predicts.
For a test of independence, the expected count in any cell of an r-row by c-column table is calculated from the table’s own margins:
E = (row total × column total) / grand total
The degrees of freedom for an r×c contingency table are (r − 1)(c − 1). For the 2×2 smoking example above, df = (2−1)(2−1) = 1. For a goodness-of-fit test with k categories, df = k − 1 (or fewer, if additional parameters had to be estimated from the data to generate the expected proportions).
Assumptions, and what to do when they don’t hold
The chi-square test’s p-value is only valid when several conditions are reasonably met:
- Independence of observations. Each subject or unit contributes to exactly one cell of the table — nobody is counted twice, and observations aren’t paired or repeated-measures. This is the assumption most often violated in practice, typically when the same participants are measured at two time points or under two conditions and their pre/post or matched counts are (incorrectly) run through a standard chi-square test anyway. If your data are paired or matched, see McNemar’s test below instead.
- Expected cell counts of at least 5. The chi-square statistic is a large-sample approximation to a continuous distribution being used to model discrete counts; when expected counts are small, that approximation breaks down and the reported p-value becomes unreliable. The commonly cited rule of thumb (attributed to Cochran) is that no more than about 20% of cells should have an expected count below 5, and no cell should have an expected count below 1.
- Mutually exclusive, exhaustive categories. Every observation must fall into exactly one category/cell.
When expected counts are too small, two options are generally used:
- Fisher’s exact test computes an exact p-value directly from the hypergeometric distribution rather than relying on the chi-square approximation, and is the standard alternative for small 2×2 tables (and, via extensions, larger tables). Most statistical software offers it as a direct substitute.
- Collapsing categories (merging sparse categories into a broader one) can raise expected counts enough to make the chi-square approximation valid. This is legitimate only when the collapsed categories are combined on substantive, pre-specified grounds decided before looking at the results — collapsing categories after the fact, in whatever way produces a significant result, is a form of data-dependent analysis that inflates the false-positive rate and should be disclosed if done at all.
Yates’ continuity correction
For 2×2 contingency tables specifically, a modification called Yates’ continuity correction is sometimes applied:
χ² = Σ (|O − E| − 0.5)² / E
The correction subtracts 0.5 from each absolute deviation before squaring, which shrinks the test statistic and makes the test more conservative (less likely to find significance). It was originally proposed to make the discrete chi-square statistic better approximate the continuous chi-square distribution in the 2×2 case. It remains debated: some statisticians consider it appropriately conservative and recommend it by default for small 2×2 tables, while others argue it over-corrects and is unnecessarily conservative, especially compared to just using Fisher’s exact test directly when counts are small. Because software defaults differ (R applies it by default for 2×2 tables via chisq.test(), for example), always report explicitly whether the correction was applied.
Effect size: why the p-value alone isn’t enough
A chi-square test’s p-value is heavily influenced by sample size: with a large enough sample, even a trivially small, practically meaningless association will produce a statistically significant result. See CASRAI’s guide on what a p value measures for why a small p-value alone never indicates a strong or important effect. Reporting an effect size alongside the test statistic is essential to communicate how strong the association actually is, independent of sample size. The standard chi-square effect sizes are:
- Phi (φ) — used for 2×2 tables:
φ = √(χ² / n). Interpreted similarly to a correlation coefficient. - Cramér’s V — the general-purpose version for any r×c table:
V = √(χ² / (n × min(r−1, c−1))). Ranges from 0 (no association) to 1 (perfect association); conventional rough benchmarks (small ≈ 0.1, medium ≈ 0.3, large ≈ 0.5) are widely cited but should be treated as approximate, field-dependent guidance rather than fixed cutoffs. - Odds ratio — for a 2×2 table specifically, the odds ratio (the ratio of the odds of the outcome in one group to the odds in the other) is often the most directly interpretable effect size, particularly in clinical and epidemiological reporting, because it expresses the association in terms of relative risk-like odds rather than an abstract association coefficient.
McNemar’s test: when the data are paired
A standard chi-square test assumes independent observations. When the same subjects are measured twice on a binary outcome — before/after an intervention, or two raters classifying the same cases — the two sets of counts are paired, not independent, and a standard chi-square test of independence is the wrong tool: it ignores the pairing structure and will generally give an incorrect p-value.
McNemar’s test is designed specifically for this paired 2×2 case. It focuses only on the “discordant” pairs — cases that changed category between the two measurements — and tests whether changes in one direction are more common than changes in the other:
χ² = (b − c)² / (b + c)
where b and c are the two off-diagonal (discordant) cell counts in the paired 2×2 table, with 1 degree of freedom. A continuity-corrected version, (|b − c| − 1)² / (b + c), is commonly used as well, particularly with small discordant-pair counts.
Post-hoc analysis after a significant result in a larger table
A significant chi-square test on an r×c table (larger than 2×2) tells you the categorical variables are associated somewhere in the table, but not which specific cells or category pairs are driving that association. Two common follow-up approaches:
- Standardized (or adjusted standardized) residuals for each cell show how far that cell’s observed count deviates from its expected count, in standard-deviation-like units. Cells with adjusted residuals beyond roughly ±2 (or ±1.96, treating them as approximately standard normal) are typically flagged as meaningful contributors to the overall association.
- Pairwise comparisons between specific categories (e.g., running a series of smaller chi-square or Fisher’s exact tests on category subsets) can localize where an association holds, but because this involves multiple tests on the same data, a multiplicity correction (such as a Bonferroni adjustment to the significance threshold) should be applied to control the inflated false-positive rate that comes from running many comparisons.
Reporting a chi-square result in APA style
APA style reports the test statistic, degrees of freedom, sample size, and p-value together, in a standard format:
χ²(1, N = 200) = 10.24, p = .001
Read as: a chi-square test with 1 degree of freedom, on a sample of 200, produced a test statistic of 10.24 and a p-value of .001. A complete write-up typically also states which test form was used (independence vs. goodness-of-fit), whether Yates’ correction or Fisher’s exact test was used instead, and the relevant effect size (e.g., Cramér’s V or the odds ratio) alongside the significance test. See CASRAI’s guide on how to report p-values for the general APA conventions this format follows.
Running a chi-square test in common software
R
The base-R function for the test of independence and goodness-of-fit is chisq.test():
tbl <- matrix(c(45, 35, 30, 90), nrow = 2, byrow = TRUE)
chisq.test(tbl) # Yates' correction applied by default for 2x2 tables
chisq.test(tbl, correct = FALSE) # without the continuity correction
# Goodness-of-fit
chisq.test(c(50, 28, 44, 38), p = c(0.25, 0.25, 0.25, 0.25))
# Fisher's exact test (small expected counts)
fisher.test(tbl)Python (SciPy)
SciPy’s scipy.stats module provides the equivalent functions:
from scipy.stats import chi2_contingency, chisquare, fisher_exact
table = [[45, 35], [30, 90]]
chi2, p, dof, expected = chi2_contingency(table) # correction=True by default for 2x2
# Goodness-of-fit
chisquare(f_obs=[50, 28, 44, 38], f_exp=[40, 40, 40, 40])
# Fisher's exact test
odds_ratio, p_value = fisher_exact(table)SPSS
In SPSS, the test of independence is run via Analyze → Descriptive Statistics → Crosstabs, entering the two categorical variables as Row and Column, then checking Chi-square under the Statistics button. SPSS’s Crosstabs output reports the Pearson chi-square value, df, and asymptotic significance by default, and can report Fisher’s exact test automatically for 2×2 tables with small expected counts, plus phi and Cramér’s V under the same Statistics dialog.
Frequently asked questions
What does the chi-square test actually tell you?
It tells you whether the pattern of counts observed across categories differs from what you’d expect under a specific null hypothesis (either that two categorical variables are independent, or that a variable follows a hypothesized distribution) by more than chance would plausibly produce.
Can chi-square be used for continuous data?
Not directly. Chi-square tests are for categorical data. Continuous data would first need to be grouped into categories (e.g., age bands) before a chi-square test could apply — CASRAI’s guide to the histogram covers how a continuous variable’s distribution is normally examined before making that kind of grouping decision — and doing so discards information, so a test designed for continuous data (t-test, ANOVA, correlation, regression) is usually preferable when the outcome is genuinely numeric.
What’s the difference between chi-square and a t-test?
A t-test compares the means of a continuous variable across two groups. A chi-square test compares the distribution of counts across categories. They answer structurally different questions and are not interchangeable.
Why did my chi-square test give a “expected count” warning?
Most software warns when one or more cells have an expected count below 5, because the chi-square approximation becomes unreliable in that situation. See the assumptions section above for when to switch to Fisher’s exact test or collapse categories.
Is a significant chi-square result always meaningful?
Not necessarily. With a large sample, even a very weak, practically trivial association can be statistically significant. Always report an effect size (phi, Cramér’s V, or an odds ratio) alongside the test result to show how strong the association actually is.
