Written and maintained by CASRAI Editorial Board
Last updated
By default, R’s aov() computes Type I (sequential) sums of squares, which are order-dependent and usually wrong for an unbalanced design. To get Type III sums of squares — the ones most journals and most other software report by default — fit the model with lm(), set options(contrasts = c('contr.sum', 'contr.poly')) before fitting, and run car::Anova(model, type = 3). Skip the contrasts step and car::Anova() will still run, print a normal-looking table, and silently give you the wrong F values and p-values for every main effect.
This page is the procedure, with real numbers worked through so you can see exactly where Type I and Type III diverge, and why. For the underlying logic of ANOVA itself — what a sum of squares is, why F-tests work the way they do — read ANOVA (Analysis of Variance) first. This page assumes you already know that and just want to run it correctly in R.
Why aov()‘s default is a trap
aov() builds its ANOVA table by adding terms to the model one at a time, in the order you wrote them in the formula, and measuring how much each new term reduces the residual sum of squares given everything already in the model. That is Type I, or sequential, sums of squares. It is exactly what a textbook one-way or perfectly balanced factorial ANOVA needs, and it is also exactly what R’s summary(aov(...)) always gives you, no matter what design you actually have.
The problem only shows up with two conditions at once: more than one factor, and unequal group sizes (an unbalanced design). When cells are unbalanced, the factors are correlated with each other in the data (not by design, just because the sample sizes differ), so the order you list them in the formula changes how much of the shared variance each one gets credited with. Whichever term you list first soaks up variance that, in a properly controlled test, should be split with the terms that come after it. The table still prints, the F values still look plausible, and nothing in the output tells you it depends on term order — that is what makes it a silent failure mode rather than an error.
A worked example: an unbalanced 2×2 design
The dataset below is a small, deliberately synthetic example built for this walkthrough — not a real study — with two factors and unbalanced cell sizes on purpose, so the Type I / Type III difference actually shows up. Two conditions (“Diet”: Control vs Supplement; “Exercise”: Sedentary vs Active) are crossed, giving four cells with these sizes:
Control × Sedentary n = 8 mean = 2.20
Control × Active n = 5 mean = 3.12
Supplement × Sedentary n = 6 mean = 2.78
Supplement × Active n = 10 mean = 4.2529 rows total, no missing cells, but no two cells the same size — a realistic shape for, say, a clinic sample where enrollment wasn’t randomized to fixed quotas. Fit it the obvious way:
fit <- aov(y ~ diet * exercise, data = df)
summary(fit) Df Sum Sq Mean Sq F value Pr(>F)
diet 1 9.422 9.422 117.70 6.04e-11 ***
exercise 1 10.166 10.166 126.99 2.73e-11 ***
diet:exercise 1 0.505 0.505 6.31 0.0188 *
Residuals 25 2.001 0.080
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Now swap the order of the two main effects in the formula — nothing about the data changes, only the order they’re written:
fit2 <- aov(y ~ exercise * diet, data = df)
summary(fit2) Df Sum Sq Mean Sq F value Pr(>F)
exercise 1 14.670 14.670 183.25 5.20e-13 ***
diet 1 4.918 4.918 61.43 3.41e-08 ***
exercise:diet 1 0.505 0.505 6.31 0.0188 *
Residuals 25 2.001 0.080 Same data, same model, same residual sum of squares (2.001 on 25 df, both tables) — but the sum of squares attributed to diet moved from 9.422 down to 4.918, and its F value dropped from 117.70 to 61.43, purely because it changed places in the formula. The interaction row is identical in both tables (SS 0.505, F 6.31) because a top-order interaction term’s sum of squares does not depend on how the lower-order terms are ordered before it — that part of the table is always trustworthy. It’s the main effects that move.
The car::Anova() trap: type III with the wrong contrasts
The usual next step is to reach for car::Anova() (the car package, John Fox and Sanford Weisberg) and ask for type III directly, expecting an order-independent answer:
library(car)
fit_lm <- lm(y ~ diet * exercise, data = df)
Anova(fit_lm, type = 3)Anova Table (Type III tests)
Sum Sq Df F value Pr(>F)
diet 1.167 1 14.57 0.000790 ***
exercise 2.604 1 32.53 6.11e-06 ***
diet:exercise 0.505 1 6.31 0.018800 *
Residuals 2.001 25 This runs without a warning, and the table looks exactly like a real Type III result: order-independent, one row per term, no obvious defect. It is also wrong. Under R’s default contrasts (contr.treatment, which codes each factor against a reference level), the “type III” sum of squares for diet that car::Anova() just computed is not testing the overall Diet main effect — it is testing the Diet difference only within the reference level of Exercise, because that is what the diet coefficient means once an interaction term is in the model coded that way. The label says “type III”; the quantity underneath is a simple effect, not a main effect. Nothing in the output distinguishes the two.
The fix: set sum-to-zero contrasts before fitting
Type III sums of squares are only a well-defined, order-independent test of “does this factor matter, averaged evenly across the other factor’s levels” when the contrast coding is orthogonal — which contr.treatment is not, once an interaction is in the model. contr.sum (sum-to-zero, sometimes called “effect coding”) is. Set it globally before fitting, then refit:
options(contrasts = c('contr.sum', 'contr.poly'))
fit_lm2 <- lm(y ~ diet * exercise, data = df)
Anova(fit_lm2, type = 3)Anova Table (Type III tests)
Sum Sq Df F value Pr(>F)
diet 4.961 1 61.98 3.15e-08 ***
exercise 9.627 1 120.26 4.83e-11 ***
diet:exercise 0.505 1 6.31 0.018800 *
Residuals 2.001 25 Same model, same data, only the contrast coding changed — and the reported F value for diet goes from 14.57 to 61.98, and for exercise from 32.53 to 120.26. Both are now real main-effect tests: is there a Diet difference, averaged equally across Sedentary and Active, controlling for the interaction; is there an Exercise difference, averaged equally across Control and Supplement, controlling for the interaction. The interaction row is unchanged (it always is — it’s the one row in this table that never depended on contrast coding or term order in the first place). options(contrasts = ...) is a session-wide setting, not a model argument, which is exactly why it’s easy to forget: set it once near the top of your script, before any model in that script gets fit, or set it inside a function so it doesn’t leak into unrelated analyses in the same session.
When none of this matters
Type I, Type II, and Type III sums of squares only disagree when a design is both unbalanced and has more than one factor. If cell sizes are exactly equal, the factors are orthogonal by construction and all three types give identical results — there is no trap to fall into with a balanced factorial. Type I and Type III also agree with each other when there is genuinely no interaction term in the model at all, since with only main effects present the sequential and marginal sums of squares coincide (this is the same reasoning that makes Type II, which controls for other main effects but not interactions, the recommended choice when you’ve confirmed the interaction is negligible and dropped it). A true one-way ANOVA with one factor has no ordering question either — Type I, II and III are all the same table. The whole issue in this page is specific to unbalanced, more-than-one-factor, interaction-included designs, which is a narrower case than “any time you run aov()” but a common enough one in real, non-experimentally-controlled data that it’s worth checking for by default.
Post hoc comparisons: use emmeans, not raw group means
Once a Type III test says a main effect is real, the natural next question is which levels differ — and the same unbalanced-cells issue that broke the naive ANOVA table also breaks a naive group mean. Compare the two ways of summarizing the Diet effect in this dataset:
# naive: mean of every row with diet == "Supplement", regardless of exercise
aggregate(y ~ diet, data = df, mean) diet y
1 Control 2.554
2 Supplement 3.700# estimated marginal means: averaged evenly across exercise levels first
library(emmeans)
emmeans(fit_lm2, ~ diet) diet emmean
Control 2.66
Supplement 3.52The naive means (2.55 vs 3.70, a 1.15 gap) are pulled around by the fact that Supplement happens to be over-represented in the Active cell (n=10, the largest cell in the design) — they mix the Diet effect with the unbalanced sampling. The emmeans estimates (2.66 vs 3.52, a 0.86 gap) average each diet’s two cell means with equal weight, matching what the Type III test above actually evaluated. Run emmeans(fit_lm2, pairwise ~ diet) to get that difference with a confidence interval and a p-value in one call, or add adjust = "tukey" when there are more than two levels to correct for multiple comparisons. Fit the model with the sum-to-zero contrasts first (as above) — emmeans reads the reference grid off whatever model you hand it, so if the model’s contrasts weren’t fixed, neither will the post hoc estimates be.
Reporting the result
Report which sums-of-squares type you used — “Type III sums of squares (car package, sum-to-zero contrasts)” in a methods note is enough — since the same design can legitimately report different F values depending on that choice, and a reviewer familiar with R will know to ask. A standard APA-style line for the corrected Diet effect above: “There was a significant main effect of diet, F(1, 25) = 61.98, p < .001.” Pair it with an effect size (partial eta-squared, from effectsize::eta_squared(fit_lm2) or by hand as SSeffect / (SSeffect + SSresidual)) and, where the design allows it, the emmeans-based estimated marginal means rather than raw group means, for the reasons above.
Frequently asked questions
Does this matter for a one-way ANOVA?
No. With a single factor there’s nothing to sequence and nothing for the coding to interact with — aov()‘s default output is already correct. The Type I/III distinction only bites once you have two or more factors and unbalanced cells.
Can I just always use car::Anova(fit, type = 3) and skip aov() entirely?
Only once you’ve set options(contrasts = c('contr.sum', 'contr.poly')) first. car::Anova() will accept a model fit under the default treatment contrasts and produce a table with no warning, so the contrasts step is the part that actually has to happen — the function call alone isn’t the fix.
Is Type II ever the better choice over Type III?
When the higher-order interaction genuinely isn’t in the model (you tested it and dropped it), Type II main-effect tests are generally preferred over Type III by statisticians, because Type III main-effect tests in a model that still contains the interaction are testing a marginal average that can be hard to interpret when the interaction is real and large. If you’ve kept the interaction in the model because it matters, Type III is the appropriate choice for the main effects; car::Anova() defaults to type = 2 if you omit the type argument, so it’s worth checking what you actually got.
Do I need the car package specifically?
For Type III with a factorial design, yes in practice — base R’s anova() only ever produces sequential (Type I) tables, no matter how the model was fit. drop1(fit, test = "F") is a base-R alternative that computes a marginal (Type III-equivalent) test one term at a time, and respects contrast coding the same way car::Anova() does, but prints one line per drop1() call rather than a single combined table.
Related reading
- ANOVA (Analysis of Variance): One-Way, Two-Way, Assumptions, Kruskal-Wallis and Effect Size — the method itself, independent of software.
- ANOVA (Analysis of Variance) — short reference definition.
- Logistic Regression in R: glm(), Odds Ratios, and Diagnostics — the same code-plus-real-output format for a different model.
- Regression Analysis: Assumptions, Interpretation, and How to Report It — ANOVA is a special case of the general linear model; this covers the shared machinery.
- The Kruskal-Wallis Test — the nonparametric alternative when ANOVA’s assumptions don’t hold.
- Degrees of Freedom in Statistics — what the Df column in every table above is actually counting.
