Written and maintained by CASRAI Editorial Board
Last updated
On this page: why a factorial design tests multiple factors together instead of one at a time, and why that’s more efficient; a fully worked 2×2 layout with a synthetic dataset; how to separate a main effect from an interaction effect using the same numbers; the ANOVA decomposition behind the test; and why detecting an interaction takes more statistical power than detecting a main effect of the same size.
Why test factors together instead of one at a time
A factorial design manipulates two or more independent variables (“factors”) within the same study, so that every combination of factor levels appears in the data. The alternative — the intuitive but usually wrong approach — is to test one factor at a time (OFAT): run a study varying Factor A while holding B constant, then a separate study varying B while holding A constant. OFAT looks simpler, but it does two things badly. First, it wastes data: a factorial design with the same total sample size estimates both main effects with the same precision an OFAT approach would need twice the runs to match, because every participant contributes information to both factors at once rather than just one. Second, and more importantly, OFAT cannot detect an interaction — whether the effect of one factor depends on the level of the other — because it never observes the combination where both factors vary simultaneously. If the two factors interact, an OFAT result is not just less efficient, it can be flatly misleading: the “effect of A” you measured while B was held at one level may not hold at all when B is at its other level.
This is the core efficiency argument for factorial designs: they don’t just save runs, they answer a question — does A’s effect change depending on B? — that a one-factor-at-a-time design structurally cannot answer at all.
Notation: 2×2, 2×3, and full vs. fractional
A factorial design is described by the number of levels each factor has, written as levels × levels. The simplest and most common case is the 2×2 factorial: two factors, each at two levels (often “absent/present” or “low/high”), giving four cells. A 2×3 design has one two-level factor and one three-level factor (six cells); a 3×3 has two three-level factors (nine cells); a 2³ (read “two-cubed”) design has three two-level factors, giving eight cells. In general, a full factorial design includes every combination of every level of every factor. That’s what makes it “full” — and it’s also what makes it expensive to scale: a 2³ design needs 8 cells, a 2⁴ needs 16, a 2⁵ needs 32. Where the number of factors is large, researchers often turn to a fractional factorial design instead, which deliberately runs only a carefully chosen subset of the combinations (see the worked fractional-factorial example in discrete choice experiments) at the cost of confounding some higher-order interactions with each other. This guide covers the full factorial case — specifically the 2×2 — because it’s the version most researchers actually design and analyze, and every larger factorial is built from the same logic.
A worked 2×2 design
The dataset below is synthetic — generated by a seeded pseudo-random-number script for illustration, not data from a real study. Suppose a researcher is testing whether a reminder email (Factor A: no / yes) and a small monetary incentive (Factor B: no / yes) each increase engagement with an online survey, measured on a continuous 0–100 engagement score. Ten simulated participants are randomly assigned to each of the four combinations (N = 40 total):
| Incentive: No | Incentive: Yes | Row (A) mean | |
|---|---|---|---|
| Reminder: No | 43.24 | 48.12 | 45.68 |
| Reminder: Yes | 52.38 | 62.82 | 57.60 |
| Column (B) mean | 47.81 | 55.47 | Grand mean: 51.64 |
Those four cell means already contain everything needed to compute both main effects and the interaction — the rest is arithmetic on numbers that are already sitting in the table.
Separating a main effect from an interaction effect
A main effect is the average effect of one factor, collapsing across (ignoring) the levels of the other factor — it’s exactly the row mean or column mean difference:
- Main effect of A (reminder) = row mean(yes) − row mean(no) = 57.60 − 45.68 = +11.92 points, averaged across both incentive conditions.
- Main effect of B (incentive) = column mean(yes) − column mean(no) = 55.47 − 47.81 = +7.67 points, averaged across both reminder conditions.
An interaction effect is a completely different quantity: it asks whether the effect of one factor changes depending on the level of the other, rather than reporting an average across it. The clean way to see it in a 2×2: compute the effect of A separately within each level of B, and compare.
- Effect of A when incentive = no: 52.38 − 43.24 = +9.14
- Effect of A when incentive = yes: 62.82 − 48.12 = +14.70
Those two numbers are not the same — the reminder email helps more when an incentive is also present (+14.70) than when it isn’t (+9.14). That gap, +5.56, is the interaction: the reminder’s effect is not constant across incentive conditions, so no single “main effect of A” number fully describes what’s happening. This is the exact distinction an OFAT design cannot see: it would report an effect of A from whichever B condition it happened to test in, with no way to know a different B condition would have shown a different-sized effect.
Equivalently, the interaction can be read off the table as the gap between the observed top-right cell (62.82) and what additive main effects alone would predict for that cell (grand mean + A’s deviation + B’s deviation = 51.64 + 5.96 + 3.83 = 61.43) — the observed cell runs about 1.4 points above the simple additive prediction, concentrated in the both-yes cell, which is the same synergy the row-by-row comparison above already showed.
The ANOVA decomposition behind the test
A two-way ANOVA partitions the total variation in the 40 scores into a piece attributable to A, a piece attributable to B, a piece attributable to the A×B interaction, and a residual error term — computed directly from the same synthetic dataset above (sums of squares verified to sum exactly to the total, not estimated):
| Source | SS | df | MS | F | p |
|---|---|---|---|---|---|
| A (reminder) | 1421.13 | 1 | 1421.13 | 60.63 | < .001 |
| B (incentive) | 587.57 | 1 | 587.57 | 25.07 | < .001 |
| A × B | 77.23 | 1 | 77.23 | 3.29 | .078 |
| Error | 843.79 | 36 | 23.44 | ||
| Total | 2929.73 | 39 |
Both main effects are clearly significant. The interaction term (F(1,36) = 3.29, p = .078) falls short of the conventional .05 threshold, even though the data-generating process behind this simulation actually included a genuine synergy bonus in the both-yes cell — see the next section for why that’s not a contradiction, it’s the expected pattern.
In R, this is aov(engagement ~ A * B, data = df), where A * B expands to the two main effects plus their interaction (equivalent to A + B + A:B). In SPSS, it’s Analyze > General Linear Model > Univariate, where “Full factorial” is the default model and includes every main effect and interaction automatically.
Reading the pattern: crossing lines vs. parallel lines
Plotted as two lines — one for “reminder: no,” one for “reminder: yes,” each connecting the no-incentive and yes-incentive cell means — a pure main-effects pattern with no interaction produces two roughly parallel lines: both go up (or down) by about the same amount from no-incentive to yes-incentive, just offset from each other. An interaction shows up as lines that are not parallel — they diverge, converge, or cross outright. In this dataset the “reminder: yes” line rises more steeply (+14.70) from no-incentive to yes-incentive than the “reminder: no” line does (+9.14): the lines fan apart rather than staying parallel, visually representing the same synergy the numbers above described. A full crossover — where the ranking of conditions actually reverses at different levels of the other factor — is the most dramatic form of interaction and the clearest case where reporting only the main effects would be actively misleading.
Why detecting an interaction costs more power
The ANOVA table above illustrates a well-known and often underestimated problem: for the same sample size and the same underlying effect size, interaction effects are harder to detect than main effects. A main effect of A pools all 20 “yes” observations against all 20 “no” observations — it uses the full sample to estimate a single contrast. Testing an interaction, by contrast, is really asking whether a difference-of-differences is nonzero, and that difference-of-differences has a larger standard error than either difference alone, because it inherits sampling variability from both. In a balanced 2×2 design, the interaction estimate is a difference of two simple differences, so its variance is four times the variance of a single main-effect estimate (each simple difference already has twice the variance of the pooled main-effect contrast, and the interaction sums two of them) — which means detecting an interaction of the same size as a main effect, at the same power, needs roughly four times the sample size. This is a routinely under-anticipated planning mistake in factorial-design studies. The simulation above is a live illustration: a genuine +8-point synergy was built directly into the data-generating process for the both-yes cell, yet with only n = 10 per cell the interaction test came back p = .078 — not significant at the conventional threshold — while both main effects, which were no larger in absolute terms, tested cleanly. If the interaction is your actual research question rather than an afterthought, the sample-size planning has to target the interaction term specifically; see power analysis and sample size calculation and the G*Power walkthrough for how to do that, rather than powering the study for the main effects and assuming the interaction test comes along for free.
When a full factorial isn’t the right tool
A full factorial is the right design when the factors can be freely and independently randomized and the number of factors is small enough that every combination is practical to run. It’s not always the fit: when there’s a nuisance variable to control for rather than a factor to test, a blocking design handles one nuisance dimension and a Latin square design handles two at once, without needing a full crossed factorial. When one factor is expensive or physically difficult to randomize at the same fine grain as the others — an oven temperature, a field’s irrigation regime — a split-plot design keeps the factorial structure but randomizes the awkward factor at a coarser level, producing two separate error terms rather than the single pooled error term used above. And once the number of factors climbs past three or four, a full factorial’s cell count grows fast enough that a fractional factorial, trading away estimability of some higher-order interactions for a fraction of the runs, is usually the more realistic choice.
Frequently asked questions
Is a 2×2 factorial design the same as a 2×2 contingency table?
No — they’re easy to confuse because both are 2×2 grids, but they answer different kinds of questions. A 2×2 factorial design is an experimental structure: two manipulated factors, each with two levels, and a continuous (or otherwise measured) outcome analyzed with ANOVA, as above. A 2×2 contingency table (see also the worked 2×2 table in the sensitivity/specificity guide) cross-tabulates two categorical variables and is analyzed with a chi-square test or an odds ratio, with no assumption that either variable was experimentally manipulated. The “2×2” in each case describes a different kind of grid.
How many participants do I need for a 2×2 factorial design?
It depends on which effect you actually care about detecting. If the main effects are the target, a conventional power analysis for a two-group comparison, then doubled into the factorial structure, is usually adequate. If the interaction is the actual research question, plan for roughly four times the sample size a same-size main effect would need — see the power section above — and run the calculation for the specific interaction contrast, not just the overall design.
What’s the difference between a full factorial and a fractional factorial design?
A full factorial runs every possible combination of every factor level. A fractional factorial deliberately runs only a systematically chosen subset, which keeps the study feasible when there are many factors, at the cost of confounding (aliasing) some higher-order interactions with each other or with main effects — usually an acceptable trade-off, since higher-order interactions are both rarer and harder to interpret substantively than main effects or two-way interactions.
Can a factorial design have more than two factors?
Yes — a factorial design can include any number of factors, each with any number of levels; a three-factor design with two levels each is written 2³ and has eight cells. The 2×2 is the version most commonly taught and reported because it’s the smallest case that actually contains an interaction, but the same main-effect/interaction logic scales directly to more factors and more levels, just with more terms in the ANOVA table.
