Skip to main content
v2026.11,986 entries · CC-BY 4.0

Factorial Designs: Testing Multiple Factors and Their Interactions at Once

A full factorial design tests every combination of two or more factors at once instead of one factor at a time — more efficient, and the only way to detect whether one factor’s effect depends on another. Worked through with a computed 2×2 layout separating a main effect from an interaction effect.

Written and maintained by CASRAI Editorial Board

Published · Last updated

On this page

On this page: why a factorial design tests multiple factors together instead of one at a time, and why that’s more efficient; a fully worked 2×2 layout with a synthetic dataset; how to separate a main effect from an interaction effect using the same numbers; the ANOVA decomposition behind the test; and why detecting an interaction takes more statistical power than detecting a main effect of the same size.

Why test factors together instead of one at a time

A factorial design manipulates two or more independent variables (“factors”) within the same study, so that every combination of factor levels appears in the data. The alternative — the intuitive but usually wrong approach — is to test one factor at a time (OFAT): run a study varying Factor A while holding B constant, then a separate study varying B while holding A constant. OFAT looks simpler, but it does two things badly. First, it wastes data: a factorial design with the same total sample size estimates both main effects with the same precision an OFAT approach would need twice the runs to match, because every participant contributes information to both factors at once rather than just one. Second, and more importantly, OFAT cannot detect an interaction — whether the effect of one factor depends on the level of the other — because it never observes the combination where both factors vary simultaneously. If the two factors interact, an OFAT result is not just less efficient, it can be flatly misleading: the “effect of A” you measured while B was held at one level may not hold at all when B is at its other level.

This is the core efficiency argument for factorial designs: they don’t just save runs, they answer a question — does A’s effect change depending on B? — that a one-factor-at-a-time design structurally cannot answer at all.

Notation: 2×2, 2×3, and full vs. fractional

A factorial design is described by the number of levels each factor has, written as levels × levels. The simplest and most common case is the 2×2 factorial: two factors, each at two levels (often “absent/present” or “low/high”), giving four cells. A 2×3 design has one two-level factor and one three-level factor (six cells); a 3×3 has two three-level factors (nine cells); a 2³ (read “two-cubed”) design has three two-level factors, giving eight cells. In general, a full factorial design includes every combination of every level of every factor. That’s what makes it “full” — and it’s also what makes it expensive to scale: a 2³ design needs 8 cells, a 2⁴ needs 16, a 2⁵ needs 32. Where the number of factors is large, researchers often turn to a fractional factorial design instead, which deliberately runs only a carefully chosen subset of the combinations (see the worked fractional-factorial example in discrete choice experiments) at the cost of confounding some higher-order interactions with each other. This guide covers the full factorial case — specifically the 2×2 — because it’s the version most researchers actually design and analyze, and every larger factorial is built from the same logic.

A worked 2×2 design

The dataset below is synthetic — generated by a seeded pseudo-random-number script for illustration, not data from a real study. Suppose a researcher is testing whether a reminder email (Factor A: no / yes) and a small monetary incentive (Factor B: no / yes) each increase engagement with an online survey, measured on a continuous 0–100 engagement score. Ten simulated participants are randomly assigned to each of the four combinations (N = 40 total):

Incentive: No Incentive: Yes Row (A) mean
Reminder: No 43.24 48.12 45.68
Reminder: Yes 52.38 62.82 57.60
Column (B) mean 47.81 55.47 Grand mean: 51.64

Those four cell means already contain everything needed to compute both main effects and the interaction — the rest is arithmetic on numbers that are already sitting in the table.

Separating a main effect from an interaction effect

A main effect is the average effect of one factor, collapsing across (ignoring) the levels of the other factor — it’s exactly the row mean or column mean difference:

  • Main effect of A (reminder) = row mean(yes) − row mean(no) = 57.60 − 45.68 = +11.92 points, averaged across both incentive conditions.
  • Main effect of B (incentive) = column mean(yes) − column mean(no) = 55.47 − 47.81 = +7.67 points, averaged across both reminder conditions.

An interaction effect is a completely different quantity: it asks whether the effect of one factor changes depending on the level of the other, rather than reporting an average across it. The clean way to see it in a 2×2: compute the effect of A separately within each level of B, and compare.

  • Effect of A when incentive = no: 52.38 − 43.24 = +9.14
  • Effect of A when incentive = yes: 62.82 − 48.12 = +14.70

Those two numbers are not the same — the reminder email helps more when an incentive is also present (+14.70) than when it isn’t (+9.14). That gap, +5.56, is the interaction: the reminder’s effect is not constant across incentive conditions, so no single “main effect of A” number fully describes what’s happening. This is the exact distinction an OFAT design cannot see: it would report an effect of A from whichever B condition it happened to test in, with no way to know a different B condition would have shown a different-sized effect.

Equivalently, the interaction can be read off the table as the gap between the observed top-right cell (62.82) and what additive main effects alone would predict for that cell (grand mean + A’s deviation + B’s deviation = 51.64 + 5.96 + 3.83 = 61.43) — the observed cell runs about 1.4 points above the simple additive prediction, concentrated in the both-yes cell, which is the same synergy the row-by-row comparison above already showed.

Main effect vs interaction effect: the short answer

A main effect is the average effect of one factor across the levels of the other factor. An interaction effect means the effect of one factor changes depending on the level of the other. In a 2×2 design, compare each factor’s simple effects: if they are about equal, only main effects matter; if they differ, you have an interaction and the averaged main effect no longer tells the whole story.

Main effect Interaction effect
Question it answers Does this factor change the outcome on average? Does the effect of this factor depend on the other factor?
Computed from Row or column means (difference of marginal means) Difference between simple effects (a difference of differences)
In the reminder/incentive example +11.92 (reminder), +7.67 (incentive) +14.70 vs +9.14, a gap of +5.56
On a line plot Lines are offset or sloped Lines are not parallel
If the interaction is large Interpret with caution, report simple effects instead Follow up with a simple effects analysis

For the software side of the same table, including how SPSS reports the three tests, see two-way ANOVA in SPSS, and for the checks that should come before reading either test, the assumptions of two-way ANOVA.

The ANOVA decomposition behind the test

A two-way ANOVA partitions the total variation in the 40 scores into a piece attributable to A, a piece attributable to B, a piece attributable to the A×B interaction, and a residual error term — computed directly from the same synthetic dataset above (sums of squares verified to sum exactly to the total, not estimated):

Source SS df MS F p
A (reminder) 1421.13 1 1421.13 60.63 < .001
B (incentive) 587.57 1 587.57 25.07 < .001
A × B 77.23 1 77.23 3.29 .078
Error 843.79 36 23.44
Total 2929.73 39

Both main effects are clearly significant. The interaction term (F(1,36) = 3.29, p = .078) falls short of the conventional .05 threshold, even though the data-generating process behind this simulation actually included a genuine synergy bonus in the both-yes cell — see the next section for why that’s not a contradiction, it’s the expected pattern.

In R, this is aov(engagement ~ A * B, data = df), where A * B expands to the two main effects plus their interaction (equivalent to A + B + A:B). In SPSS, it’s Analyze > General Linear Model > Univariate, where “Full factorial” is the default model and includes every main effect and interaction automatically.

Reading the pattern: crossing lines vs. parallel lines

Plotted as two lines — one for “reminder: no,” one for “reminder: yes,” each connecting the no-incentive and yes-incentive cell means — a pure main-effects pattern with no interaction produces two roughly parallel lines: both go up (or down) by about the same amount from no-incentive to yes-incentive, just offset from each other. An interaction shows up as lines that are not parallel — they diverge, converge, or cross outright. In this dataset the “reminder: yes” line rises more steeply (+10.44, from 52.38 to 62.82) from no-incentive to yes-incentive than the “reminder: no” line does (+4.88, from 43.24 to 48.12): the lines fan apart rather than staying parallel, visually representing the same synergy the numbers above described. A full crossover — where the ranking of conditions actually reverses at different levels of the other factor — is the most dramatic form of interaction and the clearest case where reporting only the main effects would be actively misleading.

Why detecting an interaction costs more power

The ANOVA table above illustrates a well-known and often underestimated problem: for the same sample size and the same underlying effect size, interaction effects are harder to detect than main effects. A main effect of A pools all 20 “yes” observations against all 20 “no” observations — it uses the full sample to estimate a single contrast. Testing an interaction, by contrast, is really asking whether a difference-of-differences is nonzero, and that difference-of-differences has a larger standard error than either difference alone, because it inherits sampling variability from both. In a balanced 2×2 design, the interaction estimate is a difference of two simple differences, so its variance is four times the variance of a single main-effect estimate (each simple difference already has twice the variance of the pooled main-effect contrast, and the interaction sums two of them) — which means detecting an interaction of the same size as a main effect, at the same power, needs roughly four times the sample size. This is a routinely under-anticipated planning mistake in factorial-design studies. The simulation above is a live illustration: a genuine +8-point synergy was built directly into the data-generating process for the both-yes cell, yet with only n = 10 per cell the interaction test came back p = .078 — not significant at the conventional threshold — while both main effects, which were no larger in absolute terms, tested cleanly. If the interaction is your actual research question rather than an afterthought, the sample-size planning has to target the interaction term specifically; see power analysis and sample size calculation and the G*Power walkthrough for how to do that, rather than powering the study for the main effects and assuming the interaction test comes along for free.

Can you test an interaction when one cell of a 2×2 is missing?

Not in the usual way. The interaction in a 2×2 is a difference of differences, and it needs all four cell means. If one combination was never run (for example, a design with a control, an A-only group and a B-only group but no group receiving both), the standard interaction cannot be estimated. What you can do is ask narrower questions using the three cells you have.

Statistics course notes on unbalanced factorials describe this situation as a missing (empty) cell: the cell has no observations, so some marginal means cannot be estimated and the usual Type III tests become hard to interpret. Their recommended remedy is to write your own contrasts for the questions the available means can answer; see the Iowa State course notes on unbalanced and missing-cell designs and the University of South Carolina chapter on unbalanced two-way studies, which suggests contrasts that study one factor’s effects within levels of the other, or an additive model provided the design is still connected.

A worked example with the reminder and incentive data

Suppose the both-yes cell (reminder and incentive) had never been run. Using the synthetic cell means above, three remain: 43.24 (neither), 48.12 (incentive only) and 52.38 (reminder only).

Question Contrast Estimable with three cells?
Effect of incentive without a reminder 48.12 − 43.24 = +4.88 Yes
Effect of reminder without an incentive 52.38 − 43.24 = +9.14 Yes
Incentive-only vs reminder-only 48.12 − 52.38 = −4.26 Yes
Interaction (do the two effects combine additively?) Needs the both-yes mean No
Marginal (main-effect) means for reminder or incentive Each averages over a cell that does not exist No

If you assumed no interaction, the missing cell would be predicted at 43.24 + 4.88 + 9.14 = 57.26. The cell that was actually observed in the full data is 62.82, so that assumption would have understated the combined effect by 5.56 points. The additive prediction is an assumption you impose, not a test, which is why the missing cell cannot be replaced by modelling.

What to do instead

  • Run the missing group if the study is still in progress. It is the only way to estimate the interaction.
  • Reframe as a three-group comparison with planned contrasts, and report the simple effects above rather than main effects.
  • Fit an additive model only if theory says the factors cannot interact, and state that this is an assumption.
  • Report the limitation plainly: the design cannot show whether the factors combine additively. Planning for this up front is easier; see design of experiments.

When a full factorial isn’t the right tool

A full factorial is the right design when the factors can be freely and independently randomized and the number of factors is small enough that every combination is practical to run. It’s not always the fit: when there’s a nuisance variable to control for rather than a factor to test, a blocking design handles one nuisance dimension and a Latin square design handles two at once, without needing a full crossed factorial. When one factor is expensive or physically difficult to randomize at the same fine grain as the others — an oven temperature, a field’s irrigation regime — a split-plot design keeps the factorial structure but randomizes the awkward factor at a coarser level, producing two separate error terms rather than the single pooled error term used above. And once the number of factors climbs past three or four, a full factorial’s cell count grows fast enough that a fractional factorial, trading away estimability of some higher-order interactions for a fraction of the runs, is usually the more realistic choice.

Frequently asked questions

Is a 2×2 factorial design the same as a 2×2 contingency table?

No — they’re easy to confuse because both are 2×2 grids, but they answer different kinds of questions. A 2×2 factorial design is an experimental structure: two manipulated factors, each with two levels, and a continuous (or otherwise measured) outcome analyzed with ANOVA, as above. A 2×2 contingency table (see also the worked 2×2 table in the sensitivity/specificity guide) cross-tabulates two categorical variables and is analyzed with a chi-square test or an odds ratio, with no assumption that either variable was experimentally manipulated. The “2×2” in each case describes a different kind of grid.

How many participants do I need for a 2×2 factorial design?

It depends on which effect you actually care about detecting. If the main effects are the target, a conventional power analysis for a two-group comparison, then doubled into the factorial structure, is usually adequate. If the interaction is the actual research question, plan for roughly four times the sample size a same-size main effect would need — see the power section above — and run the calculation for the specific interaction contrast, not just the overall design.

What’s the difference between a full factorial and a fractional factorial design?

A full factorial runs every possible combination of every factor level. A fractional factorial deliberately runs only a systematically chosen subset, which keeps the study feasible when there are many factors, at the cost of confounding (aliasing) some higher-order interactions with each other or with main effects — usually an acceptable trade-off, since higher-order interactions are both rarer and harder to interpret substantively than main effects or two-way interactions. For how factorial layouts fit into the wider planning process, including factor selection, replication and run order, see design of experiments.

Can a factorial design have more than two factors?

Yes — a factorial design can include any number of factors, each with any number of levels; a three-factor design with two levels each is written 2³ and has eight cells. The 2×2 is the version most commonly taught and reported because it’s the smallest case that actually contains an interaction, but the same main-effect/interaction logic scales directly to more factors and more levels, just with more terms in the ANOVA table.

Can I test the interaction if one group is missing from my 2×2 design?

No. The interaction needs all four cell means, so with an empty cell it is not estimable. You can still compare the three observed groups with planned contrasts and report simple effects; see the missing-cell section above.

What is the difference between a main effect and an interaction effect?

A main effect is one factor’s average effect across the other factor’s levels. An interaction means that factor’s effect differs depending on the other factor’s level. Parallel lines on a plot suggest no interaction; lines that fan out or cross suggest one. A large interaction may call for moderation-style interpretation instead of reading main effects alone.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about Factorial Designs: Testing Multiple Factors and Their Interactions at Once

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →