Repeated-Measures ANOVA Assumptions: Normality and Outliers in the Difference Scores

Repeated-measures ANOVA (RM-ANOVA) has more than one assumption, and sphericity is not the only one that fails silently. Two others sit specifically on the difference scores between conditions — the same pairwise differences sphericity itself is built from — and neither gets the attention sphericity does: the difference scores need to be approximately normally distributed, and they need to be free of extreme outliers. This guide covers those two, plus the design-level assumptions RM-ANOVA shares with any ANOVA, and the specific response to a violation of each. For sphericity itself — what it means, how SPSS reports Mauchly’s test, and the Greenhouse-Geisser/Huynh-Feldt correction decision — see Mauchly’s Test of Sphericity in SPSS; this page deliberately does not re-cover that ground.

The full assumption set for repeated-measures ANOVA

A within-subjects design with three or more repeated conditions carries five assumptions in total:

1. Interval or ratio measurement of the dependent variable — RM-ANOVA needs a continuous outcome, not ranks or categories.
2. Independence between subjects — one subject’s scores don’t influence another’s. (Scores within a subject are expected to be correlated across conditions; that correlation is the whole reason a repeated-measures design exists, and it’s what sphericity actually constrains.)
3. Sphericity — equal variance of the pairwise difference scores across all condition pairs. Covered in full in the linked guide above.
4. Normality of the difference scores — not the raw scores in each condition, the pairwise differences between them.
5. No extreme outliers in the difference scores — again, in the differences, not the raw values.

Assumptions 4 and 5 are the ones this guide focuses on. Both are checked on the same pairwise differences sphericity is checked on (condition A minus B, B minus C, A minus C, and so on) — which is a genuinely easy point to miss if you’ve only ever checked normality on raw condition scores in a between-subjects design.

Why the differences, not the raw scores

A one-way between-subjects ANOVA checks normality of the residuals within each group. A repeated-measures design has no independent groups to check that way — every subject contributes a score to every condition, so the relevant unit of analysis for these two assumptions is what changes within a subject from one condition to the next, not each condition’s raw distribution on its own. Two conditions can each look perfectly normal individually while their difference is heavily skewed (a ceiling or floor effect in one condition compresses the range of change for subjects who started near that ceiling), and the reverse can also happen. Test the differences directly — don’t infer their shape from the raw conditions.

Checking normality of the difference scores

For each pair of within-subject conditions, compute the difference score per subject, then assess that set of differences the same way you’d assess normality anywhere else:

• Shapiro-Wilk test on each difference-score set — the standard formal test for small-to-moderate samples (roughly n < 50). In SPSS: Analyze → Descriptive Statistics → Explore, with the computed difference variable in the Dependent List and Normality plots with tests checked under Plots. In R, compute the differences with diff <- df$condB - df$condA and run shapiro.test(diff).
• Q-Q plot of each difference-score set — a visual check that’s often more informative than a single p-value, especially at small n where Shapiro-Wilk has little power to detect real non-normality and can also flag trivial deviations as significant with a slightly larger sample.
• Skewness as a descriptive supplement — a difference-score skewness statistic more than roughly ±1 to ±2 (conventions vary; treat it as a flag to look at the Q-Q plot, not a hard cutoff on its own) suggests the normality assumption is genuinely strained rather than borderline.

Do this separately for every pairwise difference implied by your within-subjects factor — a factor with four levels has six pairwise differences, and normality can hold for some pairs and fail for others.

Violation response: non-normal difference scores

RM-ANOVA is moderately robust to non-normality, particularly as sample size grows (the F-test’s sampling distribution approaches normal via the central limit theorem even when the underlying differences aren’t perfectly normal) and particularly for mild-to-moderate skew. The response scales with how severe the violation is and how small the sample is:

• Mild skew, reasonable sample size (roughly n ≥ 30 per condition, no dramatic outliers) — proceed with RM-ANOVA as planned; note the check in your methods section. This is the majority case.
• Clear non-normality, small sample, or a heavy-tailed/highly skewed distribution — switch to the Friedman test, the rank-based nonparametric analogue of one-way RM-ANOVA, with Nemenyi or Wilcoxon-Bonferroni for pairwise follow-up. It doesn’t assume normality of the differences at all, at the cost of testing ranks rather than means.
• You need to retain an interval-scaled effect estimate rather than switch to ranks — a data transformation (log, square-root, or a rank-based inverse-normal transform) applied consistently across all conditions before differencing is a middle option, though it complicates interpretation of the original units and should be reported explicitly as a transform, not silently applied.

Checking for outliers in the difference scores

An outlier in a difference score is a within-subject change that’s extreme relative to how much other subjects changed — not necessarily a subject with an extreme raw score in any single condition. A subject who scores highest in every condition is not an outlier on the differences if their pattern of change from condition to condition matches everyone else’s.

The standard check is the same 1.5×IQR boxplot rule used anywhere else in exploratory analysis, applied to each pairwise difference-score set: compute Q1 and Q3 of the differences, the IQR (Q3 − Q1), and flag any value below Q1 − 1.5×IQR or above Q3 + 1.5×IQR. SPSS draws this automatically as a boxplot of the difference variable under Explore; in R, boxplot(diff) or boxplot.stats(diff)$out returns the flagged values directly.

Violation response: outliers in the difference scores

• Investigate first, always. Check whether a flagged case is a data-entry or measurement error before doing anything else — a genuine error should be corrected or excluded on that basis alone, not because it’s statistically inconvenient.
• If it’s a genuine extreme value, run the analysis both with and without it and report whether conclusions change (a sensitivity analysis). If the result is stable either way, the outlier isn’t consequential enough to justify excluding a real data point.
• If one or two cases are driving the result, a nonparametric alternative — again, the Friedman test — is less sensitive to extreme values than the mean-based F-test, since it works on ranks.
• Winsorizing (capping extreme differences at a fixed percentile rather than deleting the case entirely) is a defensible middle ground that keeps the subject’s data in the analysis while limiting the outlier’s leverage — report the winsorizing rule used.
• Never drop a case silently. Whatever you decide, report how many cases were flagged, on what rule, and what was done about them — that disclosure is what separates legitimate outlier handling from post-hoc cherry-picking of a significant result.

Worked example: a synthetic dataset (illustrative, not real study data)

The numbers in this section are a synthetic dataset generated with a fixed random seed specifically to illustrate the checks above — not data from any real study, published or otherwise. One case (subject 9) was deliberately altered to inject an outlier for demonstration; every statistic below is directly computed from the listed values, not invented to “look right.”

14 subjects, one within-subjects factor (a fatigue-score scale) at three timepoints. Difference scores were computed for all three pairs, then checked with the 1.5×IQR rule:

Difference pair	Mean	SD	Skewness	IQR fence	Flagged
T2 − T1	−4.8	7.8	2.6	[−12.7, −0.6]	Subject 9 (+21.4)
T3 − T2	−4.5	9.1	−2.4	[−10.3, 6.5]	Subject 9 (−34.0)
T3 − T1	−9.3	2.7	−0.7	[−15.0, −2.7]	Subject 13 (−15.8)

Two different findings here, and they call for two different responses. Subject 9 is flagged on both T2−T1 and T3−T2 with a skewness statistic well past ±2 on each — a single case distorting both the normality and the outlier check simultaneously, exactly the pattern a data-entry error produces (in this synthetic case, that’s literally what it is: the value was inserted to demonstrate the point). The right first move is to check the raw record, not run a robust test around it. Subject 13’s flag on T3−T1 is milder (just outside the fence, no comparable skew distortion) and more consistent with genuine extreme-but-real variation — the kind a sensitivity analysis (report the result with and without subject 13) is the proportionate response to, not exclusion.

Frequently asked questions

What are all the assumptions of repeated-measures ANOVA?

Five: interval/ratio measurement, independence between subjects, sphericity, normality of the difference scores between conditions, and no extreme outliers in those difference scores. Sphericity is the one unique to repeated-measures designs; the other four apply, in some form, to ANOVA generally.

Does repeated-measures ANOVA assume the raw scores are normally distributed?

No — it assumes the difference scores between conditions are approximately normal, not the raw scores within each condition. A condition-by-condition normality check is testing the wrong thing for this design.

What do I do if my repeated-measures ANOVA violates normality?

With a reasonable sample size and only mild skew, proceed as planned — RM-ANOVA is reasonably robust to moderate departures. With a small sample or clear non-normality, switch to the Friedman test, the nonparametric rank-based alternative.

How many outliers can repeated-measures ANOVA tolerate?

There’s no fixed count — it depends on how much leverage the flagged case(s) have on the result. Run the analysis with and without a flagged case: if the conclusion doesn’t change, it isn’t consequential; if it does, investigate before deciding whether to exclude, winsorize, or switch to a rank-based test.

Is sphericity the only assumption unique to repeated-measures ANOVA?

Yes, among the five — normality and outlier-freedom of difference scores are versions of assumptions every ANOVA design carries, just applied to differences instead of raw group scores because that’s the unit a within-subjects design actually varies on.

Related CASRAI guides

• Mauchly’s Test of Sphericity in SPSS — the assumption unique to repeated-measures designs, covered in full.
• Repeated-Measures ANOVA in SPSS: Setup and Corrections — running the procedure itself.
• Testing Normality in SPSS — Shapiro-Wilk, Kolmogorov-Smirnov, and the diagnostic plots, applicable to any variable including a computed difference score.
• The Friedman Test and Its Post-Hoc Procedures — the nonparametric alternative when normality or outlier assumptions fail.
• MANOVA in SPSS — a multivariate approach that avoids the sphericity assumption entirely and is worth considering alongside these fixes when several assumptions are strained at once.
• ANOVA: One-Way, Two-Way, Assumptions, Kruskal-Wallis and Effect Size — the general-ANOVA assumption set this page builds on.