Skip to main content
v2026.11,986 entries · CC-BY 4.0

Nonequivalent Control Group Design: Definition, Threats and Analysis

What a nonequivalent control group design is, which selection threats apply, and how to analyse it with difference-in-differences or ANCOVA.

Written and maintained by CASRAI Editorial Board

Published

On this page

A nonequivalent control group design (NEGD) is a quasi-experimental design that compares a group that receives an intervention with a comparison group that does not, where participants were not randomly assigned to the two groups. Because the groups are usually intact units such as two classrooms or two clinics, they can differ before the intervention starts, so the design normally adds a pretest and a posttest and asks whether the treated group changed more than the comparison group.

What is a nonequivalent control group design?

Trochim’s Research Methods Knowledge Base describes the nonequivalent groups design as structured like a pretest-posttest randomized experiment but lacking its key feature, random assignment. The groups are typically intact and believed to be similar, such as two comparable classrooms. The cost is stated plainly in the same source: the design is especially susceptible to the internal validity threat of selection, meaning that any pre-existing difference between the groups can look like a treatment effect.

A widely used psychology methods text defines it as a between-subjects design in which participants have not been randomly assigned to conditions, and gives the reason the groups may differ: for example, the parents of unusually motivated students may be the ones who request a particular class. This guide covers the parts a researcher needs in order to use the design: the variants, the threats, the analysis choices, a worked calculation and how to report the result. For the wider choice between designs, see the comparison of experimental and quasi-experimental designs.

Pretest-posttest and posttest-only variants

In the pretest-posttest form, the treatment group is measured, receives the treatment and is measured again; the comparison group is measured at the same two times without the treatment. The key question is whether the treatment group improves more than the comparison group. In the posttest-only form, the groups are compared only after the intervention. A review in Nurse Researcher (Krishnan) states that this posttest-only design can be used in natural settings where randomisation cannot be conducted for ethical or practical reasons, and that it is vulnerable to threats to internal validity. With no pretest, nothing in the data shows whether the groups already differed, so the pretest-posttest form is the stronger of the two.

How it differs from a randomized experiment

Random assignment makes groups equivalent on average before treatment. Without it, the comparison group is a stand-in, not a mirror image. The role of the control group is the same, but its credibility rests on argument and evidence about similarity, not on the assignment mechanism.

Threats to validity: the selection interactions

Trochim names three interaction threats that arise because the groups were not randomly formed. Each one produces a pretest-to-posttest change in one group that is not caused by the treatment. The data table below the sources lists them with the source’s own wording of the mechanism.

Selection-maturation

The groups mature at different rates, which creates the appearance of an effect. A treatment group that is developing faster than the comparison group will gain more even with no treatment. The general idea of maturation as a threat to internal validity applies here, with the added problem that the rate differs between groups.

Selection-history

The groups react differently to an outside event that happens between the pretest and the posttest. A policy change, a news event or a local disruption can move one group and not the other.

Selection-regression

When the groups are chosen because of extreme pretest scores, they drift toward the mean on retest: the high-scoring group tends to decline and the low-scoring group tends to rise. The LibreTexts psychology text illustrates regression to the mean with a bowler who averages 150 and bowls 220 on one occasion: an extreme score on one occasion tends to be followed by a less extreme score on the next. If your treatment group was selected for low scores and your comparison group was not, regression alone can create an apparent improvement.

Reading the pattern of results

Trochim’s page works through four possible outcome patterns and shows that the shape of the results changes which threats are plausible. The table below states the reading guidance that follows from the threat definitions above.

Pattern in the data What it suggests Basis
Treatment group rises while the comparison group stays flat Selection-maturation is hard to argue, because a maturation story needs the comparison group to be changing too Trochim, outcome pattern 1
Both groups rise, treatment group more The raw gap mixes treatment with whatever is moving both groups; a differential-growth explanation stays open Editorial reading of the selection-maturation definition
Groups were selected on extreme pretest scores and the extreme group moves toward the middle Regression to the mean is a live alternative explanation Trochim and LibreTexts
One group changes after an external event that touched only that group Selection-history is a live alternative explanation Trochim

Nonequivalent control group design analysis

There are three common ways to turn the two-group, two-time data into one effect estimate. Which is credible depends on how the groups were formed and how reliable the pretest is.

Change scores and difference-in-differences

The change-score approach subtracts the comparison group’s pretest-to-posttest change from the treatment group’s change. This is the difference-in-differences (DiD) estimate: DiD = (PostT − PreT) − (PostC − PreC), where T is the treatment group and C the comparison group, and Pre and Post are group means. Its logic is that whatever pre-existing difference exists is removed by differencing, provided the two groups would have changed by the same amount without the treatment. That assumption is the one that fails under selection-maturation. The CASRAI guide to difference-in-differences and parallel trends covers testing it.

ANCOVA with the pretest as covariate

Trochim describes the intuitive model as an analysis of covariance: regress the posttest on the pretest and a group dummy, and read the group coefficient as the adjusted group difference on the posttest. With a slope b relating pretest to posttest, the adjusted difference equals the raw posttest gap minus b times the raw pretest gap. See ANCOVA in R for the model fitting itself.

The pretest measurement error problem

Trochim states that standard ANCOVA is biased in the nonequivalent groups design. Measurement error in the pretest attenuates the estimated slope, and the bias grows as the groups become more dissimilar; measurement error in the posttest does not bias the slope. In the source’s simulation the true effect was 10, standard ANCOVA estimated 11.28 with the true value outside its 95% confidence interval, and a reliability-corrected ANCOVA estimated 9.30 with the true value inside the interval.

Analysis in the Trochim simulation Estimate True effect (10) inside the 95% CI?
Standard ANCOVA 11.28 No
Reliability-corrected ANCOVA 9.30 Yes

The correction replaces each pretest score X with Xadj = group mean + r(X − group mean), computed separately within each group, where r is the pretest reliability. Trochim recommends running the analysis with an upper-bound reliability (Cronbach’s alpha) and a lower-bound one (test-retest): if the effect is significant under both, it is more trustworthy.

Worked example with arithmetic

The numbers below are an illustrative composite invented for this guide, not data from a real study. Two classes took the same 100-point test before and after a new teaching method; class T received it and class C did not.

Group Pretest mean Posttest mean Change
T (treatment) 50 62 62 − 50 = 12
C (comparison) 54 58 58 − 54 = 4
  • Raw posttest gap: 62 − 58 = 4. This ignores that T started 4 points lower.
  • Raw pretest gap: 50 − 54 = −4.
  • Difference-in-differences: 12 − 4 = 8 points.
  • ANCOVA-style adjustment with an assumed pooled within-group slope of b = 0.6: 4 − 0.6 × (−4) = 4 + 2.4 = 6.4 points. With b = 1 the adjustment equals the DiD result, 4 − 1 × (−4) = 8.

The two estimates differ because they make different assumptions about how pretest differences carry into the posttest. Report both when the choice is not obvious, and state the assumption behind each.

Assumptions and common mistakes

  • Treating the pretest as proof of equivalence. Similar pretest means do not show that groups would have changed at the same rate.
  • Ignoring pretest reliability. Standard ANCOVA with an unreliable pretest can over- or understate the effect, as in the simulation above.
  • Choosing groups on extreme scores. This invites selection-regression. Where possible, select on something other than the outcome measure.
  • Leaving out the pattern check. Look at the shape of the change in both groups, not only the single adjusted number.
  • Overstating causation. Without random assignment, describe the result as an estimate that depends on stated assumptions.

When an intervention affects one group at a known time and many pre- and post-measurements exist, an interrupted time series design adds protection that two time points cannot give.

How to report a nonequivalent control group design (APA style)

State how the groups were formed, that assignment was not random, the pretest comparison, the analysis model and the effect with its interval. An illustrative sentence using the worked example’s numbers:

Classes were not randomly assigned. At pretest, class T (M = 50) scored 4 points lower than class C (M = 54). Using a difference-in-differences estimate, class T gained 8 points more than class C (12 vs. 4), 95% CI [lower, upper].

Replace the bracketed interval with the one from your own model. Add the standard deviations and group sizes your data have, and name the threats you considered.

How to calculate it in Excel, R or Python

Each snippet reproduces the worked example’s DiD of 8 from the four group means.

Excel

With pretest and posttest means in cells B2:C3 (row 2 = T, row 3 = C): =(C2-B2)-(C3-B3) returns 8. For individual data, an ANCOVA can be run with the Regression tool of the Analysis ToolPak, using the pretest and a 0/1 group column as the X range.

R

pre_t = 50; post_t = 62; pre_c = 54; post_c = 58
did = (post_t - pre_t) - (post_c - pre_c)   # 8
# individual data frame d with columns post, pre, group:
# fit = lm(post ~ pre + group, data = d); summary(fit)  # group = adjusted difference

Python

pre_t, post_t, pre_c, post_c = 50, 62, 54, 58
did = (post_t - pre_t) - (post_c - pre_c)   # 8
# individual data in a DataFrame df:
# import statsmodels.formula.api as smf
# fit = smf.ols("post ~ pre + C(group)", data=df).fit(); print(fit.summary())

Sources

Data: Selection-interaction threats in the nonequivalent groups design

Compiled from the Research Methods Knowledge Base page on the nonequivalent groups design and the LibreTexts Research Methods in Psychology chapter on quasi-experimental research; lists each selection-interaction threat with the mechanism and the example the sources give.

Selection-interaction threats in the nonequivalent groups design
ThreatMechanismSource example or pattern
Selection-maturationGroups mature at different rates, creating the appearance of a treatment effectHard to argue when the comparison group stays flat while the treatment group rises (Trochim outcome pattern 1)
Selection-historyGroups react differently to an outside event between pretest and posttestAn external event that affects one group more than the other
Selection-regressionGroups chosen for extreme pretest scores drift toward the mean on retestHigh-scoring group declines and low-scoring group rises; bowler averaging 150 who bowls 220 (LibreTexts)

Sources

Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about Nonequivalent Control Group Design: Definition, Threats and Analysis

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →