Written and maintained by CASRAI Editorial Board
Last updated
Stepwise regression is a way of choosing predictors for a regression model by adding or removing one term at a time according to a fixed rule, such as a p-value threshold or AIC. This page gives the numbered steps for forward selection, backward elimination and bidirectional (stepwise) selection, how to set the entry and removal thresholds, a worked trace on real data, and runnable recipes for Stata, R and Python. For why reviewers push back on the method, see why reviewers reject stepwise regression; this page is the procedure.
What are the steps of stepwise regression?
Every variant repeats one cycle: fit the current model, test each candidate change, make the single best change if it passes a threshold, and stop when no change passes. The variants differ only in the starting model and in which changes are allowed. The Stata manual entry [R] stepwise defines them through two options: pe(#), the significance level for addition, and pr(#), the significance level for removal. Specifying only pe() gives forward selection, only pr() gives backward selection, both give backward stepwise, and both plus the forward option give forward stepwise.
Forward selection, step by step
- Choose the candidate predictors in advance, from subject-matter reasoning, and set an entry level (pe), for example 0.10.
- Fit the constant-only (intercept-only) model.
- For each candidate not in the model, fit the model with that term added and record its p-value.
- If the smallest p-value is below pe, add that term. Stata describes this as adding the most significant term when p < pe().
- Repeat steps 3 and 4 with the enlarged model. Stop when no remaining candidate has p < pe. Terms already added are never dropped in pure forward selection.
Backward elimination, step by step
- Choose the candidate predictors and a removal level (pr), for example 0.20.
- Fit the full model containing every candidate.
- Find the term with the largest p-value (the least significant).
- If that p-value is greater than or equal to pr, remove the term. The Stata manual states the rule as removing a term when p >= pr().
- Refit and repeat. Stop when every remaining term has p below pr. Removed terms are never re-entered in pure backward elimination.
Bidirectional (stepwise) selection, step by step
Bidirectional selection combines the two. After each addition, the procedure re-checks whether any term already in the model has become non-significant and removes it. This is Stata’s pr() pe() form (backward stepwise, starting from the full model) or pr() pe() forward (forward stepwise, starting from the empty model); R’s step() with direction = "both" is the AIC-based analogue.
- Choose candidates, an entry level pe and a removal level pr.
- Start from the empty model (forward stepwise) or the full model (backward stepwise).
- Run one forward step as above; add the best term if p < pe.
- Run one backward step as above; drop the worst term if p >= pr.
- Stop when neither step changes the model.
Editorial note on thresholds: if pe were larger than pr, a term could pass the entry test and then fail the removal test at once, so the procedure could cycle. Setting the entry level stricter than (below) the removal level, as in the Stata manual’s pr(.2) pe(.1) quick-start example, avoids that.
Choosing entry and removal criteria
| Tool | Criterion that drives each step | Control |
|---|---|---|
Stata stepwise |
Wald test p-value (likelihood-ratio test with the lr option) |
pe(#) addition level, pr(#) removal level; at least one is required |
R stats::step() |
AIC via extractAIC, not p-values |
direction = “both”, “backward” or “forward”; scope; penalty k (k = 2 is AIC; k = log(n) is sometimes called BIC or SBC) |
| Python (statsmodels) | A p-value loop over OLS fits, written by the analyst (sketch below) | Whatever thresholds you code (see snippet below) |
The difference matters when you write the methods section: a Stata run with pe(.1) and an R step() run are different selection rules and can return different models from the same data.
Worked trace: backward elimination on the 1978 automobile data
The Stata manual’s first example uses the 74-car automobile dataset and the command stepwise, pr(.2): regress mpg c.weight##c.weight displ gear turn headroom i.foreign price. The starting model holds every candidate, and the manual’s output shows each decision in order:
| Step | Least significant term | p-value | Rule (pr = 0.20) | Decision |
|---|---|---|---|---|
| 1 | headroom | 0.7116 | 0.7116 ≥ 0.2000 | Remove |
| 2 | displacement | 0.6138 | 0.6138 ≥ 0.2000 | Remove |
| 3 | price | 0.3278 | 0.3278 ≥ 0.2000 | Remove |
| 4 | (all remaining terms below 0.20) | – | Nothing to remove | Stop |
The final model has F(5, 68) = 33.39, R² = 0.7106 and adjusted R² = 0.6893. Check the arithmetic: model SS 1736.31455 divided by total SS 2443.45946 gives 0.7106, and 1 − (1 − 0.7106) × 73/68 = 0.6893, where 73 is n − 1 and 68 the residual degrees of freedom. The same manual then shows that grouping engine displacement and gear ratio as one term, (displ gear), changes the path (those two are removed together at p = 0.3944) and the final model (F(4, 69) = 40.76, R² = 0.7026). Two defensible runs on one dataset gave two different final models, which is why the grouping rule, and your own pre-specification, should be reported.
Running it in Stata, R and Python
Stata
stepwise, pr(.2): regress y x1 x2 x3 x4 // backward
stepwise, pe(.1): regress y x1 x2 x3 x4 // forward
stepwise, pr(.2) pe(.1): regress y x1 x2 x3 x4 // backward stepwise
stepwise, pr(.2) pe(.1) forward: regress y x1 x2 x3 x4 // forward stepwisePut variables in parentheses to enter or remove them together, for example the dummy variables of one categorical predictor.
R
full <- lm(y ~ x1 + x2 + x3 + x4, data = d)
null <- lm(y ~ 1, data = d)
step(null, scope = list(lower = null, upper = full), direction = "both")The R documentation notes that all models must be fitted to the same data, so remove rows with missing values first. This selects by AIC, not by p-value.
Python (statsmodels)
import pandas as pd, statsmodels.api as sm
def stepwise(X, y, p_enter=0.05, p_remove=0.10):
chosen = []
while True:
changed = False
rest = [c for c in X.columns if c not in chosen]
pvals = pd.Series({c: sm.OLS(y, sm.add_constant(X[chosen + [c]])).fit().pvalues[c]
for c in rest}, dtype=float)
if len(pvals) and pvals.min() < p_enter:
chosen.append(pvals.idxmin()); changed = True
if chosen:
fit = sm.OLS(y, sm.add_constant(X[chosen])).fit()
worst = fit.pvalues.drop("const").idxmax()
if fit.pvalues[worst] >= p_remove:
chosen.remove(worst); changed = True
if not changed:
return chosenThis is a teaching sketch of the bidirectional cycle above, with p_enter below p_remove. On a simulated test set in which only two of four predictors truly drive the outcome, it returned exactly those two.
How to determine which variables to include in a regression
Stepwise selection answers a narrower question than “which variables belong in the model”. Heinze, Wallisch and Dunkler (2018) review variable selection for practising statisticians, note that selection is often considered with roughly 10 to 30 candidate covariates, and warn that selection can undermine model stability, unbiased coefficients and the validity of p-values and confidence intervals; they recommend stability checks and resampling. If the goal is estimating one exposure’s effect, choose adjustment variables from a causal diagram instead (see DAG-based confounder selection). If the goal is prediction, check the candidate count against your sample with events per variable and guard against overfitting. For the general model, see linear and multiple regression.
How to report a stepwise selection
- The full candidate set and how it was chosen before looking at the outcome.
- The direction (forward, backward, bidirectional), the criterion (p-value, AIC or BIC) and the entry and removal thresholds.
- The software and command or function, with version.
- The order in which terms entered or left, as in the trace table above.
- That p-values and confidence intervals from the selected model are conditional on the selection process.
Example wording: “Predictors were selected by backward elimination from eight candidates using a removal level of p = 0.20 (Stata stepwise, pr(.2)); three terms were removed (headroom, displacement, price).”
FAQ
What is the difference between forward, backward and stepwise selection?
Forward selection starts empty and only adds terms; backward elimination starts full and only removes terms; stepwise (bidirectional) selection allows both additions and removals at each cycle.
What threshold should I use?
There is no universal value. The Stata manual’s quick-start examples use removal at 0.2 and entry at 0.1. Whatever you choose, state it before analysis and report it.
Does R’s step() use p-values?
No. It compares AIC (or a penalised variant set by k), so it is not the same rule as Stata’s stepwise.
How do I run stepwise regression in Python?
Write the cycle yourself as a loop over statsmodels OLS fits and p-values, as in the sketch above, and report the thresholds you coded.
Sources
- StataCorp. [R] stepwise: Stepwise estimation. Stata manual, https://www.stata.com/manuals/rstepwise.pdf (read 2026-11-07).
- R Core Team. step: Choose a model by AIC in a stepwise algorithm. R stats documentation, https://stat.ethz.ch/R-manual/R-devel/library/stats/html/step.html (read 2026-11-07).
- Heinze G, Wallisch C, Dunkler D. Variable selection: a review and recommendations for the practicing statistician. Biometrical Journal. 2018;60(3):431-449. doi:10.1002/bimj.201700067 (PMC5969114).
