Stepwise Regression: Why Reviewers Reject It (and What to Use Instead)

Stepwise regression is one of the few statistical methods that draws a specific, predictable objection from peer reviewers. Not “your model needs more controls” or “please justify this transformation” — the objection is to the selection procedure itself, and it shows up in biomedical, social-science, and ecology journals alike. This guide covers exactly what the objection is, why it is statistically well-founded rather than a matter of taste, and which alternatives actually satisfy a reviewer who raises it.

What “stepwise regression” means

Stepwise regression is a family of algorithms that build a regression model by adding or removing predictors one at a time, based on a statistical criterion computed after looking at the outcome data — usually a p-value threshold for entry/removal, sometimes AIC or BIC. Three variants cover most usage:

• Forward selection starts with no predictors and adds the one that most improves fit at each step, stopping when no remaining candidate clears the entry threshold.
• Backward elimination starts with every candidate predictor in the model and removes the least useful one at each step, stopping when every remaining predictor clears the retention threshold.
• Bidirectional (mixed) stepwise alternates forward and backward steps, allowing a variable added earlier to be dropped later if a subsequent addition makes it redundant.

All three share the same structural feature that draws the objection: the final model is chosen by the same data used to test it, and the selection path is not part of what gets reported.

Why reviewers flag it

The criticism is not that stepwise selection “sometimes gets it wrong.” It is that the procedure’s own logic guarantees specific, quantifiable distortions in the reported model — distortions that don’t show up as an error in any individual step, only in what the final, selected model implies about the population.

Inflated Type I error from repeated testing

Each step of a stepwise procedure is a hypothesis test conducted at the nominal significance threshold (commonly 0.05), but the procedure runs many such tests across many candidate predictors and does not correct for that multiplicity. If you offer the algorithm, say, 20 candidate predictors with no true relationship to the outcome at all and test each at α = 0.05, the expected number that clear the bar by chance alone is α × k ≈ 1 — not zero. With a more realistic mix of genuinely predictive and noise candidates, the same mechanism inflates the effective false-positive rate for the variables that do make it into the final model well above the nominal 0.05, because the reported p-values are computed as though the model had been specified in advance, not selected from a search over many candidates. This is the same underlying problem documented under researcher degrees of freedom and the garden of forking paths: a large number of defensible-looking analytic choices, made after seeing the data, that individually look reasonable but collectively produce results that don’t replicate.

Unstable model selection

Because the algorithm’s choices at each step depend on which predictors happen to correlate most strongly with the outcome in this particular sample, a different sample from the same population — or even a bootstrap resample of the same data — frequently produces a different final model. Derksen and Keselman’s frequently-cited 1992 comparison of forward, backward, and stepwise subset-selection algorithms found that all three routinely retained variables with no real relationship to the outcome and dropped variables that did have one, with the problem worsening as candidate predictors became more correlated with each other — the exact condition (multicollinearity among candidates) that is common in observational research and that makes stepwise selection most attractive to run in the first place. A model whose composition changes depending on which 80% of the data you happened to sample is not a stable description of anything.

Overstated R² and downward-biased standard errors

Because the retained predictors are, by construction, the ones that fit this sample’s noise best (in addition to any real signal), the model’s R² and the precision implied by its standard errors both overstate what the model will achieve on new data. The standard errors and p-values reported for the final model are computed as if that model had been the only one tested — they take no account of the search that produced it — so confidence intervals built from a stepwise model are narrower, and its coefficients look more precise, than the selection process actually earned. Frank Harrell’s Regression Modeling Strategies devotes sustained attention to this point: the reported model looks better calibrated than it is precisely because the fitting and the reporting share the same data with no adjustment for the search in between.

The algorithm you choose changes the answer

Forward selection, backward elimination, and bidirectional stepwise do not reliably converge on the same final model from the same data and the same candidate set — the order in which variables are considered can determine which correlated variable “wins” a spot in the model, particularly when candidates show meaningful collinearity. A method whose output depends on which of three broadly equivalent algorithms you happened to run is difficult to describe as having found “the” best model.

What the reviewer comment usually looks like

Reviewer language varies, but comments on stepwise selection converge on a recognizable shape: a request to either drop the stepwise procedure in favor of a pre-specified model, or to report what was actually done in enough detail to let a reader judge the multiplicity involved — how many candidate predictors were offered to the algorithm, what entry/removal criterion was used, and whether the reported p-values and confidence intervals were adjusted for the search. A paper that used stepwise selection but reports the final model’s coefficients and p-values as though no search occurred is the version of this that draws the strongest objection, because the omission hides exactly the information a reviewer needs to evaluate how much the result should be discounted — it is the same underlying transparency gap that draws an objection when it shows up as p-hacking in a different analytic context.

Defensible alternatives

Theory-driven variable entry

Specify the model’s predictors before looking at the outcome data, based on subject-matter theory, prior literature, or a pre-registered analysis plan, and report that single specified model. This is the alternative reviewers most often ask for directly, because it removes the multiplicity problem at its source rather than trying to correct for it after the fact — there was only ever one model tested, so the reported inference is exactly what the standard errors and p-values assume.

Hierarchical block entry

Group candidate predictors into theoretically motivated blocks (e.g., demographic covariates, then clinical/exposure variables, then a variable of primary interest) and enter the blocks in a fixed, pre-specified order, testing the change in R² each block contributes. The order is set by theory before the data are examined, not by which variables fit best — the incremental-R² test at each step answers a specific, pre-planned question rather than searching for whichever combination fits best.

LASSO and other regularized regression

LASSO (least absolute shrinkage and selection operator) regression, introduced by Tibshirani in 1996, performs variable selection and coefficient estimation simultaneously by penalizing the size of the coefficients rather than testing each one for significance. It handles a large candidate-predictor set without repeated significance testing at each step, and the penalty term is chosen by cross-validation rather than a stepwise p-value threshold — which avoids the specific multiple-testing mechanism reviewers object to in classical stepwise selection, though it introduces its own reporting obligations (the standard errors from a penalized model require a separate adjustment; naively treating LASSO-selected coefficients as if from an unpenalized model repeats a version of the same overstatement problem).

Reporting the full model

When the candidate-predictor set is small relative to the sample size, the simplest defensible option is often to report the full model with every candidate predictor included and skip selection entirely. This sacrifices parsimony but sidesteps the entire selection-inference problem, since there was no search to adjust for.

If you already ran stepwise regression

A manuscript that used stepwise selection is not automatically unpublishable — the practical path through a reviewer objection is usually one of three responses, and which one fits depends on what data you still have access to:

• Re-run with a pre-specified model if the theoretical basis for one exists and the data are still available — this is the response reviewers most often accept without further back-and-forth.
• Report the selection process transparently if a pre-specified alternative genuinely isn’t available: state the full candidate-predictor list, the entry/removal criterion and threshold, and frame the reported model explicitly as exploratory/hypothesis-generating rather than confirmatory — reviewers who cannot get a re-analysis can often be satisfied by an honest account of what the procedure actually did.
• Validate the selected model on held-out data if a split-sample or cross-validation check is feasible — a stepwise-selected model whose performance is confirmed on data it did not help select addresses the instability objection directly, even though it doesn’t address the Type I error inflation on its own.

Frequently asked questions

Is stepwise regression ever acceptable in a published paper?

Reviewers are most likely to accept it when it is explicitly framed as exploratory or hypothesis-generating rather than confirmatory, with the full candidate list and selection criterion reported, and ideally validated on held-out data. What draws rejection is presenting a stepwise-selected model’s p-values and confidence intervals as though they came from a single pre-specified test.

Is backward elimination less problematic than forward selection?

No — both are stepwise procedures that select the final model using the same outcome data used to test it, and both are subject to the same Type I error inflation and selection instability. Backward elimination is sometimes preferred on statistical grounds (it starts from the full model and so is less prone to omitting a predictor that only matters in combination with others), but that is a difference in which mistakes each algorithm tends to make, not an exemption from the underlying multiplicity problem.

Does using AIC or BIC instead of a p-value threshold fix the problem?

It removes the explicit repeated-hypothesis-testing framing, since AIC/BIC-based stepwise selection is not literally running a significance test at each step, but the selection is still a data-driven search over many candidate models using the same data that will be used to report and interpret the winner, so the resulting model’s apparent precision is still overstated relative to what the search actually earned. Reviewers who object to stepwise selection generally object to this version too, for the same underlying reason.

Is LASSO regression a form of stepwise regression?

No. LASSO selects variables by shrinking coefficients toward zero through a penalty term tuned by cross-validation, in a single optimization, rather than through a sequence of significance tests that add or remove one variable at a time. It is generally treated by reviewers as a more defensible alternative to stepwise selection, though a LASSO-selected model’s standard errors still need proper post-selection adjustment before being reported as though from an unpenalized regression.

Does hierarchical regression avoid the stepwise problem?

It avoids the specific problem of the computer choosing which variables enter, since the researcher pre-specifies both the blocks and their entry order based on theory before seeing the results. It does not eliminate the underlying multiple-testing concern if the blocks or their ordering were themselves chosen after looking at the data — the defensibility depends on the ordering genuinely being fixed in advance.

What should I report if a reviewer asks me to justify a stepwise-selected model?

At minimum: the full list of candidate predictors offered to the algorithm, the specific variant used (forward, backward, or bidirectional), the entry/removal criterion and its threshold, and an explicit statement of whether the reported inferential statistics were adjusted for the selection process. Most reviewer objections are really requests for this information, not a demand to discard the analysis outright.

Sources

• Whittingham, M.J., Stephens, P.A., Bradbury, R.B., and Freckleton, R.P. (2006). “Why do we still use stepwise modelling in ecology and behaviour?” Journal of Animal Ecology, 75(5), 1182–1189.
• Derksen, S., and Keselman, H.J. (1992). “Backward, forward and stepwise automated subset selection algorithms: Frequency of obtaining authentic and noise variables.” British Journal of Mathematical and Statistical Psychology, 45(2), 265–282.
• Flom, P.L., and Cassell, D.L. (2007). “Stopping stepwise: Why stepwise and similar selection methods are bad, and what you should use.” Presented at NESUG (Northeast SAS Users Group).
• Harrell, F.E. Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. Springer.
• Tibshirani, R. (1996). “Regression Shrinkage and Selection via the Lasso.” Journal of the Royal Statistical Society, Series B, 58(1), 267–288.