Regression analysis is the statistical technique for estimating how one or more predictor variables relate to an outcome variable, expressed as an equation you can use to describe, and sometimes explain, that relationship. It is the single most commonly used inferential technique in quantitative research, which is also why it is the most commonly misreported: a coefficient table with no assumption checks, an R² presented as proof the model is “good,” or a causal verb (“increases,” “drives,” “leads to”) applied to an association from a cross-sectional dataset. This guide covers the part most treatments skip: which regression to run, what each one actually assumes, how to check those assumptions instead of ignoring them, how to interpret the output honestly, and how to write it up so a peer reviewer or methods examiner accepts it on the first pass.
Choosing the right regression model
The choice is driven almost entirely by what kind of variable your outcome (dependent variable) is. Getting this wrong — running linear regression on a binary outcome, for instance — is the single most common structural error in submitted manuscripts using regression.
| Outcome variable | Model | Example outcome |
|---|---|---|
| Continuous, one predictor | Simple linear regression | Predicting reaction time from age |
| Continuous, multiple predictors | Multiple linear regression | Predicting blood pressure from BMI, age, and sodium intake |
| Binary (yes/no, event/no event) | Logistic regression (binomial) | Predicting grant funded vs. not funded |
| Unordered categories (3+) | Multinomial logistic regression | Predicting choice among three career-track outcomes |
| Ordered categories | Ordinal logistic regression | Predicting a 5-point Likert satisfaction rating |
| Count data (non-negative integers) | Poisson or negative binomial regression | Predicting number of publications per year |
| Time-to-event, with censoring | Cox proportional hazards regression | Predicting time to manuscript acceptance |
If you are unsure whether your outcome is “continuous enough” for linear regression: a 5-point or shorter Likert scale is usually better modeled as ordinal; a proportion bounded at 0 and 1 usually needs a beta or logistic-based approach, not raw linear regression, because linear regression can predict values outside the possible range.
The assumptions of linear regression — and how to actually check each one
Linear regression (simple or multiple) rests on five assumptions. Reporting that you “checked assumptions” without saying how is not sufficient for a methods section; each one below has a specific, standard diagnostic.
1. Linearity
What it means: the relationship between each predictor and the outcome is genuinely a straight-line relationship, not curved.
How to check it: plot residuals (the model’s prediction errors) against fitted values, and against each continuous predictor. A random scatter around zero is consistent with linearity; a curve or fan shape is not. If it’s violated, consider a polynomial or log transform of the offending predictor, or a non-linear model.
2. Independence of errors
What it means: one observation’s residual gives you no information about another’s. This is violated by repeated measures on the same subject, clustered data (e.g., students within schools, patients within clinics), or time-series data.
How to check it: for time-ordered data, the Durbin-Watson statistic (values near 2 indicate no first-order autocorrelation). For clustered or repeated-measures data, the fix is structural, not diagnostic — use a mixed-effects/multilevel model or generalized estimating equations instead of ordinary least squares, because no post-hoc test rescues independence once the design violates it.
3. Homoscedasticity (constant variance of residuals)
What it means: the spread of residuals is roughly constant across the range of fitted values, rather than fanning out (heteroscedasticity).
How to check it: the same residuals-vs-fitted plot used for linearity, looking specifically for a funnel shape; formally, a Breusch-Pagan or White test. If violated, options include a variance-stabilizing transform of the outcome (e.g., log), or reporting heteroscedasticity-robust (“sandwich”) standard errors instead of re-running the whole model.
4. Normality of residuals
What it means: the residuals, not the raw outcome variable itself, are approximately normally distributed. This is a common misconception — regression does not require your outcome variable to be normally distributed, only the errors.
How to check it: a Q-Q plot of the residuals (points should track the diagonal), supplemented by a Shapiro-Wilk test in smaller samples. In large samples, normality of residuals matters much less because the Central Limit Theorem protects the sampling distribution of the coefficients — don’t over-index on a Shapiro-Wilk p-value in a sample of several hundred or more.
5. No severe multicollinearity
What it means: in multiple regression, predictors shouldn’t be so strongly correlated with each other that the model can’t separate their individual effects.
How to check it: the Variance Inflation Factor (VIF) for each predictor. A VIF above 5 is commonly flagged for attention; above 10 is a widely used cutoff for a real problem. The practical consequence of high multicollinearity is inflated standard errors on the affected coefficients (unstable, hard-to-interpret estimates), not biased predictions from the model as a whole — a distinction worth stating explicitly if a reviewer raises it.
Logistic regression has a different assumption set
Because the outcome and the estimation method differ, do not check logistic regression against the linear-regression checklist above. Its actual requirements are:
- Linearity of the logit — each continuous predictor should be linearly related to the log-odds of the outcome, not to the outcome itself. Checked via the Box-Tidwell test or by adding predictor × log(predictor) interaction terms and checking their significance.
- Independence of observations — same underlying requirement as linear regression; clustered data still needs a mixed/multilevel logistic model.
- No severe multicollinearity — checked the same way, via VIF among the predictors.
- No complete or quasi-complete separation — a logistic-specific failure mode where a predictor (or combination) perfectly or near-perfectly predicts the outcome, producing coefficients and standard errors that inflate toward infinity. Checked by inspecting for implausibly large coefficients or standard errors in the output.
- Adequate events per variable — a commonly cited rule of thumb calls for at least 10 events (the rarer outcome category) per predictor included in the model, to avoid an overfit, unstable model.
Note that logistic regression does not assume normally distributed residuals or homoscedasticity — these are linear-regression-specific and do not transfer.
Interpreting coefficients and R² without the common overclaims
Unstandardized vs. standardized coefficients
The unstandardized coefficient (B) tells you the expected change in the outcome for a one-unit increase in that predictor, holding other predictors constant. It is in the original units of the predictor, which makes it directly meaningful but not comparable across predictors measured on different scales. The standardized coefficient (beta, or a model run on z-scored variables) rescales each predictor to standard-deviation units specifically so you can compare relative strength of predictors within one model — it should not be interpreted as a real-world effect size on its own.
R² and adjusted R²
R² is the proportion of variance in the outcome explained by the model. Two overclaims to avoid:
- “High R² means the model is correct.” R² says nothing about whether the assumptions hold, whether the relationship is causal, or whether an omitted variable is confounding the result. A model can have a high R² and a badly misspecified functional form.
- “Low R² means the finding isn’t meaningful.” In fields studying complex human or biological behavior (psychology, education, epidemiology), an R² of 0.10–0.20 can be a genuinely useful, statistically and practically significant finding — the appropriate benchmark is field-specific, not a universal threshold.
Adjusted R² corrects for the fact that R² mechanically rises every time you add a predictor, even a useless one. Report adjusted R² whenever comparing models with different numbers of predictors, and expect it to be at or below R², never above.
Odds ratios in logistic regression
Exponentiating a logistic coefficient gives an odds ratio (OR): the multiplicative change in the odds of the outcome per one-unit increase in the predictor. An OR of 1.8 means the odds of the outcome are 1.8 times higher (an 80% relative increase in odds), not an 80% increase in probability — odds and probability diverge, especially as the base rate moves away from 50%. This is one of the most common misreadings in published logistic regression results and worth double-checking in your own write-up.
The overclaim to eliminate entirely: causal language from observational data
Unless your data come from a randomized experiment (or you have applied a design explicitly built for causal inference, such as an instrumental-variable or regression-discontinuity approach), regression coefficients describe association, not causation. Replace “X increases Y” or “X drives Y” with “X was associated with Y” or “higher X predicted higher Y” in any observational analysis. Reviewers in most quantitative fields will flag causal verbs applied to correlational or cross-sectional regression results.
A worked interpretation (illustrative data)
The numbers below are a constructed, illustrative example for teaching interpretation — not a real study, dataset, or reported finding attributed to any institution.
Suppose a multiple linear regression predicts a researcher’s annual publication count from two predictors: years since PhD, and weekly hours protected for research (as opposed to teaching/admin). The (illustrative) output:
| Predictor | B (unstandardized) | SE | t | p | 95% CI |
|---|---|---|---|---|---|
| Years since PhD | 0.18 | 0.05 | 3.60 | <.001 | [0.08, 0.28] |
| Weekly protected hours | 0.09 | 0.03 | 3.00 | .003 | [0.03, 0.15] |
Model: R² = 0.34, adjusted R² = 0.32, F(2, 197) = 50.7, p < .001.
Correct interpretation: “Holding weekly protected hours constant, each additional year since PhD was associated with 0.18 more publications per year on average (95% CI [0.08, 0.28]). Holding years since PhD constant, each additional weekly protected hour was associated with 0.09 more publications per year (95% CI [0.03, 0.15]). Together, the two predictors accounted for approximately 32% of the variance in publication count (adjusted R² = .32).”
Overclaims to avoid with this same output: “Protecting research time causes higher publication output” (causal language from an observational design); “This model explains publication productivity” (32% of variance is a partial account, not an explanation of the construct); treating the two B coefficients as directly comparable in strength without standardizing them first, since they are on different original scales (years vs. hours).
How to report a regression in a methods and results section
A results section a reviewer will accept without a methods query generally includes all of the following, in this order:
- Which model and why. Name the specific regression (simple linear, multiple linear, binomial logistic, etc.) and the outcome and predictor variables, ideally in the methods section, not just the results.
- Sample size and missing-data handling. N used in the model, and how missing data were handled (listwise deletion, multiple imputation, etc.) — this changes the effective N and needs to be stated, not implied.
- Assumption checks, briefly. A sentence naming which assumptions were checked and how (e.g., “Variance Inflation Factors were all below 2, indicating no problematic multicollinearity; residual plots showed no evidence of heteroscedasticity or non-linearity”). This does not need its own section in most journals, but its absence is a very common reviewer query.
- Overall model fit. For linear regression: R², adjusted R², and the omnibus F-test with its degrees of freedom. For logistic regression: a model chi-square (likelihood ratio) test, and a pseudo-R² (Nagelkerke or McFadden) reported as a supplementary, not a primary, fit statistic, since it is not directly comparable to linear R².
- Each coefficient, with unstandardized B (or the odds ratio, for logistic), standard error, a significance test (t or Wald), a p-value, and a confidence interval — the CI matters as much as the p-value, since it shows the plausible range of the effect, not just whether it cleared a threshold.
- A coefficient table rather than only prose, for any model with more than one or two predictors — this is close to universal convention and much easier for a reviewer to audit than values embedded in running text.
- Plain-language interpretation of the key coefficients using the association language discussed above, tied back to the research question.
Most quantitative journals in psychology, education, health, and the social sciences follow this same skeleton, whether or not they formally cite the APA Publication Manual’s reporting standards — the components above are the substance of what “APA-style regression reporting” means in practice, independent of citation-format specifics.
Frequently asked questions
What’s the difference between linear and logistic regression?
Linear regression predicts a continuous outcome and estimates it directly; logistic regression predicts the log-odds of a binary (or categorical) outcome and is typically reported back-transformed as an odds ratio or predicted probability. They use different estimation methods (ordinary least squares vs. maximum likelihood) and different fit statistics, and their assumption sets only partially overlap — see the sections above.
What counts as a “good” R²?
There is no universal threshold. Physical and engineering models often expect R² above 0.90; psychology, education, and social-science models routinely report R² in the 0.10–0.40 range as meaningful, because human behavior has many unmeasured sources of variance. Judge R² against comparable published work in your own field, not a fixed rule of thumb.
What does the p-value on a regression coefficient actually test?
It tests the null hypothesis that the true population coefficient for that predictor is zero (no relationship), holding the other predictors in the model constant. A small p-value means the observed coefficient would be unlikely under that null hypothesis; it does not measure effect size, practical importance, or the probability that the null hypothesis is true.
When should I use multiple regression instead of several simple regressions?
Whenever your predictors are plausibly correlated with each other and with the outcome, which is the normal case in most non-experimental research. Multiple regression estimates each predictor’s association with the outcome while statistically holding the others constant; running separate simple regressions on the same outcome instead will generally overstate each predictor’s individual contribution, because it never accounts for shared variance between predictors.
Can I use regression with a small sample?
Regression is possible with small samples but progressively underpowered and unstable as the ratio of observations to predictors drops. A commonly cited minimum for multiple linear regression is at least 10–20 observations per predictor; logistic regression additionally needs roughly 10 events of the rarer outcome per predictor (see the events-per-variable rule above). Below these thresholds, run a formal power analysis rather than proceeding on intuition — see the CASRAI guide to sample size and power for that step.
Do I need to worry about outliers?
Yes — a small number of high-leverage points can distort a regression far out of proportion to their number. Cook’s distance is the standard diagnostic for identifying influential points (values above roughly 4/n are commonly flagged for review); the correct response is to investigate why the point is unusual, not to delete it automatically.
Related CASRAI resources
- Research Methods pillar hub — the full quantitative-analysis, sampling, and study-design collection this guide belongs to.
- Cronbach’s alpha — the companion reliability check for a multi-item scale you plan to use as a predictor or outcome in a regression model.
- Confidence interval, explained — the interval-estimation concept behind every CI reported alongside a regression coefficient above.
- Null hypothesis, explained — what the p-value on each coefficient is actually testing against.
- Cohort study design and case-control study design — two common observational designs whose data regression is used to analyze.
