Assumptions of Simple Regression: LINE Checks

Q: What is linear regression?

Linear regression models an outcome as a straight-line function of one or more predictors plus random error. With one predictor it is simple linear regression, fitted by least squares, which picks the line that minimises the sum of squared residuals.

Q: What is linear regression analysis?

Linear regression analysis is the full workflow: plot the data, fit the line, estimate the slope and intercept, test or put an interval on them, and check the residuals against the LINE assumptions above before reporting the result.

Q: Which regression equation best fits these data?

The best-fitting line is the one with the smallest residual sum of squares. For the worked example above that is ŷ = 0.1893 + 1.9607 x , with a residual sum of squares of 1.6039; any other slope and intercept gives a larger sum.

Q: What are the simple regression model assumptions in short?

Linearity, independent errors, constant error variance and approximately normal errors with mean 0, all checked on the residuals.

Q: What is the linearity assumption in regression?

It is the requirement that the straight-line part of the model adequately describes the systematic variation in the data. Check it with a scatter plot before fitting and a residuals-versus-fitted plot after; a curved pattern means the line has the wrong form.

The assumptions of simple regression are conditions on the random errors of the model y = β₀ + β₁x + ε: the relationship is linear, the errors are independent, they have constant variance, and they are approximately normally distributed with a mean of 0. You check them on the residuals, not on the raw x or y values, and you check them before you trust a slope, a p-value or a confidence interval.

Definition: what the assumptions of simple regression are

Simple linear regression fits one straight line to one predictor and one outcome. The line is the systematic part of the model; everything the line does not explain is the random part. The NIST/SEMATECH e-Handbook of Statistical Methods (section 5.2.4, “Are the model residuals well-behaved?”) states the requirement directly: residuals are expected to be “(roughly) normal and (approximately) independently distributed with a mean of 0 and some constant variance.” Four conditions follow, usually remembered as LINE:

Linearity: the straight-line (functional) part of the model describes the systematic variation in the data.
Independence: the random errors do not depend on one another, for example through time order.
Normality: the random errors are approximately normally distributed.
Equal variance: the scatter of the errors stays the same across the range of fitted values (also called constant variance or homoscedasticity).

Some textbooks add “no multicollinearity” to the list, but that condition only applies once there is more than one predictor, so it is not an assumption of simple regression.

Formula and symbols

y_i = β₀ + β₁x_i + ε_i

y_i: the observed outcome for case i.
x_i: the predictor value for case i.
β₀ and β₁: the population intercept and slope; the fitted estimates are b₀ and b₁.
ε_i: the random error for case i. The assumptions are statements about ε.
ŷ_i = b₀ + b₁x_i: the fitted value (see y-hat).
e_i = y_i − ŷ_i: the residual, the observable stand-in for ε_i (see how to calculate residuals).

Least squares estimates: b₁ = Σ(x − x̄)(y − ȳ) / Σ(x − x̄)² and b₀ = ȳ − b₁x̄.

Worked example with residuals

The data below are a small constructed dataset (n = 8) used only to show the arithmetic.

Step 1: means. x̄ = (1+2+…+8)/8 = 4.5. ȳ = (2.3+3.8+6.4+7.5+10.6+11.4+14.5+15.6)/8 = 72.1/8 = 9.0125.

Step 2: sums of squares. Σ(x − x̄)² = 42. Σ(x − x̄)(y − ȳ) = 82.35.

Step 3: coefficients. b₁ = 82.35 / 42 = 1.9607. b₀ = 9.0125 − 1.9607 × 4.5 = 0.1893. The fitted equation is ŷ = 0.1893 + 1.9607x.

x	y	ŷ = 0.1893 + 1.9607x	Residual e = y − ŷ	e²
1	2.3	2.1500	0.1500	0.0225
2	3.8	4.1107	−0.3107	0.0965
3	6.4	6.0714	0.3286	0.1080
4	7.5	8.0321	−0.5321	0.2832
5	10.6	9.9929	0.6071	0.3686
6	11.4	11.9536	−0.5536	0.3064
7	14.5	13.9143	0.5857	0.3431
8	15.6	15.8750	−0.2750	0.0756

Step 4: check the residuals. They sum to 0 (as least-squares residuals always do when the model has an intercept). The residual sum of squares is 1.6039, so the residual standard error is √(1.6039 / (8 − 2)) = 0.517 and R² = 1 − 1.6039 / 163.0688 = 0.990. The signs alternate with no run of same-signed residuals and no growth in size from left to right, so a plot of residuals against fitted values would show a random band about 0. With only eight points, this is a reading of the plot, not a formal test.

Interpretation: what each residual plot pattern means

Assumption	Plot	Pattern that supports it	Pattern that signals a problem
Linearity (and constant variance)	Residuals vs fitted values	Scattered randomly about 0	Systematic curve, or a funnel in which the residuals have “larger and larger scatter as the value of the response increases” (NIST 5.2.4)
Independence	Run sequence plot of residuals (data in collection order)	No trend or drift over time	Any time dependency (NIST)
Normality	Normal probability plot (Q-Q plot) of residuals	Points form an approximately straight line	Points depart systematically from a straight line

NIST section 4.4.4.2 describes a horn-shaped residual plot as “a typical plot indicating that the assumptions of the analysis are not satisfied with this model.” For the Q-Q plot read-through, see what is a Q-Q plot; for the funnel check, see how to read a residual plot for homoscedasticity.

Linearity assumption in simple regression

The linearity assumption (also searched as “linearity assumption regression”) says the straight line must adequately describe the systematic variation in the data. Plot y against x before fitting anything, as NIST advises, and then look at residuals against fitted values after fitting. A curved band of residuals means the line is the wrong shape. The full treatment is in the linearity assumption in linear regression; for the scatter plot itself, see the linear regression scatter plot guide.

Independence, constant variance and normality of residuals

Independence

Independence concerns how the data were collected. If observations were taken in a time order, plot the residuals in that order; NIST uses the run sequence plot to look for time dependency. A numerical check for serial correlation is covered in the Durbin-Watson test guide.

Constant variance

Constant variance means the vertical spread of the residuals is about the same at every fitted value. A funnel or horn shape in the residuals-versus-fitted plot is the warning sign.

Normality of the errors

Normality applies to the residuals, not to x or y. A skewed predictor does not by itself violate the assumption. Use the normal probability plot of the residuals.

Assumptions and common mistakes

Testing normality on the wrong variable. The model makes a statement about the errors, so the check belongs on the residuals.
Checking residuals from a badly shaped model. NIST section 4.4.4.2 notes that the functional part of the model must describe the systematic variation, and that “because the residuals are not random, they cannot be used to answer questions about the random part of the model.” If the residual plot shows a curve, fix the form first; the variance and normality checks are not reliable until the residuals look random.
Fitting before looking. Plot the data first, then fit, then plot the residuals.
Reading a plot of eight points as proof. Plots are judgement aids; say what you saw rather than claiming the assumptions are confirmed.
Ignoring data order. If the order of collection matters, the independence check needs the residuals in that order.

For what to do after the checks, see hypothesis testing in linear regression and the broader regression analysis guide.

How to report it (APA example)

State the model result, then one sentence on diagnostics describing what you examined. For the worked example above:

A simple linear regression predicted y from x, F(1, 6) = 604.01, p < .001, R² = .99, b₁ = 1.96. A plot of residuals against fitted values showed a random scatter about zero with no funnel or curvature, and a normal probability plot of the residuals was approximately linear.

Here F = (SSR / 1) / (SSE / 6) = (163.0688 − 1.6039) / (1.6039 / 6) = 161.4649 / 0.2673 = 604.01. Report the diagnostics you actually ran; if you did not collect data in a meaningful order, do not claim an independence check on time order.

How to calculate and check it in Excel, R or Python

Excel

With x in A2:A9 and y in B2:B9: put =SLOPE(B2:B9,A2:A9) in E1 and =INTERCEPT(B2:B9,A2:A9) in E2. In C2 enter =$E$2+$E$1*A2 (fitted) and in D2 enter =B2-C2 (residual), fill both down, then insert a scatter chart of D2:D9 (vertical) against C2:C9 (horizontal).

R

x <- 1:8
y <- c(2.3, 3.8, 6.4, 7.5, 10.6, 11.4, 14.5, 15.6)
fit <- lm(y ~ x)
plot(fitted(fit), resid(fit)); abline(h = 0)
qqnorm(resid(fit)); qqline(resid(fit))

Python

import numpy as np
import matplotlib.pyplot as plt
x = np.arange(1, 9)
y = np.array([2.3, 3.8, 6.4, 7.5, 10.6, 11.4, 14.5, 15.6])
b1, b0 = np.polyfit(x, y, 1)      # 1.9607, 0.1893
fitted = b0 + b1 * x
resid = y - fitted
plt.scatter(fitted, resid); plt.axhline(0); plt.show()

FAQ

What is linear regression?