Principal Component Analysis Regression (PCR): Definition, Formula and Worked Example

Principal component analysis regression, usually called principal component regression (PCR) or PCA regression, is a two-step method. First, principal component analysis (PCA) is applied to the predictor variables and replaces them with a smaller set of uncorrelated components. Second, the outcome is regressed on those components instead of on the original predictors. The scikit-learn documentation describes it the same way: PCR is “a regressor composed of two steps: first, PCA is applied to the training data, possibly performing dimensionality reduction; then, a regressor (e.g. a linear regressor) is trained on the transformed samples.”

Researchers reach for PCR when predictors are strongly correlated (see variance inflation factor and multicollinearity) or when there are many predictors relative to observations (see high-dimensional data). This guide gives the definition, every symbol in the formula, a small worked example with the arithmetic shown, the main failure mode with published numbers, how to choose the number of components, and how to run and report the method in Excel, R and Python.

Definition: what is principal component regression?

PCR is ordinary least squares regression run on principal component scores. A principal component is a linear combination of the original predictors; the components are constructed so that they are mutually uncorrelated and ordered by how much predictor variance each one carries. Keeping only the first few components discards the directions along which the predictors vary least, which is what removes the collinearity problem and reduces the number of coefficients that must be estimated.

Two points separate PCR from an ordinary multiple regression. The components are built from the predictors alone, with no use of the outcome, and the final model is fitted on fewer variables than the data originally contained. The coefficients can still be translated back to the original predictors (shown below), so the result is interpretable on the original scale.

PCA regression, principal component regression and PCR: are they the same thing?

Yes. “PCA regression”, “principal component regression”, “principal components regression” and “PCR” all name the same procedure: PCA on the predictors followed by regression on the retained component scores. The Journal of Statistical Software article on the R pls package (Mevik and Wehrens, 2007) uses “principal component regression (PCR)” and treats it alongside partial least squares regression. Note that “PCR” is also the abbreviation for polymerase chain reaction in laboratory science; in statistics it means this method.

Formula with every symbol explained

Let X be an n × p matrix of predictors (n observations, p variables) and y the outcome vector.

1. Standardize. z_ij = (x_ij − x̄_j) / s_j, where x̄_j is the mean and s_j the sample standard deviation of predictor j. Standardizing puts every predictor on the same scale, and the scikit-learn PCR example standardizes “as recommended by good practice”. The scikit-learn PCA class on its own centers the data but does not scale it, so scaling must be added explicitly.
2. Decompose the correlation matrix. R = ZᵀZ / (n − 1) is the p × p correlation matrix of the standardized predictors. Solve Rv_j = λ_jv_j. Each λ_j (eigenvalue) is the variance of component j, and each v_j (eigenvector, the “loadings”) gives the weights of the predictors in that component. For a correlation matrix the eigenvalues sum to p, the number of variables, as in the NIST/SEMATECH e-Handbook worked PCA example.
3. Compute component scores. T = Z V_k, where V_k holds the first k eigenvectors as columns. T is n × k.
4. Regress. ŷ = ȳ + T γ̂, where γ̂ is the vector of k least-squares coefficients. Because the columns of T are uncorrelated, each γ̂_j is computed independently: γ̂_j = Σ t_ij(y_i − ȳ) / Σ t_ij².
5. Back-transform (optional). The coefficients on the standardized predictors are β̂_z = V_k γ̂, and on the original units β̂_j = β̂_z,j / s_j, with intercept β̂₀ = ȳ − Σ β̂_j x̄_j.

When k = p, no information is discarded and PCR reproduces the ordinary least squares fit exactly. The method only differs from OLS when k below p.

Worked example with real numbers

This illustrative dataset is invented for teaching and is not from any study. Eight observations of two correlated predictors and an outcome:

Obs.	x1	x2	y
1	2	3	5
2	4	3	8
3	5	6	9
4	7	6	13
5	8	9	14
6	10	8	18
7	11	12	19
8	13	11	24

Step 1, standardize. x̄₁ = 7.5, s₁ = 3.7417; x̄₂ = 7.25, s₂ = 3.3700; ȳ = 13.75, s_y = 6.3640. Observation 1 becomes z₁ = (2 − 7.5)/3.7417 = −1.470 and z₂ = (3 − 7.25)/3.3700 = −1.261.

Step 2, decompose. The correlation between x₁ and x₂ is r = 0.9290. For any 2 × 2 correlation matrix with correlation r, the eigenvalues are 1 + r and 1 − r, so λ₁ = 1.9290 and λ₂ = 0.0710. They sum to 2, the number of predictors. The first component carries 1.9290 / 2 = 96.45% of the predictor variance. For r above 0 the eigenvectors are (1, 1)/√2 and (1, −1)/√2, so PC1 = (z₁ + z₂)/√2 and PC2 = (z₁ − z₂)/√2.

Step 3, scores. Observation 1: PC1 = (−1.470 − 1.261)/1.4142 = −1.931. Across the eight observations PC1 is −1.931, −1.553, −0.735, −0.357, 0.462, 0.630, 1.658, 1.826.

Step 4, regress y on PC1 only (k = 1). Σ PC1·(y − ȳ) = 59.802 and Σ PC1² = 13.503 (that is, 7 × 1.9290), so γ̂₁ = 59.802 / 13.503 = 4.4288. The fitted model is ŷ = 13.75 + 4.4288 × PC1, with R² = 0.934.

Step 5, back-transform. β̂_z = 4.4288 / √2 = 3.1316 for each standardized predictor. In original units: β̂₁ = 3.1316 / 3.7417 = 0.8370 and β̂₂ = 3.1316 / 3.3700 = 0.9293, with intercept 13.75 − 0.8370 × 7.5 − 0.9293 × 7.25 = 0.736. The one-component PCR model is therefore ŷ = 0.736 + 0.837 x₁ + 0.929 x₂.

Check against keeping both components. Adding PC2 gives γ̂₂ = 5.916 and R² = 0.9956, which equals the ordinary least squares R² because k = p. The first component alone captured 96.45% of the predictor variance and 93.4% of the outcome variance; the small second component still matters a little for y, which leads into the next section.

Interpretation: how to read the output

Output	What it tells you
Eigenvalue λj	Variance of component j. With standardized predictors the eigenvalues sum to p (NIST/SEMATECH e-Handbook, section 5.5.2).
Explained variance ratio	Share of total predictor variance carried by each component; it sums to 1.0 when every component is kept (scikit-learn explained_variance_ratio_).
Loadings (eigenvector entries)	Weight of each standardized predictor in a component. In the example both weights are 1/√2 for PC1, so PC1 is an overall “size” index of x1 and x2, and PC2 is their contrast.
Component coefficient γ̂j	Change in y per one-unit change in component j. Components are uncorrelated, so adding or dropping one does not change the others.
Test R² below zero	The model predicts worse than always predicting the mean of the outcome. The scikit-learn example reports R² = −0.026 for a one-component PCR (below).

Components with large variance are not automatically the ones that predict the outcome. That is the central limitation of PCR, covered next.

Assumptions and common mistakes

PCR ignores the outcome when building components

The scikit-learn documentation states that PCA is unsupervised: “no information about the targets is used. As a result, PCR may perform poorly in some datasets where the target is strongly correlated with directions that have low variance.” Its example generates 500 samples from a two-dimensional Gaussian with covariance [[3, 3], [3, 4]], with the target driven by the second, low-variance component. The data table below reports the held-out results. Keeping one component in PCR fails (R² = −0.026), while the supervised alternative, partial least squares (PLS), reaches 0.658 with one component, and PCR needs both components to reach 0.673.

Other mistakes to avoid

• Forgetting to scale. The scikit-learn PCA class centers but does not scale the data, so predictors measured in large units dominate the first component unless you standardize first.
• Choosing the number of components on the full dataset. If k is picked using all observations and the model is then evaluated on the same observations, the performance estimate is optimistic. Fit the scaler, PCA and regression on training data only, and choose k with cross-validation.
• Using whitened scores without noticing. In scikit-learn, whiten=True “will remove some information from the transformed signal (the relative variance scales of the components)”. It is rarely needed for PCR.
• Reading components as causes. A component is a weighted blend of predictors. Interpret it through its loadings, not as a measured variable.
• Assuming a linear relationship. The final step is linear regression, so the usual linear-model checks (residual plots, influential points) still apply.

How many components should you keep?

Two families of rules exist. The first looks only at the predictors: keep enough components to reach a target share of variance, or use an automatic choice. In scikit-learn, setting n_components to a float between 0 and 1 with svd_solver=’full’ keeps the smallest number of components whose explained variance exceeds that fraction, and n_components=’mle’ uses Minka’s maximum likelihood estimate. The second looks at prediction: treat k as a tuning parameter and select it by cross-validation, which is what the R pls package supports through its cross-validation option. Because PCA never sees the outcome, a variance-only rule can discard the component that predicts y, as the example above shows, so a predictive criterion is the safer default when the goal is prediction.

PCR vs PLS vs OLS: which should you use?

Use OLS when predictors are few and not badly collinear. Use PCR when the aim is to remove collinearity or reduce dimension using the predictor structure alone. Consider PLS when the outcome should guide the construction of the components; the scikit-learn example shows the PLS transformation is supervised and succeeds with one component where one-component PCR fails. For the full comparison see the CASRAI guide to partial least squares regression.

How to report principal component regression (APA style)

Report enough detail for a reader to repeat the analysis: how predictors were scaled, how many components were kept and why, the variance they explain, the model fit, and the validation scheme. A template, with statistical symbols in italics:

Predictors were standardized and reduced with principal component analysis. The first component (λ = 1.93) explained 96.5% of the predictor variance and was retained. A regression of the outcome on this component showed R² = .93 (N = 8; illustrative data). The number of components was selected by cross-validation within the training data, and all preprocessing was fitted on the training data only.

State the software and package versions, and give the loadings in a table so readers can interpret the components. If you report coefficients on the original predictors, state that they were back-transformed from the component model.

How to calculate PCR in Excel, R or Python

Excel

Excel has no built-in eigen-decomposition, so the full method needs an add-in. For two predictors the closed form above works with ordinary functions. Standardize each predictor column into columns F and G with =STANDARDIZE(B2, AVERAGE(B$2:B$9), STDEV.S(B$2:B$9)), compute PC1 as =(F2+G2)/SQRT(2) when the correlation of the two predictors is positive, and regress with =SLOPE(D$2:D$9, H$2:H$9) and =INTERCEPT(D$2:D$9, H$2:H$9). The slope returns 4.4288 for the example data.

R

library(pls)
d = data.frame(x1 = c(2,4,5,7,8,10,11,13),
                x2 = c(3,3,6,6,9,8,12,11),
                y  = c(5,8,9,13,14,18,19,24))
fit = pcr(y ~ x1 + x2, data = d, ncomp = 1, scale = TRUE, validation = "CV")
summary(fit)

The pls package implements both PCR and PLS regression with a formula interface (Mevik and Wehrens, 2007).

Python

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.linear_model import LinearRegression

pcr = make_pipeline(StandardScaler(), PCA(n_components=1), LinearRegression())
pcr.fit(X_train, y_train)
print(pcr.score(X_test, y_test))  # test R-squared

This is the pipeline used in the scikit-learn PCR vs PLS example. For SPSS or Stata routes to the PCA step, see PCA in SPSS and PCA in Stata.

Frequently asked questions

What is principal component analysis regression?

It is regression of an outcome on the principal component scores of the predictors, rather than on the predictors themselves. PCA supplies uncorrelated components; ordinary regression is then fitted on the first k of them.

Does PCR fix multicollinearity?

Components are uncorrelated by construction, so a regression on them has no collinearity among its inputs. The cost is that the model is fitted on fewer dimensions than the original predictors.

Should the data be standardized before PCR?

Usually yes. Scaling stops variables with large units from dominating the first component. The scikit-learn example standardizes before PCA, and its PCA class centers but does not scale by itself.

Can PCR perform worse than OLS or PLS?

Yes. When the outcome depends on low-variance directions of the predictors, PCR with too few components can fail; in the published example one-component PCR scored R² = −0.026.

Is PCR the same as PCA?

No. PCA only transforms the predictors. PCR adds a regression step on the transformed values.

Sources

• scikit-learn documentation, “Principal Component Regression vs Partial Least Squares Regression” (read 2026-11-12): scikit-learn.org.
• scikit-learn documentation, PCA class reference (read 2026-11-12): scikit-learn.org.
• Mevik, B.-H., and Wehrens, R. (2007). The pls package: Principal component and partial least squares regression in R. Journal of Statistical Software, 18(2). jstatsoft.org (read 2026-11-12).
• NIST/SEMATECH e-Handbook of Statistical Methods, section 5.5.2, principal components (read 2026-11-12): itl.nist.gov.