Linear Discriminant Analysis (LDA): Formula, Example and Code

Short answer: linear discriminant analysis (LDA) is a classification method that assigns each observation to the class whose multivariate normal model, with a covariance matrix shared by all classes, gives it the highest posterior probability. Because the shared covariance makes the class boundaries straight lines (hyperplanes), it is called linear. This guide gives the formula with every symbol defined, a two-class example worked by hand with the arithmetic shown, a comparison with quadratic discriminant analysis (QDA), the assumptions, an APA-style reporting template and short Python, R and Excel recipes.

What is discriminant analysis, and what is LDA used for in machine learning?

Discriminant analysis is a family of supervised methods that use measured predictors to separate known groups and to classify new cases into those groups. LDA and QDA are, in the words of the scikit-learn documentation, two classic classifiers with a linear and a quadratic decision surface respectively. They model the predictors within each class as multivariate Gaussian and pick the class with the largest posterior probability. They have closed-form solutions, handle more than two classes natively and have no hyperparameters to tune in their basic form.

LDA is used in two ways. As a classifier, it predicts class membership (for example, which of several diagnostic groups a participant belongs to, from a battery of test scores). As a supervised dimensionality-reduction step, it projects the data onto at most K – 1 directions (K = number of classes, and never more than the number of predictors) that best separate the classes. Unlike unsupervised methods, it uses the class labels. If you want to test whether group means differ on several outcomes rather than predict group membership, the related inferential tool is covered in MANOVA in SPSS; if your aim is a predictive latent-variable model, see partial least squares regression.

What is the formula for linear discriminant analysis?

For an observation vector x with p predictors, class k has a mean vector μ_k and a prior probability π_k. Under the LDA assumption every class shares one covariance matrix Σ. The (linear) discriminant score for class k is:

δk(x) = x′ Σ-1 μk - ½ μk′ Σ-1 μk + ln πk

• x: the p-vector of predictor values for the case being classified.
• μ_k: the mean vector of class k, estimated from the training data.
• Σ: the common within-class covariance matrix, estimated as the pooled covariance matrix: S_p = (Σ_k (n_k – 1) S_k) / (n – K), where S_k is the sample covariance of class k, n_k its size, n the total sample size.
• π_k: the prior probability of class k. In scikit-learn the default is to infer priors from the class proportions in the training data.

The observation is assigned to the class with the largest δ_k(x). Because δ_k is linear in x, the boundary between two classes is the set of points where their scores are equal, which is a hyperplane. For two classes with equal priors this reduces to a single direction w = S_p^-1 (μ₁ – μ₂) (Fisher’s linear discriminant): a case is assigned to class 1 when w′(x – (μ₁ + μ₂)/2) is positive.

Worked example: two classes, two predictors, by hand

This is a deliberately tiny, made-up data set (six cases) chosen so the arithmetic is exact. It is the same data set used in the scikit-learn documentation example for LinearDiscriminantAnalysis, so you can check the result in software.

Class	Case (x1, x2)
1	(-1, -1), (-2, -1), (-3, -2)
2	(1, 1), (2, 1), (3, 2)

1. Class means. μ₁ = (-2, -4/3); μ₂ = (2, 4/3).
2. Within-class sums of squares and cross-products. For class 1: Σ(x₁ – mean)² = 1 + 0 + 1 = 2; Σ(x₂ – mean)² = 1/9 + 1/9 + 4/9 = 2/3; cross-product = (1)(1/3) + (0)(1/3) + (-1)(-2/3) = 1. Class 2 gives the identical matrix [[2, 1], [1, 2/3]].
3. Pooled covariance. Add the two matrices to get [[4, 2], [2, 4/3]] and divide by n – K = 6 – 2 = 4: S_p = [[1, 1/2], [1/2, 1/3]].
4. Invert. det S_p = (1)(1/3) – (1/2)² = 1/12, so S_p^-1 = 12 × [[1/3, -1/2], [-1/2, 1]] = [[4, -6], [-6, 12]].
5. Direction. μ₁ – μ₂ = (-4, -8/3), so w = S_p^-1(μ₁ – μ₂) = (4(-4) – 6(-8/3), -6(-4) + 12(-8/3)) = (0, -8).
6. Midpoint. (μ₁ + μ₂)/2 = (0, 0), so the decision rule with equal priors is: class 1 if w′x > 0, i.e. if -8·x₂ > 0, i.e. if x₂ < 0.
7. Classify a new case x = (-0.8, -1): score = -8 × (-1) = 8 > 0, so the case is assigned to class 1.

Cross-check: fitting LinearDiscriminantAnalysis() (default svd solver) in scikit-learn to this data returns a coefficient vector of magnitude (0, 8) and a zero intercept, and predicts class 1 for [-0.8, -1], matching the documented output. The sign of coef_ differs from w because, for two classes, scikit-learn reports the score for the second class relative to the first. Running the lsqr solver on the same data gives (0, 12): the vector is rescaled but the classification is the same. Here the first predictor receives a weight of zero because x₁ and x₂ are correlated within classes, and once that correlation is accounted for x₁ adds no separation beyond x₂.

Interpretation: reading the output

Output	What it tells you
Discriminant direction (w, coef_)	The weighted combination of predictors that best separates classes, given the pooled covariance. Compare raw weights only when predictors are on comparable scales.
Class means projected on w	How far apart the classes lie along the discriminant axis. In the example the projected centroids are +32/3 and -32/3.
Posterior probability	The model’s probability that a case belongs to each class, given the Gaussian and equal-covariance assumptions.
Number of discriminant axes	At most min(K – 1, p). A two-class problem yields one axis.
Classification accuracy	Judge on held-out data or cross-validation, not on the training data that fitted the model.

LDA vs QDA: which is more flexible?

QDA is more flexible. QDA drops the equal-covariance assumption and estimates a separate covariance matrix for each class, which produces quadratic decision surfaces. LDA is the special case of QDA in which every class has the same covariance matrix. The extra flexibility has a cost: QDA estimates more parameters, so it needs more data per class.

	LDA	QDA
Class covariance	One shared matrix Σ	A matrix Σk per class
Decision surface	Linear	Quadratic
Parameters for covariance	One matrix	K matrices
Special case	Special case of QDA	With diagonal covariances it equals Gaussian naive Bayes

Assumptions and common mistakes

• Multivariate normal predictors within each class. See the multivariate normal distribution. This is the generative model behind the posterior.
• Equal covariance matrices across classes. If classes clearly differ in spread or correlation, QDA is the natural alternative. Box’s M test is commonly used to examine this, but it is sensitive to non-normality, so treat it as a screen rather than a verdict.
• Independent observations. Repeated measures on the same units violate the model.
• More cases than predictors. When samples are few relative to features, the covariance estimate is unreliable. scikit-learn offers shrinkage (shrinkage='auto' uses the Ledoit-Wolf estimate), which works only with the lsqr or eigen solvers, not the default svd.
• Common mistakes: reporting accuracy on the training sample only; expecting more than K – 1 discriminant axes; comparing weights across predictors measured in different units; ignoring unequal class sizes, which the priors setting controls.

How to report LDA (APA-style template)

Report the predictors, the classes and their sizes, the priors, the assumption checks, and out-of-sample performance. A template (replace bracketed values with your own results; the numbers are placeholders, not findings):

A linear discriminant analysis with [predictors] as predictors classified participants into [K] groups (n = [n₁], [n₂]). Priors were set to the observed group proportions. Box’s M indicated [equal / unequal] covariance matrices, M = [value], p = [value]. Under [10-fold] cross-validation the model correctly classified [percentage] of cases (chance level = [percentage]).

How to calculate LDA in Python, R or Excel

Python (scikit-learn)

import numpy as np
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
X = np.array([[-1,-1],[-2,-1],[-3,-2],[1,1],[2,1],[3,2]])
y = [1,1,1,2,2,2]
clf = LDA().fit(X, y)
print(clf.coef_, clf.intercept_)
print(clf.predict([[-0.8, -1]]))   # [1]

R

library(MASS)
df <- data.frame(x1=c(-1,-2,-3,1,2,3), x2=c(-1,-1,-2,1,1,2), g=factor(c(1,1,1,2,2,2)))
fit <- lda(g ~ x1 + x2, data = df)
predict(fit, data.frame(x1=-0.8, x2=-1))$class

Note that lda() scales its discriminant coefficients differently from the hand formula; the predicted classes agree.

Excel

Excel has no built-in discriminant analysis command, but the two-class case follows the worked example directly: compute each class mean with AVERAGE, build the pooled covariance matrix cell by cell with COVARIANCE.S weighted by (n_k – 1) and divided by (n – K), then obtain w with =MMULT(MINVERSE(Sp), mean1 - mean2) entered as an array formula. Classify a case by the sign of w′(x – midpoint).

FAQ

Is LDA or QDA more flexible?

QDA. It fits a covariance matrix per class and so yields quadratic boundaries; LDA is the special case with one shared covariance matrix.

What is linear discriminant analysis (LDA) and how is it used in machine learning?

A supervised classifier and dimensionality-reduction method that models each class as a Gaussian with a shared covariance matrix, classifies by the highest posterior probability, and can project data onto at most K – 1 axes.

What is meant by discriminant analysis?

Statistical methods that use predictor variables to describe differences between known groups and to assign new observations to a group.

Can LDA handle more than two classes?

Yes. It is inherently multiclass; the number of discriminant axes is min(K – 1, p).

Sources

• scikit-learn documentation, “Linear and Quadratic Discriminant Analysis”, scikit-learn.org/stable/modules/lda_qda.html (read 2026-11-08).
• scikit-learn documentation, LinearDiscriminantAnalysis API reference, scikit-learn.org (API page) (read 2026-11-08).
• Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer. Free PDF at hastie.su.domains/ElemStatLearn. Textbook treatment of LDA and QDA.

Part of the Research Methods hub.