Written and maintained by CASRAI Editorial Board
Last updated
The likelihood function measures how well each possible parameter value of a statistical model explains the data you actually observed. It is the starting point of maximum likelihood estimation, likelihood ratio tests and information criteria such as AIC. This guide defines it, shows the formula, works one example by hand with every step of arithmetic, and gives code you can paste into Excel, R or Python.
Definition: what is the likelihood function?
The likelihood function is the probability (or probability density) of the observed data, written as a function of the model’s unknown parameter while the data are held fixed. NIST’s e-Handbook of Statistical Methods puts it this way: maximum likelihood estimation “begins with the mathematical expression known as a likelihood function of the sample data”, the likelihood of a set of data is the probability of obtaining that particular set of data under the chosen model, and the parameter values that maximize it are the maximum likelihood estimates.
Likelihood vs probability
The same expression plays two roles. In a probability statement the parameter is fixed and the data vary: “if p = 0.5, how likely is 7 heads?” In a likelihood statement the data are fixed and the parameter varies: “given 7 heads, how well does each value of p explain them?” Because the likelihood is not a probability distribution over the parameter, its values need not add up to 1 across parameter values, as the worked example below shows.
Formula and what every symbol means
For independent observations x1, x2, …, xn with density or mass function f(x | θ):
L(θ) = f(x1 | θ) × f(x2 | θ) × … × f(xn | θ)
The log-likelihood is the natural logarithm of this product, which turns the product into a sum:
ℓ(θ) = ln L(θ) = ln f(x1 | θ) + … + ln f(xn | θ)
- θ (theta): the unknown parameter, or a vector of parameters, such as a probability p or a failure rate λ.
- x1, …, xn: the observed data values. They are constants once collected.
- f(x | θ): the model’s probability mass or density function for one observation.
- L(θ): the likelihood. Any constant factor that does not depend on θ can be dropped, because it does not change which θ is largest.
- ℓ(θ): the log-likelihood. It has its maximum at the same θ as L(θ), and it is far easier to differentiate and to compute without numerical underflow.
- θ̂ (theta-hat): the maximum likelihood estimate, the value of θ at which L(θ) (equivalently ℓ(θ)) is largest.
Worked example: 7 heads in 10 tosses
A coin is tossed 10 times and lands heads 7 times and tails 3 times, in one particular order. Let p be the probability of heads. Each head contributes a factor p and each tail a factor (1 − p), so for this data set:
L(p) = p7 × (1 − p)3
(If you count only the number of heads, the binomial formula multiplies this by C(10, 7) = 120, a constant that does not depend on p and so does not change where the maximum is.) Evaluating L(p) at five values of p:
| p | Arithmetic | L(p) |
|---|---|---|
| 0.1 | 0.17 × 0.93 = 0.0000001 × 0.729 | 0.0000000729 |
| 0.3 | 0.37 × 0.73 = 0.0002187 × 0.343 | 0.0000750 |
| 0.5 | 0.57 × 0.53 = 0.510 = 1/1024 | 0.000977 |
| 0.7 | 0.77 × 0.33 = 0.0823543 × 0.027 | 0.002224 |
| 0.9 | 0.97 × 0.13 = 0.4782969 × 0.001 | 0.000478 |
Two things stand out. First, the five values sum to about 0.00375, not 1, so L(p) is not a probability distribution over p. Second, the data are best explained near p = 0.7. Setting the derivative of the log-likelihood to zero confirms it:
ℓ(p) = 7 ln p + 3 ln(1 − p), so ℓ′(p) = 7/p − 3/(1 − p) = 0. Solving gives 7(1 − p) = 3p, so 7 = 10p and p̂ = 0.7, the observed proportion of heads.
At the maximum, ℓ(0.7) = 7 ln 0.7 + 3 ln 0.3 = 7(−0.35667) + 3(−1.20397) = −6.1086. At p = 0.5, ℓ(0.5) = 10 ln 0.5 = −6.9315. The ratio L(0.7)/L(0.5) = 0.0022236 / 0.0009766 = 2.28, so p = 0.7 explains these data about 2.3 times as well as a fair coin does.
Interpretation: reading the numbers
| What you have | What it tells you | What it does not tell you |
|---|---|---|
| L(θ) at one value of θ | The probability (or density) of the observed data if that θ is true. | The probability that θ is true. A small value is normal for any large data set. |
| L(θ1) / L(θ2) for the same data | How many times better θ1 explains the data than θ2 (2.28 for p = 0.7 against p = 0.5 above). | Anything if the two likelihoods come from different data sets or different outcome definitions. |
| The maximizing value θ̂ | The parameter value that makes the observed data most probable under the model (0.7 above). | That the model is correct, or that θ̂ is close to the true value in a small sample. |
| Log-likelihood ℓ(θ̂) | A negative number on a scale where higher (closer to zero) means a better fit to the same data (−6.11 above). | An absolute measure of fit. Compare it only across models fitted to identical data. |
When some observations are censored
Lifetime studies often stop before every unit has failed. NIST’s reliability chapter shows that the likelihood then contains one term per unit: the density f(ti) for a unit that failed at time ti, and the survivor probability 1 − F(T) for a unit still working when observation stopped at time T. With Type I censoring after r failures out of n units, L = C × [f(t1) × … × f(tr)] × [1 − F(T)]n−r, where C is a constant that does not affect the estimate. For an exponential lifetime model with failure rate λ, NIST gives the estimate λ̂ = r / (Σti + (n − r)T).
Illustration with invented numbers: 6 units are tested for T = 1,000 hours and 4 fail, at 120, 340, 560 and 780 hours. Then Σti = 120 + 340 + 560 + 780 = 1,800 and (n − r)T = 2 × 1,000 = 2,000, so λ̂ = 4 / 3,800 = 0.001053 failures per hour, an estimated mean life of 1/λ̂ = 950 hours. Dropping the two survivors and using only the failures would give 4/1,800 = 0.00222 per hour (450 hours), far too pessimistic. The censored terms in the likelihood are what correct for this.
Assumptions and common mistakes
- Independence is built into the product. Multiplying the individual terms assumes the observations are independent given θ. With clustered or repeated measures, the likelihood needs a model for the dependence.
- The model family must be specified. A likelihood exists only relative to a chosen distribution, such as binomial, Poisson, normal or exponential.
- Reading L(θ) as P(θ). The likelihood is not the probability that the parameter equals θ and does not integrate to 1 over θ.
- Comparing likelihoods across different data. Likelihood values are comparable only for the same observations. Dropping or adding a data point changes every value.
- Dropping a constant, then comparing absolute values. Dropping C is harmless for finding θ̂, but two software packages that drop different constants will report different log-likelihoods for the same model.
- Multiplying many small numbers directly. Products of hundreds of probabilities underflow to zero in floating-point arithmetic. Work with the log-likelihood.
How to report it (APA example)
Report the model, the estimate and the log-likelihood, and say which software and estimation method you used. Example wording, using the coin data above:
A binomial model was fitted by maximum likelihood. The estimated probability of heads was p̂ = .70, with a log-likelihood of −6.11 at the maximum (n = 10 tosses).
When you compare models, report each model’s log-likelihood on the same data together with the test or criterion you used, for example a likelihood ratio test or AIC (see likelihood ratio tests for nested models and AIC vs BIC for model selection).
How to calculate it in Excel, R or Python
Excel
Put candidate values of p in A2:A6 (0.1, 0.3, 0.5, 0.7, 0.9). Then:
B2: =A2^7*(1-A2)^3 (likelihood, copy down)
C2: =7*LN(A2)+3*LN(1-A2) (log-likelihood, copy down)The largest value in column B (or the largest, least negative value in column C) is at p = 0.7.
R
p = c(0.1, 0.3, 0.5, 0.7, 0.9)
L = p^7 * (1 - p)^3 # likelihood
ll = 7 * log(p) + 3 * log(1 - p) # log-likelihood
opt = optimize(function(p) 7*log(p) + 3*log(1-p),
interval = c(0.001, 0.999), maximum = TRUE)
opt$maximum # about 0.7
# dbinom(7, 10, p) equals L multiplied by choose(10, 7) = 120Python
import numpy as np
from scipy.optimize import minimize_scalar
p = np.array([0.1, 0.3, 0.5, 0.7, 0.9])
L = p**7 * (1 - p)**3 # likelihood
ll = 7*np.log(p) + 3*np.log(1 - p) # log-likelihood
res = minimize_scalar(lambda q: -(7*np.log(q) + 3*np.log(1 - q)),
bounds=(0.001, 0.999), method="bounded")
print(res.x) # about 0.7For a worked closed-form case with a different distribution, see the maximum likelihood estimator of the Poisson distribution. The likelihood also underlies generalized linear models and full information maximum likelihood (FIML) for missing data. For the binomial model used above, see Bernoulli distribution vs binomial distribution.
FAQ
What is the likelihood function?
It is the probability (or density) of the observed data viewed as a function of the model’s parameter, with the data held fixed. The parameter value that maximizes it is the maximum likelihood estimate.
Is the likelihood the same as a probability?
It uses the same formula but answers a different question. Probability varies the data for a fixed parameter; likelihood varies the parameter for fixed data. Likelihood values do not sum or integrate to 1 over the parameter.
Why use the log-likelihood?
The logarithm turns a product of many terms into a sum, which is easier to differentiate and avoids numerical underflow, and it peaks at the same parameter value as the likelihood itself.
Can a likelihood be greater than 1?
For continuous data the likelihood is built from densities, which can exceed 1, so L(θ) can too. For discrete data it is a probability and cannot exceed 1.
Sources
- NIST/SEMATECH e-Handbook of Statistical Methods, section 1.3.6.5.2, Maximum Likelihood: https://www.itl.nist.gov/div898/handbook/eda/section3/eda3652.htm (date checked: 2026-10-29).
- NIST/SEMATECH e-Handbook of Statistical Methods, Assessing Product Reliability, likelihood equations for censored data: https://www.itl.nist.gov/div898/handbook/apr/section4/apr412.htm (date checked: 2026-10-29).
The coin and lifetime numbers above are illustrative values computed for this guide, not data from either source.
