Written and maintained by CASRAI Editorial Board
Last updated
A parsimonious model is the simplest model that still explains the data adequately: it reaches an acceptable fit with as few estimated parameters as possible. The aim is not the smallest model, but the smallest one that does not leave real structure unexplained. This guide gives the definition, the formulas that put a number on parsimony (adjusted R², AIC and BIC), a worked example with the arithmetic shown, and code to reproduce it.
Definition: what is a parsimonious model?
A parsimonious model uses the fewest predictors, terms or parameters needed to describe the data well. Given two models that fit about equally well, the one with fewer parameters is the parsimonious choice. It is easier to interpret, needs less data to estimate, and is less likely to chase noise in the sample (overfitting).
Parsimony model and Occam’s razor
The phrase “parsimony model” usually refers to the principle behind the choice, often linked to Occam’s razor: do not add complexity that the evidence does not demand. Treat this as a heuristic, not a rule. A parsimonious model can still be wrong if it omits a term that matters, so parsimony is always weighed against fit.
Parsimonious models versus larger models
“Parsimonious models” is relative. A model is parsimonious compared with a named alternative fitted to the same data, for example a two-predictor regression against a five-predictor one. Nested models, where the smaller model is the larger one with some coefficients fixed at zero, are the cleanest case; see the likelihood-ratio test for nested models.
Formulas: putting a number on parsimony
Three criteria are in routine use. Each rewards fit and penalises the number of parameters.
- Adjusted R² = 1 − (1 − R²)(n − 1)/(n − p − 1), where R² is the proportion of variance explained, n is the number of observations and p is the number of predictors (not counting the intercept).
- AIC = −2 log L + 2k, where log L is the maximised log-likelihood and k is the number of estimated parameters. The R manual writes the general form as “-2 log-likelihood + k n_par” with k = 2 for AIC.
- BIC = −2 log L + k·ln(n), the same expression with the penalty weight k = log(n), where n is the number of observations. In R, BIC is defined as AIC(object, …, k = log(nobs(object))).
For a linear regression with normally distributed errors, −2 log L = n·ln(2π) + n·ln(SSE/n) + n, where SSE is the residual sum of squares. Smaller AIC or BIC is better, and the values are only comparable between models fitted to the same data and the same response. The R manual puts it this way: “When comparing models fitted by maximum likelihood to the same data, the smaller the AIC or BIC, the better the fit.” For the likelihood itself, see the likelihood function; for the two criteria head to head, see AIC vs BIC for model selection.
Software counts k differently. R counts the error variance as a parameter (k = p + 2 for a regression with an intercept), while the statsmodels OLS result uses −2·llf + 2·(df_model + 1), which is p + 1. The two AIC values for the same model therefore differ by a constant, which cancels when you compare models inside one program.
Worked example: a two-predictor model against a five-predictor model
Illustrative example: the figures below were constructed for this guide and do not come from a published study. Suppose 100 observations are used to predict one outcome. Model A uses 2 predictors and Model B uses 5, and Model A is nested in Model B. The total sum of squares is 1,000.
| Quantity | Model A (2 predictors) | Model B (5 predictors) |
|---|---|---|
| SSE | 420 | 380 |
| R² = 1 − SSE/1000 | 0.580 | 0.620 |
| Adjusted R² | 0.571 | 0.600 |
| Parameters k (p + intercept + variance) | 4 | 7 |
| −2 log L | 427.30 | 417.29 |
| AIC | 435.30 | 431.29 |
| BIC | 445.72 | 449.52 |
The arithmetic, step by step:
- n·ln(2π) = 100 × 1.8379 = 183.79 (the same for both models).
- Model A: n·ln(SSE/n) = 100 × ln(4.2) = 100 × 1.4351 = 143.51, so −2 log L = 183.79 + 143.51 + 100 = 427.30. AIC = 427.30 + 2 × 4 = 435.30. BIC = 427.30 + 4 × ln(100) = 427.30 + 4 × 4.6052 = 445.72.
- Model B: n·ln(SSE/n) = 100 × ln(3.8) = 100 × 1.3350 = 133.50, so −2 log L = 183.79 + 133.50 + 100 = 417.29. AIC = 417.29 + 2 × 7 = 431.29. BIC = 417.29 + 7 × 4.6052 = 449.52.
- Adjusted R²: Model A = 1 − 0.42 × 99/97 = 0.571; Model B = 1 − 0.38 × 99/94 = 0.600.
The three criteria do not agree. AIC and adjusted R² prefer the five-predictor model (AIC lower by 4.01). BIC prefers the two-predictor model (BIC lower by 3.80), because at n = 100 its penalty of ln(100) ≈ 4.6 per parameter is more than twice AIC’s penalty of 2. The three extra predictors cut the SSE by about 9.5%, which is worth 100 × ln(420/380) ≈ 10.0 units of −2 log L, more than AIC’s cost of 6 but less than BIC’s cost of 13.8. Which model is “parsimonious” therefore depends on the criterion, and a report should say which one was used.
Interpretation: how to read the criteria
| Criterion | What it rewards | Penalty for each added parameter | Reading |
|---|---|---|---|
| Adjusted R² | Variance explained, corrected for p | Built into (n − 1)/(n − p − 1) | Higher is better; it falls if a new predictor adds too little |
| AIC | Fit to the data | 2 | Lower is better; differences matter, not the absolute value |
| BIC | Fit to the data | ln(n) | Lower is better; penalises complexity more once n exceeds about 8 |
BIC’s penalty ln(n) exceeds AIC’s 2 whenever n is above e² ≈ 7.4, so for any realistic sample size BIC leans towards the smaller model. A high R² alone says little about parsimony: the NIST/SEMATECH e-Handbook notes that a high R² does not guarantee that the model fits the data well, and points to residual plots as the main validation tool. A guide to what R² can and cannot tell you is at how to interpret R-squared.
Assumptions and common mistakes
- Comparing across different data. AIC and BIC only compare models fitted to the same observations and the same response. Dropping a predictor that has missing values changes n and breaks the comparison.
- Treating “fewer terms” as the goal. A model that omits a needed term is not parsimonious, it is underfitted. Check the residuals before preferring the smaller model.
- Selecting by automatic search and then reporting p-values as if no search happened. This is the problem covered in why reviewers reject stepwise regression.
- Comparing raw AIC values across software. As noted above, programs count parameters differently; compare within one program.
- Ignoring collinearity. Redundant predictors inflate variance without improving fit; see variance inflation factor and multicollinearity.
- Using too many candidate terms for the sample. Candidate terms should be limited by the sample size, so that the number of estimated parameters stays small relative to n.
How to report a parsimonious model choice
State the sample size, the candidate models, the criterion, the values and the decision. An example in APA-style prose, using the illustrative figures above:
Two nested linear models were compared on the same 100 observations. The two-predictor model (AIC = 435.30, BIC = 445.72) and the five-predictor model (AIC = 431.29, BIC = 449.52) disagreed: AIC favoured the larger model, whereas BIC favoured the smaller one. We retained the two-predictor model as the more parsimonious option, and report the five-predictor results as a sensitivity analysis.
Say which software produced the values, because parameter counting differs, and report the criterion you chose before looking at the results where possible.
How to calculate it in Excel, R or Python
R
m1 = lm(y ~ x1 + x2, data = d)
m2 = lm(y ~ x1 + x2 + x3 + x4 + x5, data = d)
AIC(m1, m2) # smaller is better
BIC(m1, m2) # BIC = AIC with k = log(nobs)
summary(m1)$adj.r.squaredPython (statsmodels)
import statsmodels.api as sm
m1 = sm.OLS(y, sm.add_constant(X[:, :2])).fit()
m2 = sm.OLS(y, sm.add_constant(X)).fit()
print(m1.aic, m2.aic) # -2*llf + 2*(df_model + 1)
print(m1.bic, m2.bic)
print(m1.rsquared_adj, m2.rsquared_adj)Excel
Run each regression (Data Analysis, Regression, or LINEST) to get SSE, then compute the OLS equivalent of AIC and BIC. With n in B1, SSE in B2 and the number of fitted coefficients including the intercept in B3:
AIC (up to a constant): =B1*LN(B2/B1)+2*B3
BIC (up to a constant): =B1*LN(B2/B1)+B3*LN(B1)These drop the constant n·ln(2π) + n and count coefficients without the variance, so they differ from R’s values by a fixed amount for a given dataset. The difference between two models is identical (for the example, Model A minus Model B is 4.01 for AIC and −3.80 for BIC).
FAQ
What is a parsimony model?
A parsimony model is a model chosen on the principle of parsimony: among adequate explanations, prefer the one with the fewest parameters. In practice it means comparing models with a criterion such as adjusted R², AIC or BIC rather than choosing by fit alone.
Is a parsimonious model always the best model?
No. Parsimony is a preference between models that fit comparably well. If the smaller model leaves clear structure in the residuals, or omits a variable that the design requires, it underfits and the larger model is the better one.
Does AIC or BIC pick the more parsimonious model?
BIC usually does, because its penalty of ln(n) per parameter is larger than AIC’s 2 once n is above about 7. In the worked example, AIC chose the five-predictor model and BIC the two-predictor one.
Sources
- R Core Team. AIC, R stats package manual: stat.ethz.ch/R-manual/R-devel/library/stats/html/AIC.html (checked 2026-10-29).
- statsmodels documentation. RegressionResults.aic: statsmodels.org (checked 2026-10-29).
- NIST/SEMATECH e-Handbook of Statistical Methods, section 4.4.4, model validation: itl.nist.gov/div898/handbook/pmd/section4/pmd44.htm (checked 2026-10-29).
- Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716-723.
- Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461-464.
- Burnham, K. P., and Anderson, D. R. (2002). Model selection and multimodel inference (2nd ed.). Springer.
