Written and maintained by CASRAI Editorial Board
Published
On this page
Bayesian regression is regression in which the unknown coefficients and the error variance are treated as random quantities with probability distributions. You state a prior distribution for them, combine it with the likelihood of the observed data through Bayes’ theorem, and get a posterior distribution that you summarise with a mean, a credible interval and predictions. This page is for researchers and analysts who need to understand it, run it and report it. It includes a worked example computed by hand and checked in code.
What is Bayesian regression? (Definition in one paragraph)
In ordinary least squares (OLS) the coefficients are fixed unknown numbers estimated by a single best-fit value. In Bayesian linear regression they are given a prior distribution, and the result is a full posterior distribution for each coefficient and for the residual standard deviation. With a flat (non-informative) prior and a Gaussian error model, the posterior mode equals the OLS estimate; an informative prior pulls the estimate toward the prior mean, more strongly when the data are few or noisy. The same idea extends to logistic, Poisson and multilevel models, usually fitted by Markov chain Monte Carlo sampling when no closed form exists. For choosing priors, see prior distributions; for multilevel extensions, see Bayesian hierarchical models.
Bayesian linear regression formula
The model is y = Xβ + ε, with ε ~ N(0, σ²I). Here y is the N×1 vector of outcomes, X is the N×K matrix of predictors (including a column of ones if there is an intercept), β is the K×1 vector of coefficients, and σ² is the error variance.
With σ² known and a normal prior β ~ N(β₀, σ²V₀), the posterior is also normal (Statlect, “Bayesian regression”):
- Posterior covariance: V_N = (V₀⁻¹ + X′X)⁻¹ (scaled by σ²)
- Posterior mean: β_N = V_N (V₀⁻¹β₀ + X′y)
V₀⁻¹ is the prior precision and X′X is the information in the data, so the posterior mean is a precision-weighted average of the prior mean and the OLS estimate. When σ² is also unknown, the conjugate choice is a normal-inverse-gamma prior, and the marginal posterior of the coefficients is a multivariate Student t distribution.
Link to ridge regression. With prior β ~ N(0, τ²I), the posterior mode minimises ‖y − Xβ‖² + (σ²/τ²)‖β‖². This is ridge regression with penalty λ = σ²/τ², so a narrower prior means stronger shrinkage. This follows directly from taking the log of the posterior.
Worked example with the arithmetic shown
Take one predictor, no intercept, and five observations: x = 1, 2, 3, 4, 5 and y = 2.1, 3.9, 6.2, 7.8, 10.1. Assume the error standard deviation is known, σ = 1, and choose the prior β ~ N(0, 1²), that is prior mean 0 and variance τ² = 1.
- Σx² = 1 + 4 + 9 + 16 + 25 = 55.
- Σxy = 2.1 + 7.8 + 18.6 + 31.2 + 50.5 = 110.2.
- OLS estimate: 110.2 / 55 = 2.0036.
- Posterior precision: 1/τ² + Σx²/σ² = 1 + 55 = 56, so the posterior variance is 1/56 = 0.01786 and the posterior SD is 0.1336.
- Posterior mean: (0/τ² + Σxy/σ²) / 56 = 110.2 / 56 = 1.9679.
- 95% credible interval: 1.9679 ± 1.96 × 0.1336 = 1.9679 ± 0.2619, or [1.706, 2.230].
The posterior mean (1.968) sits slightly below the OLS value (2.004) because the prior centred on 0 pulls it down. The prior carries weight equal to one observation of x = 1 (precision 1) against the data’s 55, so the pull is small: 1/56 of the way. Widen the prior to τ² = 100 and the posterior mean becomes 110.2 / 55.01 = 2.0033, practically the OLS answer. The calculation was repeated in code and matches these figures.
Interpreting Bayesian regression results
| Output | How to read it |
|---|---|
| Posterior mean or median | A central summary of the coefficient given the model, the prior and the data. |
| 95% credible interval | Under the model and prior, the coefficient lies in this interval with 95% posterior probability. A frequentist 95% confidence interval instead describes the long-run coverage of the procedure over repeated samples. |
| Posterior probability, e.g. P(β > 0) | Direct probability statement about the coefficient, again conditional on model and prior. |
| Rhat (MCMC fits) | If considerably greater than 1, the sampler has not converged (brms documentation). |
| Effective sample size (ESS) | How many independent draws the correlated chain is worth; low values mean unstable summaries. |
See summarizing a posterior distribution and interpreting Bayes factors for more on intervals and model comparison. For the frequentist counterpart of the interval, see confidence interval.
Bayesian regression vs OLS
With a flat prior and Gaussian errors the point estimates coincide; the differences are that Bayesian regression (1) lets you encode prior information or regularisation explicitly, (2) returns a full posterior from which any function of the coefficients can be summarised, and (3) gives intervals with a direct probability reading. The cost is that results depend on the prior, which must be chosen and reported. For the OLS side, see linear and multiple regression.
Assumptions and common mistakes
- Model assumptions still apply. The likelihood (for example linearity, independent errors, constant variance in the Gaussian model) is as much an assumption as in OLS. A prior does not repair a misspecified likelihood.
- Unexamined default priors. Software defaults are a choice. State them, and check how much the posterior changes under a plausible alternative (prior sensitivity analysis).
- Skipping convergence checks. For sampled posteriors, inspect Rhat, ESS and trace plots before interpreting anything.
- Reading a credible interval as a confidence interval, or the reverse. The two answer different questions (see the table above).
- Scale-blind priors. A N(0, 10²) prior means different things for coefficients measured in metres and in millimetres; standardise predictors or set priors on a meaningful scale.
How to report Bayesian regression (APA-style example)
Report the model, the priors, the software and sampler settings, convergence diagnostics, and posterior summaries with intervals. A sentence in the usual APA style, using the worked example:
We fitted a Bayesian linear regression with a Normal(0, 1) prior on the slope and a known error SD of 1. The posterior mean slope was 1.97 (posterior SD = 0.13, 95% credible interval [1.71, 2.23]). Under a Normal(0, 10) prior the posterior mean slope was 2.00.
For sampled models also give the number of chains and iterations, and confirm that all Rhat values were close to 1. For reporting frequentist intervals, compare how to report confidence intervals.
How to calculate Bayesian regression in Excel, R or Python
Excel (conjugate example)
With predictor values in A2:A6, outcomes in B2:B6, error variance in D1 (σ²) and prior variance in D2 (τ², prior mean 0), the posterior mean and SD of the slope are:
Posterior mean: =SUMPRODUCT(A2:A6,B2:B6)/(D1/D2+SUMSQ(A2:A6))
Posterior SD: =SQRT(D1/(D1/D2+SUMSQ(A2:A6)))
With D1 = 1 and D2 = 1 these return 1.9679 and 0.1336.
R (brms)
The brms package fits Bayesian generalized (non-)linear multivariate multilevel models using Stan, with a formula syntax very similar to lme4 (brms documentation):
library(brms)
fit <- brm(y ~ x, data = d, family = gaussian(),
prior = prior(normal(0, 10), class = b))
summary(fit) # posterior means, 95% credible intervals, Rhat, ESS
Python (PyMC)
PyMC specifies the model with explicit priors and samples with NUTS by default (PyMC documentation):
import pymc as pm, arviz as az
with pm.Model():
alpha = pm.Normal("alpha", mu=0, sigma=10)
beta = pm.Normal("beta", mu=0, sigma=10)
sigma = pm.HalfNormal("sigma", sigma=1)
mu = alpha + beta * x
pm.Normal("y_obs", mu=mu, sigma=sigma, observed=y)
idata = pm.sample()
az.summary(idata, kind="stats")
FAQ
What is Bayesian linear regression?
It is linear regression in which coefficients and error variance have prior distributions, updated by the data into a posterior distribution. With a conjugate normal prior and known variance, the posterior is normal with the precision-weighted mean shown above.
Is Bayesian regression the same as ridge regression?
Not in general, but the posterior mode under a zero-mean normal prior is the ridge estimate with λ = σ²/τ². Bayesian regression additionally returns the whole posterior, not only the mode.
Do I need MCMC for Bayesian regression?
No for simple conjugate cases such as the worked example, which have closed forms. Yes for most realistic models, such as logistic or multilevel regression.
How do I choose a prior?
Set it on the scale of the data using domain knowledge, state it in the write-up, and show that conclusions hold under reasonable alternatives.
Sources
- Statlect, Bayesian regression (conjugate normal results; read 2026-11-08).
- brms documentation, paulbuerkner.com/brms (read 2026-11-08).
- PyMC documentation, PyMC overview (read 2026-11-08).
- Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian Data Analysis (3rd ed.). Chapman and Hall/CRC.







