Written and maintained by CASRAI Editorial Board
Last updated
A generalised additive model (GAM) lets the relationship between a predictor and a response take a smooth, data-driven shape instead of a straight line, while keeping the familiar generalised linear model framework for the response distribution. In R, the standard way to fit one is the gam() function in the mgcv package. This guide covers fitting a GAM with mgcv and, because the fit is only half the job, how to check it: the effective degrees of freedom, the approximate smooth-term p-values, and the gam.check() basis-dimension test.
What it is used for in research
The mgcv documentation describes gam() as fitting a generalized additive model to data, with the response distribution and link set through a family object. Researchers reach for a GAM when a scatterplot or residual plot shows a curved relationship that a straight-line term, or a hand-picked transformation, does not describe well. The CRAN entry for mgcv (Mixed GAM Computation Vehicle with Automatic Smoothness Estimation, maintained by Simon Wood) lists the package scope as generalized additive (mixed) models, some of their extensions, and other generalized ridge regression, with smoothing parameters estimated by (Restricted) Marginal Likelihood, Cross Validation or iterated nested Laplace approximation. In other words, you do not choose how wiggly each curve is by hand; the software estimates it.
If your outcome is skewed and you are weighing a transformation first, see log and other data transformations and left-skewed vs right-skewed data. If your question is about group differences with a covariate, ANCOVA in R covers the linear case.
Install or access (official source only)
mgcv is distributed on CRAN, the official R package repository. The CRAN package page (checked 2026-10-29) lists version 1.9-4, published 2025-11-07, with priority “recommended” and a GPL-2 | GPL-3 licence. Install and load it with:
install.packages("mgcv")
library(mgcv)Because the CRAN page marks mgcv as a recommended package, it is typically already present in a standard R installation (not confirmed on the CRAN page itself); if library(mgcv) runs without error you can skip the install step.
Step-by-step: fit and check a GAM with mgcv
The usage line on the gam help page begins gam(formula, family=gaussian(), data=list(), weights=NULL, subset=NULL, na.action, offset=NULL, method="GCV.Cp", ...). The steps below use a generic data frame dat with a response y and a continuous predictor x; replace these with your own names.
- Fit the model with REML smoothness selection. The help page lists
method="REML"for REML estimation, including of unknown scale, and notes it is the default beyond the exponential family (the page’s own usage line and this statement are not reconciled). The printed default in the usage line is"GCV.Cp", so request REML explicitly if that is what you want:m1 <- gam(y ~ s(x), data = dat, family = gaussian(), method = "REML") - Choose the family for your response. The
familyargument is the family object that specifies the distribution and link, so a count or binary outcome uses the corresponding family in place ofgaussian(). - Read the summary.
summary(m1) - Run the model checks. The
gam.check()help page gives the usagegam.check(b, old.style=FALSE, type=c("deviance","pearson","response"), k.sample=5000, k.rep=200, rep=0, level=.9, ...):gam.check(m1) - Optionally let terms shrink to zero. Setting
select=TRUEadds an extra penalty to each term, so, in the help page’s words, it can be penalized to zero. Thegammaargument can be increased to penalize more strongly.m2 <- gam(y ~ s(x), data = dat, method = "REML", select = TRUE)
Output and how to read it
edf: how wiggly is each smooth?
In summary() output, the effective degrees of freedom (edf) are the estimated degrees of freedom for each model term. Read an edf as a measure of how much flexibility the smooth actually used: it is estimated from the data, not set by you. The summary help page does not define the Ref.df column, so this guide does not either; consult that page directly if you need it.
p-values for smooth terms are approximate
The summary help page warns that the p-values are approximate and neglect smoothing parameter uncertainty. Treat a smooth-term p-value close to a threshold as indicative rather than decisive, and report it as such. Random-effect terms are tested with a likelihood ratio test.
r.sq and dev.expl
r.sq is the adjusted R-squared, and the help page notes it can be negative if your model is worse than a one-parameter constant model. dev.expl is the proportion of the null deviance explained by the model. For a non-normal response family, report which of the two you quote.
gam.check: the k-index and its p-value
gam.check() produces four standard diagnostic plots, smoothing parameter estimation convergence information, and tests whether the basis dimension k is adequate. Two rules from its help page matter most:
- The k-index falls below 1 when a pattern remains in the residuals. The further below 1 it is, the more likely it is that missed pattern is left in the residuals.
- The p-value comes from randomly re-shuffling the residuals (
k.reptimes), so it can vary widely between replicates. Do not read a single run as final; repeat the check before acting on a borderline value.
Residual diagnostics are also where outliers surface; Cook’s distance and influential observations explains how to judge whether a single observation is driving a fit.
Troubleshooting
Low k-index and a low p-value
The help page says low p-values may indicate that the basis dimension k has been set too low, especially if the edf is close to its maximum, and that doubling a suspect k and re-fitting is sensible. The same page cautions that a low p-value can have other causes, so a larger k that does not move the k-index suggests looking elsewhere. For example, with a generic term:
m3 <- gam(y ~ s(x, k = 20), data = dat, method = "REML")
gam.check(m3)Convergence messages
The gam.check() output includes a summary of convergence of the smoothness selection step. If it reports that smoothness selection did not converge, see the optimizer argument in the gam help page for the available options before changing it.
Residuals from gamm()
For models fitted with gamm(), the help page notes that residuals are not standardised, so interpret the residual plots with that in mind.
Free alternatives
Package descriptions below are from each package’s CRAN page (checked 2026-10-29). All are open-source R packages.
| Package | What the CRAN page says it does |
|---|---|
| mgcv | Generalized additive (mixed) models, some of their extensions and other generalized ridge regression, with automatic smoothness estimation |
| gam | Functions for fitting and working with generalized additive models, as described in chapter 7 of “Statistical Models in S” (Chambers and Hastie, 1991) and “Generalized Additive Models” (Hastie and Tibshirani, 1990) |
| gamlss | Generalized Additive Models for Location Scale and Shape (Rigby and Stasinopoulos, 2005), a distributional regression approach in which all parameters of the response distribution are modelled using explanatory variables |
| VGAM | Titled “Vector Generalized Linear and Additive Models” on CRAN |
FAQ
Which R package fits generalised additive models?
The mgcv package, through its gam() function. CRAN lists it with priority “recommended”. The gam and gamlss packages are separate implementations with different scopes, listed in the table above.
Should I use method = “REML” or the default?
The usage line on the gam help page prints method="GCV.Cp" as the default, while the help text lists "REML" as the choice for REML estimation. State the method you used in your methods section so the fit can be reproduced.
How do I know whether k is large enough?
Run gam.check(). A k-index below 1 with a low p-value, and an edf near its maximum, suggest k is too low; the documentation suggests doubling it and refitting. Because the p-value is based on random reshuffling, repeat the check if the result is borderline.
Are the smooth-term p-values exact?
No. The summary.gam help page says they are approximate and neglect smoothing parameter uncertainty.
Sources (official docs)
- CRAN: mgcv package page (checked 2026-10-29)
- mgcv manual: gam (checked 2026-10-29)
- mgcv manual: gam.check (checked 2026-10-29)
- mgcv manual: summary.gam (checked 2026-10-29)
- CRAN package pages for gam, gamlss and VGAM (checked 2026-10-29)
