Written and maintained by CASRAI Editorial Board
Last updated
A calibration curve in machine learning (also called a reliability diagram) plots the empirical fraction of positive outcomes on the vertical axis against the model’s predicted probability on the horizontal axis across partitioned probability bins. While classical biostatistical calibration plots for prediction models and calibration slope and calibration-in-the-large (CITL) evaluate logistic and survival regressions on the log-odds scale, machine learning calibration engineering focuses on quantifying Expected Calibration Error (ECE), diagnosing architectural overconfidence or margin distortion, and applying post-hoc recalibration algorithms—Platt scaling, isotonic regression, and temperature scaling—on held-out validation splits.
Why Do Deep Neural Networks and Gradient-Boosted Trees Produce Poorly Calibrated Probability Estimates?
A binary or multi-class classifier is well-calibrated if, among all validation instances assigned a predicted probability $hat{p} = 0.80$, approximately $80%$ truly belong to that class: $mathbb{P}(Y = 1 mid hat{P} = p) = p$. High discrimination (AUROC) does not imply calibration: any strictly monotonic transformation of $hat{p}$ leaves AUROC unchanged while arbitrarily distorting absolute risk estimates. Two landmark machine learning studies explain why distinct model families systematically violate calibration:
- Boosted Decision Trees and SVMs (Sigmoid Under-Confidence — Niculescu-Mizil & Caruana, ICML 2005): In Predicting Good Probabilities With Supervised Learning, Niculescu-Mizil and Caruana demonstrated that maximum-margin methods (Support Vector Machines) and boosted trees (AdaBoost, gradient-boosted decision trees) concentrate predicted scores away from $0.0$ and $1.0$ because minimizing exponential or hinge margin loss penalizes examples near the decision boundary rather than optimizing probability fidelity at the extremes. On a reliability diagram, their raw scores form a characteristic sigmoid ($S$-shaped) curve that lies below the $y = x$ diagonal for low predictions and above the diagonal for high predictions. (Conversely, Naive Bayes violates conditional independence and pushes probabilities toward $0$ and $1$, producing an inverse-sigmoid curve.)
- Modern Deep Neural Networks (Softmax Overconfidence — Guo et al., ICML 2017): In On Calibration of Modern Neural Networks, Guo, Pleiss, Sun, and Weinberger showed that while shallow neural networks (such as 1998 LeNet-5) were well-calibrated, modern deep convolutional and transformer architectures (e.g., ResNet-110, DenseNet) with increased depth, width, batch normalization, and minimal L2 weight decay exhibit severe overconfidence. During training, after top-1 classification error plateaus, the network continues minimizing cross-entropy / negative log-likelihood (NLL) on the training set by magnifying logit magnitudes $z_i$, driving $max_k text{softmax}(z_i)_k to 1.0$ even on misclassified test examples. On a reliability diagram, the curve bows below the $y = x$ diagonal.
How Is Expected Calibration Error (ECE) Calculated and What Are Its Binning Limitations?
To summarize a reliability diagram into a scalar metric alongside proper scoring rules like the Brier score and Negative Log-Likelihood (NLL), machine learning benchmarks compute the Expected Calibration Error (ECE) and Maximum Calibration Error (MCE).
Given $N$ validation samples ${(x_i, y_i)}_{i=1}^N$ with predicted class $hat{y}_i = argmax_k hat{p}_{i,k}$ and predicted confidence $hat{p}_i = max_k hat{p}_{i,k} in [0, 1]$, partition $[0, 1]$ into $M$ mutually exclusive intervals (bins) $I_m = (frac{m-1}{M}, frac{m}{M}]$ for $m = 1, dots, M$, and let $B_m$ denote the set of sample indices whose confidence $hat{p}_i$ falls into $I_m$:
acc(B_m) = (1 / |B_m|) * sum_{i in B_m} 1(y_hat_i == y_i)
conf(B_m) = (1 / |B_m|) * sum_{i in B_m} p_hat_i
ECE = sum_{m=1}^{M} (|B_m| / N) * |acc(B_m) - conf(B_m)|
MCE = max_{m in {1..M}} |acc(B_m) - conf(B_m)|Worked Numerical Example of ECE and MCE ($M = 5$ Bins, $N = 200$ Patients)
Suppose a deep learning sepsis classifier is evaluated on $N = 200$ held-out emergency department encounters partitioned into $M = 5$ equal-width bins:
| Bin $m$ | Probability Interval $I_m$ | Sample Count $|B_m|$ | Weight $|B_m|/N$ | Mean Confidence $text{conf}(B_m)$ | Empirical Accuracy $text{acc}(B_m)$ | Absolute Gap $|text{acc} – text{conf}|$ | Weighted Contribution |
|---|---|---|---|---|---|---|---|
| 1 | $[0.0, 0.2]$ | 50 | 0.25 | 0.08 | 0.10 | 0.02 | 0.0050 |
| 2 | $(0.2, 0.4]$ | 20 | 0.10 | 0.31 | 0.25 | 0.06 | 0.0060 |
| 3 | $(0.4, 0.6]$ | 15 | 0.075 | 0.52 | 0.40 | 0.12 | 0.0090 |
| 4 | $(0.6, 0.8]$ | 25 | 0.125 | 0.74 | 0.60 | 0.14 | 0.0175 |
| 5 | $(0.8, 1.0]$ | 90 | 0.45 | 0.94 | 0.78 | 0.16 | 0.0720 |
Summing the weighted contributions gives $text{ECE} = 0.0050 + 0.0060 + 0.0090 + 0.0175 + 0.0720 = 0.1095$ ($10.95%$ average calibration error), while the worst-case bin error is $text{MCE} = max(0.02, 0.06, 0.12, 0.14, 0.16) = 0.160$ ($16.0%$ in Bin 5). Notice that $90 / 200$ predictions ($45%$) pile into the highest confidence bin $(0.8, 1.0]$ where the model predicts $94%$ confidence but achieves only $78%$ empirical accuracy—the classic signature of neural network overconfidence.
Known Statistical Limitations of Bin-Based ECE
- Uniform vs. Quantile Binning Sensitivity: Under `strategy=’uniform’` (equal-width intervals), overconfident models place almost all samples in the top bin, leaving intermediate bins with high-variance small sample counts $|B_m|$. Switching to `strategy=’quantile’` (Adaptive Calibration Error, ACE, where each bin contains $N/M$ samples) stabilizes bin variance.
- Bias-Variance Tradeoff in $M$: Increasing $M$ reduces bias (fewer opposite-sign errors cancelling inside a bin) but increases finite-sample estimation variance, causing raw ECE to be positively biased upward on small validation sets.
- Top-Label vs. Marginal/Classwise Scope: Standard multi-class ECE evaluates only $max_k hat{p}_{i,k}$; for clinical risk models Required by TRIPOD+AI, always report class-specific calibration curves, CITL, calibration slope, and the Integrated Calibration Index (ICI) via LOESS/splines alongside ECE.
Platt Scaling vs. Isotonic Regression vs. Temperature Scaling: Post-Hoc Recalibration Compared
Post-hoc recalibration fits a monotonic mapping from raw model logits $z_i$ or uncalibrated scores $s_i$ to calibrated probabilities $hat{q}_i$ using an independent held-out calibration dataset (or out-of-fold cross-validation predictions) that was never used to train model weights:
| Method | Mathematical Transformation | Free Parameters | Minimum Calibration Sample Size | Primary Model Fit & Properties |
|---|---|---|---|---|
| Platt Scaling (Platt, 1999) | $hat{q}_i = sigma(A s_i + B) = frac{1}{1 + exp(A s_i + B)}$ | 2 scalars ($A, B$) for binary classification | Small ($n approx 200text{–}1,000$) | Corrects parametric sigmoid distortion in SVMs, boosted trees, and random forests; low overfitting risk |
| Temperature Scaling (Guo et al., 2017) | $hat{q}_{i,k} = frac{exp(z_{i,k} / T)}{sum_j exp(z_{i,j} / T)}, quad T > 0$ | 1 scalar ($T$) shared across all $K$ classes | Very small ($n approx 100text{–}500$) | Multi-class deep neural networks; $T > 1$ softens overconfident logits while strictly preserving $argmax_k z_{i,k}$ and AUROC |
| Isotonic Regression (Zadrozny & Elkan, 2002) | $min_{hat{q}} sum_{i=1}^n (y_i – hat{q}_i)^2 text{ s.t. } hat{q}_i le hat{q}_j ; forall s_i le s_j$ | Non-parametric ($m$ piecewise-constant steps via PAVA) | Large ($n ge 1,000text{–}5,000$) | Corrects arbitrary non-sigmoidal monotonic distortions; overfits into jagged step functions on small validation sets |
Python Implementation: Plotting Calibration Curves and Recalibrating with Scikit-Learn and PyTorch
In Python (`scikit-learn`), `calibration_curve` computes the empirical bin coordinates, while `CalibratedClassifierCV` fits either Platt scaling (`method=’sigmoid’`) or Isotonic Regression (`method=’isotonic’`) on a held-out calibration split (`cv=’prefit’` in older versions or `FrozenEstimator` in `scikit-learn >= 1.6`):
import numpy as np
from sklearn.calibration import calibration_curve, CalibratedClassifierCV
from sklearn.metrics import brier_score_loss
# 1. Reliability diagram coordinates (10 quantile bins)
prob_true, prob_pred = calibration_curve(
y_val, y_prob_uncal, n_bins=10, strategy="uniform"
)
# 2. Expected Calibration Error (ECE) with M = 10 uniform bins
bins = np.linspace(0.0, 1.0, 11)
bin_ids = np.digitize(y_prob_uncal, bins[1:-1], right=True)
ece = sum(
(np.sum(bin_ids == m) / len(y_val))
* abs(np.mean(y_val[bin_ids == m]) - np.mean(y_prob_uncal[bin_ids == m]))
for m in range(10) if np.sum(bin_ids == m) > 0
)
# 3. Post-hoc Platt scaling ('sigmoid') vs Isotonic Regression ('isotonic')
platt_model = CalibratedClassifierCV(base_clf, method="sigmoid", cv=5).fit(X_cal, y_cal)
iso_model = CalibratedClassifierCV(base_clf, method="isotonic", cv=5).fit(X_cal, y_cal)
print(f"Uncalibrated ECE: {ece:.4f} | Brier: {brier_score_loss(y_test, y_prob_uncal):.4f}")Frequently Asked Questions
Why do deep neural networks and gradient-boosted trees produce poorly calibrated probability estimates?
They fail in opposite directions for distinct mathematical reasons. As Niculescu-Mizil and Caruana (ICML 2005) showed, maximum-margin classifiers and boosted decision trees (AdaBoost, gradient boosting) push predicted scores away from the extremes (0 and 1) toward the margin, producing an under-confident, sigmoid-shaped reliability diagram. Conversely, as Guo et al. (ICML 2017) demonstrated, modern deep neural networks with high depth, wide layers, batch normalization, and cross-entropy loss continue reducing negative log-likelihood (NLL) on training examples long after classification error plateaus, pushing softmax outputs toward 1.0 and producing severe overconfidence.
How is Expected Calibration Error (ECE) calculated and what are its binning limitations?
Expected Calibration Error (ECE) partitions $N$ validation predictions into $M$ probability bins $B_1, dots, B_M$ and computes the weighted average of the absolute difference between empirical accuracy $text{acc}(B_m)$ and mean predicted confidence $text{conf}(B_m)$: $text{ECE} = sum_{m=1}^M (|B_m|/N) |text{acc}(B_m) – text{conf}(B_m)|$. Its primary limitations are sensitivity to the choice of $M$ and binning scheme (equal-width uniform bins often leave middle bins sparse in overconfident neural networks compared to equal-frequency quantile bins), evaluation of only the top-1 class rather than the full probability vector, and sample-size bias.
When should clinical AI engineers use Platt scaling, temperature scaling, or isotonic regression for post-hoc recalibration?
Use Platt scaling (2-parameter logistic transform) for binary classifiers such as SVMs, boosted trees, or logistic regressions when the reliability curve exhibits a parametric sigmoid distortion or when the held-out calibration dataset is small (fewer than 1,000 samples). Use temperature scaling (1-parameter logit divisor $T > 0$) for multi-class deep neural networks because it softens overconfident softmax probabilities without altering the $argmax$ class prediction or AUROC. Use isotonic regression (non-parametric piecewise-constant monotonic fit via PAVA) only when the distortion is non-sigmoidal and you have at least 1,000 to 5,000 independent calibration samples to prevent step-function overfitting.
Sources
- Guo C, Pleiss G, Sun Y, Weinberger KQ. On Calibration of Modern Neural Networks. Proceedings of the 34th International Conference on Machine Learning (ICML 2017), PMLR 70:1321–1330.
- Niculescu-Mizil A, Caruana R. Predicting Good Probabilities With Supervised Learning. Proceedings of the 22nd International Conference on Machine Learning (ICML 2005), pp. 625–632.
- Scikit-Learn Developers. Probability Calibration User Guide (
calibration_curveandCalibratedClassifierCV).
