Written and maintained by CASRAI Editorial Board
Last updated
When biostatisticians, clinical machine learning engineers, and hospital AI governance committees evaluate binary diagnostic tests or clinical prediction models—under reporting standards such as the TRIPOD+AI statement and Coalition for Health AI (CHAI) model cards—two threshold-independent discrimination summaries dominate the literature: AUROC (Area Under the Receiver Operating Characteristic Curve) and AUPRC (Area Under the Precision-Recall Curve). While both evaluate a classifier across all continuous probability thresholds from 0 to 1, they behave fundamentally differently when the clinical outcome is rare.
What Is the Difference Between AUROC and AUPRC (Area Under the Precision-Recall Curve)?
At any probability threshold τ, a binary clinical classifier divides a validation cohort into four confusion-matrix counts: True Positives (TP), False Positives (FP), True Negatives (TN), and False Negatives (FN). AUROC and AUPRC construct their curves from different pairs of conditional rates:
- AUROC (Receiver Operating Characteristic Space): Plots the True Positive Rate (Sensitivity or Recall) on the y-axis against the False Positive Rate (1 − Specificity) on the x-axis:
- Sensitivity (Recall / TPR) = TP / (TP + FN) — conditioned solely on actual positive cases.
- False Positive Rate (FPR) = FP / (FP + TN) — conditioned solely on actual negative cases.
Mathematically, AUROC equals the non-parametric Wilcoxon-Mann-Whitney probability (the C-statistic) that a randomly selected patient with the event is assigned a higher predicted risk score than a randomly selected patient without the event.
- AUPRC (Precision-Recall Space): Plots Precision (Positive Predictive Value, PPV) on the y-axis against Recall (Sensitivity / TPR) on the x-axis:
- Precision (PPV) = TP / (TP + FP) — the proportion of flagged positive alerts that are true cases.
- Recall (Sensitivity) = TP / (TP + FN) — the proportion of true cases captured by the alert.
Crucially, True Negatives (TN) do not appear anywhere in either axis of the Precision-Recall curve.
| Evaluation Dimension | AUROC (Area Under ROC Curve) | AUPRC (Area Under Precision-Recall Curve) |
|---|---|---|
| Y-Axis vs. X-Axis | Sensitivity (TP / [TP + FN]) vs. False Positive Rate (FP / [FP + TN]) | Precision / PPV (TP / [TP + FP]) vs. Recall / Sensitivity (TP / [TP + FN]) |
| Dependence on True Negatives (TN) | Directly includes TN in the denominator of the x-axis (FPR) | Completely excludes TN; evaluates positive predictions and true events only |
| No-Skill (Random Classifier) Baseline | Fixed at 0.50 regardless of disease prevalence | Equals the event prevalence π = P / (P + N) (e.g., 0.01 for a 1% event) |
| Invariance to Class Skew | Invariant to case-control sampling ratio if conditional distributions are unchanged | Directly shifts with outcome prevalence; must be reported alongside cohort prevalence |
| Curve Interpolation Rule | Linear trapezoidal interpolation between operating points is exact | Linear interpolation is biased upward; requires non-linear Davis-Goadrich or Average Precision (AP) step integration |
Why Can AUROC Look Deceptively High in Low-Prevalence Clinical Screening or Rare-Event Sepsis Alerts?
As demonstrated in landmark methodological analyses by Davis & Goadrich (ICML, 2006) and Saito & Rehmsmeier (PLOS ONE, 2015), relying solely on AUROC to validate inpatient clinical deterioration, SOFA score sepsis early-warning models, or rare-cancer screening algorithms creates a severe blind spot whenever the non-event class (N = FP + TN) vastly outnumbers the event class (P = TP + FN).
Worked Numerical Example (1% Sepsis Prevalence across 10,000 Patient-Days):
- Suppose an inpatient cohort contains 100 true sepsis events (P = 100) and 9,900 non-sepsis patient-days (N = 9,900), so baseline prevalence π = 1.0% (0.01).
- At a chosen operating threshold, an EHR machine learning model correctly detects 80 of the 100 sepsis cases (TP = 80, FN = 20) while firing 720 false-positive sepsis alerts (FP = 720, TN = 9,180).
- In ROC Space: Sensitivity = 80 / 100 = 80.0%, and Specificity = 9,180 / 9,900 = 92.7% (False Positive Rate = 720 / 9,900 = 7.3%). A classifier operating near (FPR = 0.073, TPR = 0.80) typically achieves an overall AUROC > 0.90, which looks excellent on a vendor specification sheet because the 9,180 True Negatives dominate the FPR denominator.
- In Precision-Recall Space: Precision (PPV) = 80 / (80 + 720) = 80 / 800 = 10.0%. In clinical practice, 9 out of every 10 bedside sepsis alerts (90%) are false alarms, triggering severe nursing and physician alert fatigue and leading to an AUPRC near 0.15–0.25.
How Does Baseline Disease Prevalence Determine the No-Skill Baseline of a Precision-Recall Curve?
Unlike AUROC—where a coin-flip classifier always traces the diagonal line y = x with an area of 0.50—an uninformative random classifier in Precision-Recall space produces a flat horizontal line at y = π = P / (P + N), where π is the positive event prevalence in the evaluation dataset:
- Reporting Prevalence Is Mandatory for AUPRC Interpretation: An AUPRC of 0.35 represents a 35-fold lift over a random baseline when evaluating an outcome with 1% prevalence (π = 0.01), whereas the exact same AUPRC of 0.35 is worse than random guessing in a balanced 1:1 case-control dataset where π = 0.50.
- Correct Numerical Integration (Avoiding Linear PR Interpolation Bias): As proved by Davis & Goadrich (2006), connecting sparse operating points in Precision-Recall space with straight lines violates the hyperbolic definition TP / (TP + FP) and artificially inflates AUPRC. Clinical data scientists should compute AUPRC using either Davis-Goadrich hyperbolic interpolation or rectangular step-wise Average Precision (AP) across all unique predicted probability thresholds.
Frequently Asked Questions
What is the difference between AUROC and AUPRC (Area Under the Precision-Recall Curve)?
AUROC plots Sensitivity (True Positive Rate) against 1 − Specificity (False Positive Rate), measuring rank-order discrimination across both positive and negative classes with a fixed random baseline of 0.50. AUPRC plots Precision (Positive Predictive Value) against Recall (Sensitivity), excluding True Negatives entirely to focus on the accuracy and false-alarm burden of positive predictions.
Why can AUROC look deceptively high in low-prevalence clinical screening or rare-event sepsis/deterioration alerts?
In low-prevalence clinical settings (such as a 1% event rate), the massive count of True Negatives (TN) inflates the denominator of the False Positive Rate, FP / (FP + TN), keeping FPR small even when hundreds of false-positive alerts occur. As a result, a model can exhibit an AUROC above 0.90 while suffering from a Precision (PPV) below 10% and a low AUPRC.
How does baseline disease prevalence determine the no-skill baseline of a Precision-Recall curve?
A random (no-skill) classifier achieves a constant Precision equal to the positive class prevalence π = P / (P + N) across all Recall levels. Therefore, the baseline AUPRC of a random model is exactly equal to the disease prevalence in the cohort (for example, 0.02 for a 2% incidence outcome), making it essential to report cohort prevalence alongside every AUPRC value.
