Written and maintained by CASRAI Editorial Board
Last updated
Decision curve analysis (DCA) answers a question that discrimination and calibration cannot: given a prediction model that already ranks cases well and already reports trustworthy predicted probabilities, is acting on it actually better than the two default strategies every clinical decision already has available — treating everyone, or treating no one? A model can have excellent AUC and a calibration slope of 1.0 and still fail this test, because AUC and calibration describe statistical properties of the predictions themselves, not the consequences of acting on them. Introduced by Andrew Vickers and Elena Elkin in a 2006 Medical Decision Making paper, decision curve analysis quantifies a model’s clinical usefulness across a range of threshold probabilities using a single summary metric — net benefit — and plots it against those two reference strategies so a reader can see, at a glance, whether the model is ever worth using and over what range of decision thresholds.
What decision curve analysis actually asks
Discrimination (AUC, the c-statistic) asks whether a model ranks people who will have the outcome above people who will not, on average — see CASRAI’s Sensitivity vs. Specificity guide for the 2×2 mechanics ROC curves are built from. Calibration asks whether the model’s predicted probabilities can be taken at face value — among everyone predicted a 20% risk, do roughly 20% actually have the outcome, as covered in CASRAI’s calibration plots guide and Hosmer-Lemeshow test guide. Neither question touches what happens once a clinician acts on a prediction. A model can discriminate well and be perfectly calibrated and still not be worth deploying, if the false positives it generates cost more — in unnecessary procedures, anxiety, or downstream harm — than the true positives it catches are worth. Decision curve analysis is the method that puts a number on exactly that trade-off, expressed as a single quantity called net benefit.
This is also why net benefit, not a reclassification statistic, is the standard the methodological literature now recommends for summarizing whether adding a marker or refitting a model actually helps. Kerr et al. (2014) reviewed net reclassification indices and concluded plainly that “the preferred single-number summary of the prediction increment is the improvement in net benefit” — the category-free NRI, by contrast, “can mislead investigators by overstating the incremental value of a biomarker, even in independent validation data.” Net benefit is harder to inflate with a spuriously useful-looking marker because, unlike NRI, it is anchored to a real clinical decision threshold rather than to an arbitrary reclassification table.
Threshold probability: the exchange rate the whole method runs on
The threshold probability (pt) is the minimum predicted probability of the outcome at which a decision-maker would take the action being evaluated — order the biopsy, start the drug, flag the case for review. It is a clinical judgment, not a statistical one, and different actions have very different thresholds. A low-morbidity screening test might have a threshold around 5-10%: a clinician is willing to act on fairly weak evidence because acting is cheap. A major surgery with real complication risk might have a threshold closer to 50%: nobody operates unless the predicted risk clears a much higher bar.
The threshold does real mathematical work, not just intuitive work. Choosing pt implicitly states how many false positives a decision-maker would tolerate to catch one true positive: at threshold pt, a rational decision-maker is indifferent between one true positive and pt/(1−pt) false positives. A 10% threshold implies tolerating up to 0.10/0.90 ≈ 0.11 false positives per true positive caught — a very permissive exchange rate. A 50% threshold implies tolerating exactly one false positive per true positive — a much stricter one. That ratio, pt/(1−pt), is the odds at the threshold, and it is the exact weight net benefit uses to convert false positives into the same units as true positives.
Because no single threshold is universally correct, decision curve analysis does not evaluate a model at one pt. It plots net benefit across a clinically plausible range of thresholds and lets the reader judge where, if anywhere, the model beats the alternatives.
The net benefit formula
For a cohort of size n, with TP true positives and FP false positives identified by the model at threshold pt, net benefit is:
Net Benefit = (TP / n) − (FP / n) × [pt / (1 − pt)]
Read term by term: TP/n is the proportion of the whole cohort correctly identified and treated — the benefit. FP/n is the proportion unnecessarily treated — the harm, in raw terms. Multiplying FP/n by the odds at the threshold, pt/(1−pt), re-expresses that harm in true-positive-equivalent units, using the exchange rate the threshold itself defines. Subtracting gives a single number in units of “net true positives per patient,” which is why it is often reported as net benefit × 100, read as “true positives per 100 patients.” A net benefit of 0.12, for instance, is routinely reported as 12 true positives per 100 patients, net of the harm from false positives at that threshold.
The two reference strategies: treat-all and treat-none
Treat-none is the trivial baseline: nobody is flagged, so TP = 0 and FP = 0, and net benefit is 0 at every threshold. It is a flat horizontal line at zero on a decision curve.
Treat-all means acting on everyone regardless of what any model predicts. Every diseased case becomes a true positive and every healthy case becomes a false positive, so with prevalence π, treat-all’s net benefit is:
Net Benefittreat-all = π − (1 − π) × [pt / (1 − pt)]
This line starts at prevalence when pt is near 0 (treating everyone looks great if the bar for treating is almost nothing) and falls as pt rises, crossing zero exactly at pt = π: once the threshold exceeds the disease prevalence itself, treating everyone stops being defensible even without a model, because most of the people treated are false positives by definition. A model is only clinically useful at thresholds where its own net benefit exceeds both the treat-all and treat-none lines — beating one but not the other is not enough.
A worked net benefit calculation
The numbers below are an illustrative worked calculation, not a result from any specific published study — they are chosen to be internally consistent and are computed directly from the stated inputs, not fitted to any real dataset. Take a hypothetical validation cohort of n = 1,000 patients with an outcome prevalence of 20% (200 with the outcome, 800 without), and a prediction model evaluated at a clinical threshold of pt = 20%. Suppose the model, at that threshold, correctly flags 150 of the 200 true cases (sensitivity 75%) and incorrectly flags 100 of the 800 non-cases (specificity 87.5%) — TP = 150, FP = 100, FN = 50, TN = 700.
At pt = 0.20, the odds at the threshold are 0.20 / 0.80 = 0.25. Net benefit of the model:
NBmodel = (150/1000) − (100/1000) × 0.25 = 0.1500 − 0.0250 = 0.1250
Net benefit of treat-all, at the same threshold:
NBtreat-all = 0.20 − 0.80 × 0.25 = 0.20 − 0.20 = 0.0000
And treat-none is 0.0000 by definition. At the 20% threshold, the model clearly wins: 0.1250 versus 0.0000 for both alternatives, or 12.5 true-positive-equivalents per 100 patients gained by using the model instead of either default strategy.
That advantage is not constant across thresholds, which is exactly the point of plotting a curve rather than reporting one number. Recomputing the same TP/FP counts against the same formula at other thresholds:
| Threshold pt | NB, model | NB, treat-all | NB, treat-none |
|---|---|---|---|
| 5% | 0.1447 | 0.1579 | 0.0000 |
| 10% | 0.1389 | 0.1111 | 0.0000 |
| 15% | 0.1324 | 0.0588 | 0.0000 |
| 20% | 0.1250 | 0.0000 | 0.0000 |
| 25% | 0.1167 | −0.0667 | 0.0000 |
| 30% | 0.1071 | −0.1429 | 0.0000 |
Below about pt = 6.7% (the exact crossover, solved from the same two formulas), treat-all actually beats the model: when the bar for acting is set that low, treating everyone captures almost as many true positives as the model does while the model’s false positives still cost something at that threshold’s exchange rate. Above roughly pt = 60%, the model’s own net benefit drops to zero and treat-none becomes just as good, because the harm from its remaining false positives, weighted by that high threshold’s steep exchange rate, cancels out its remaining true positives. Between those two crossover points — roughly 7% to 60% in this example — the model is the best of the three strategies. In a real analysis the useful range to report is narrower still: whatever span of thresholds is clinically plausible for the actual decision being modeled, not the full mathematical range where the curves happen to cross.
Reading a decision curve plot
A decision curve plots threshold probability on the x-axis and net benefit on the y-axis, with three lines: treat-none (flat at zero), treat-all (the downward-sloping prevalence-anchored line above), and the model. Over the range of thresholds a clinician would actually consider reasonable for the decision at hand, the model is worth using only where its curve sits above both reference lines. If the model’s curve dips below treat-all or treat-none anywhere inside that clinically relevant range, that is direct evidence the model should not be used for decisions made at that threshold, even if its AUC and calibration both look good. Comparing two competing models is done the same way: whichever curve is higher across the clinically relevant threshold range provides more net benefit, and by how much can be read directly off the y-axis in true-positive-equivalents per 100 patients.
Why this is a different question from AUC or calibration
AUC is threshold-free by construction — it summarizes ranking ability averaged over every possible cutoff, most of which no clinician would ever actually use. Two models with identical AUC can have very different net benefit at the one or two thresholds that matter clinically, because AUC weights all cutoffs equally while a real decision only cares about the ones near its own threshold. Calibration is a necessary condition for a model’s predicted probabilities to be trustworthy at any given threshold, but it is not sufficient on its own to show the model is worth using — a model can be well calibrated and still add nothing over treat-all if its true and false positive rates land in the wrong place relative to the threshold’s exchange rate. Decision curve analysis does not replace discrimination or calibration reporting; TRIPOD+AI-era prediction-model reporting expects all three, because they answer three genuinely different questions — can the model rank cases, can its numbers be trusted, and is acting on it actually better than not using a model at all.
Limitations and common pitfalls
The threshold range has to be clinically defensible, not arbitrarily wide. Plotting net benefit from 0% to 100% and reporting wherever the model wins is not a meaningful claim if nobody would ever set a real decision threshold at, say, 2% or 85% for the action in question. The threshold range reported should reflect an actual clinical judgment about acceptable trade-offs, ideally elicited from the clinicians or patients who would use the model, not chosen after the fact because it makes the curve look favorable.
Net benefit is cohort-level, not a statement about any individual patient. It answers “would using this model across a population of patients like this one do more good than harm,” not “is this specific prediction correct.” Calibration still matters for the latter question.
External validation still applies. A decision curve built on the same data used to fit the model will look better than the model’s real-world performance, for the same overfitting reasons any other performance metric does — net benefit needs out-of-sample or externally validated predictions to mean anything for deployment decisions.
The basic formula assumes a fixed follow-up with no censoring. Time-to-event outcomes (survival data with loss to follow-up) need a survival-adapted version of the net benefit calculation that accounts for censoring rather than the simple TP/FP counts used above; using the plain formula on censored data will misstate net benefit.
Running and reporting decision curve analysis
Vickers and Elkin’s original 2006 paper included example code, and the method is now implemented in maintained R packages, including dcurves (available on CRAN, maintained by Daniel D. Sjoberg), which takes a data set of observed outcomes and model-predicted probabilities and produces both the net benefit table and the plot directly. When reporting decision curve analysis in a manuscript, state the threshold range used and why it was chosen, report the underlying TP/FP counts or sensitivity/specificity the curve was built from (not just the plot), and present it alongside — not instead of — discrimination and calibration reporting, consistent with current prediction-model reporting guidance; see CASRAI’s STARD and TRIPOD+AI checklist guide for the full reporting-item list these study types are now expected to satisfy.
Frequently asked questions
Is decision curve analysis the same as ROC analysis?
No. ROC analysis and its AUC summary evaluate discrimination — ranking ability — averaged across every possible threshold. Decision curve analysis evaluates clinical usefulness at specific, clinically meaningful thresholds, incorporating the relative cost of false positives versus false negatives that ROC/AUC deliberately ignores. See CASRAI’s Sensitivity vs. Specificity guide for the ROC mechanics decision curve analysis builds on but goes beyond.
Can a model have high AUC and still show no net benefit?
Yes, and this is one of the main reasons decision curve analysis exists. A model can rank cases well overall while its true/false positive balance at the specific thresholds that matter clinically fails to beat treat-all or treat-none. High AUC is evidence the model has some discriminative signal; it is not evidence the model is worth deploying for a specific decision.
What counts as a “good” net benefit value?
There is no universal good/bad cutoff — net benefit only means something relative to the treat-all and treat-none lines at the same threshold. A model with a small positive net benefit that still exceeds both reference strategies across the clinically relevant threshold range is doing real work; a much larger net benefit that never clears treat-all is not.
How is threshold probability chosen for a real analysis?
Ideally by eliciting it from clinicians or patients as the point at which they would actually change their decision — for example, the risk level at which a clinician would order a biopsy. Where no single number is agreed on, decision curve analysis is reported across a plausible range rather than at one point, which is the more common approach in practice.
Does decision curve analysis apply to non-binary or time-to-event outcomes?
The core method was developed for binary outcomes; survival/time-to-event extensions that account for censoring exist and are implemented in the same maintained R packages, but they use a modified net benefit calculation, not the plain TP/FP formula above.
