Written and maintained by CASRAI Editorial Board
Last updated
Harrell’s C-index (also called the concordance index for time-to-event outcomes) is the foundational non-parametric measure of predictive discrimination for right-censored survival models, including the Cox proportional hazards regression model, accelerated failure time (AFT) models, and survival machine learning algorithms. Introduced by Frank E. Harrell Jr. and colleagues in JAMA (1982;247(18):2543–2546) and expanded in Statistics in Medicine (1996;15(4):361–387), Harrell’s C-index generalizes the binary-outcome C-statistic (AUROC equivalence) to longitudinal cohorts where follow-up times are incomplete due to right-censoring.
Whereas a binary logistic regression model predicts the probability of an event over a fixed, uncensored observation window, clinical survival studies track patients over varying follow-up durations until an event (such as death, disease recurrence, or hospital readmission) or censoring (loss to follow-up or administrative study end). Harrell’s C-index quantifies the proportion of all usable (comparable) patient pairs in which the direction of the model’s predicted prognostic risk aligns with the observed order of survival times.
Mathematical Definition and Estimator Formula
Let $T_i$ denote the true latent event time and $C_i$ denote the right-censoring time for subject $i in {1, dots, n}$. In an observed clinical cohort, each subject provides an observed follow-up time $tilde{T}_i = min(T_i, C_i)$ and an event indicator $delta_i = I(T_i le C_i)$, where $delta_i = 1$ indicates that the event occurred at $tilde{T}_i$ and $delta_i = 0$ indicates that the subject was right-censored at $tilde{T}_i$. Let $hat{eta}_i$ denote the model-predicted prognostic risk score for subject $i$ (such as the Cox linear predictor $hat{eta}_i = X_i^top hat{beta}$, where a higher value indicates a higher hazard and shorter expected survival).
At the population level, the survival concordance probability is defined as:
$$C = P(hat{eta}_i > hat{eta}_j mid T_i < T_j)$$
Because true event times are only partially observed under right-censoring, Harrell’s empirical C-index restricts evaluation to comparable (usable) pairs where the ordering of true survival times $T_i < T_j$ is known with certainty:
$$hat{C}_{text{Harrell}} = frac{sum_{i neq j} delta_i , I(tilde{T}_i < tilde{T}_j) left[ I(hat{eta}_i > hat{eta}_j) + 0.5 , I(hat{eta}_i = hat{eta}_j) right]}{sum_{i neq j} delta_i , I(tilde{T}_i < tilde{T}_j)}$$
Rules for Comparable vs. Incomparable Pairs Under Right-Censoring
For any distinct pair of subjects $(i, j)$ ordered such that $tilde{T}_i le tilde{T}_j$, pair usability depends strictly on the event indicators $(delta_i, delta_j)$:
| Pair Scenario ($tilde{T}_i$ vs. $tilde{T}_j$) | Event Indicators ($delta_i, delta_j$) | Comparable (Usable)? | Contribution to Harrell’s Estimator |
|---|---|---|---|
| Both subjects experience the event at different times ($tilde{T}_i < tilde{T}_j$) | $delta_i = 1, ; delta_j = 1$ | Yes ($T_i < T_j$ is certain) | $+1$ to denominator; $+1$ to numerator if $hat{eta}_i > hat{eta}_j$, $+0.5$ if $hat{eta}_i = hat{eta}_j$, $+0$ if $hat{eta}_i < hat{eta}_j$ |
| Earlier subject has event; later subject is censored ($tilde{T}_i < tilde{T}_j$) | $delta_i = 1, ; delta_j = 0$ | Yes ($T_i = tilde{T}_i < tilde{T}_j le T_j$, so $T_i < T_j$ is certain) | $+1$ to denominator; $+1$ to numerator if $hat{eta}_i > hat{eta}_j$, $+0.5$ if $hat{eta}_i = hat{eta}_j$, $+0$ if $hat{eta}_i < hat{eta}_j$ |
| Earlier subject is censored; later subject has event or is censored ($tilde{T}_i < tilde{T}_j$) | $delta_i = 0, ; delta_j in {0, 1}$ | No (Subject $i$’s true event time $T_i > tilde{T}_i$ may occur before or after $T_j$) | Excluded from both numerator and denominator |
| Both subjects are censored at any times | $delta_i = 0, ; delta_j = 0$ | No (Neither true event time is observed) | Excluded from both numerator and denominator |
Note on Tied Predictor Scores vs. Tied Event Times: When a comparable pair ($tilde{T}_i < tilde{T}_j, delta_i = 1$) has tied predicted risk scores ($hat{eta}_i = hat{eta}_j$), the pair is half-concordant and contributes $0.5$ to the numerator. In standard implementations (such as the R survival::concordance() function and Python lifelines), pairs with identical observed event times ($tilde{T}_i = tilde{T}_j, delta_i = delta_j = 1$) are excluded from the strict inequality $T_i < T_j$ denominator, whereas pairs where one subject has an event at time $t$ and another is censored at the exact same time $t$ ($tilde{T}_i = tilde{T}_j = t, delta_i = 1, delta_j = 0$) are treated as comparable because censoring at time $t$ implies survival past the instant of the event at $t$.
Worked Numerical Calculation
Consider a 5-patient oncology survival validation cohort evaluated with a Cox proportional hazards risk score $hat{eta}_i$ (higher score = higher predicted mortality hazard):
- Patient A: $tilde{T}_A = 4$ months, $delta_A = 1$ (event), predicted risk $hat{eta}_A = 2.4$
- Patient B: $tilde{T}_B = 6$ months, $delta_B = 0$ (censored), predicted risk $hat{eta}_B = 1.9$
- Patient C: $tilde{T}_C = 9$ months, $delta_C = 1$ (event), predicted risk $hat{eta}_C = 1.5$
- Patient D: $tilde{T}_D = 12$ months, $delta_D = 1$ (event), predicted risk $hat{eta}_D = 1.8$
- Patient E: $tilde{T}_E = 15$ months, $delta_E = 0$ (censored), predicted risk $hat{eta}_E = 1.5$
Out of $binom{5}{2} = 10$ total pairs, we evaluate all pairs $(i, j)$ with $tilde{T}_i < tilde{T}_j$:
- (A, B) [$4 < 6, delta_A = 1$]: Comparable. Since $hat{eta}_A = 2.4 > hat{eta}_B = 1.9$, Concordant (+1.0).
- (A, C) [$4 < 9, delta_A = 1$]: Comparable. Since $hat{eta}_A = 2.4 > hat{eta}_C = 1.5$, Concordant (+1.0).
- (A, D) [$4 < 12, delta_A = 1$]: Comparable. Since $hat{eta}_A = 2.4 > hat{eta}_D = 1.8$, Concordant (+1.0).
- (A, E) [$4 < 15, delta_A = 1$]: Comparable. Since $hat{eta}_A = 2.4 > hat{eta}_E = 1.5$, Concordant (+1.0).
- (B, C), (B, D), (B, E) [$tilde{T}_B = 6, delta_B = 0$]: Incomparable (excluded) because Patient B was censored at 6 months before the observed follow-up times of C, D, and E.
- (C, D) [$9 < 12, delta_C = 1$]: Comparable. Since $hat{eta}_C = 1.5 < hat{eta}_D = 1.8$, Discordant (+0.0).
- (C, E) [$9 < 15, delta_C = 1$]: Comparable. Since $hat{eta}_C = 1.5 = hat{eta}_E = 1.5$, Tied risk (+0.5).
- (D, E) [$12 < 15, delta_D = 1$]: Comparable. Since $hat{eta}_D = 1.8 > hat{eta}_E = 1.5$, Concordant (+1.0).
Summing over the $7$ comparable pairs yields $5$ concordant pairs, $1$ discordant pair, and $1$ tied-risk pair:
$$hat{C}_{text{Harrell}} = frac{5(1.0) + 1(0.0) + 1(0.5)}{7} = frac{5.5}{7} approx 0.7857$$
Censoring Distribution Bias and Uno’s IPCW C-Statistic
A critical methodological limitation demonstrated by Uno et al. (Statistics in Medicine, 2011;30(10):1105–1117) is that as sample size $n to infty$, Harrell’s unweighted estimator $hat{C}_{text{Harrell}}$ converges to a study-dependent limit that is a function of the censoring survival distribution $G(t) = P(C > t)$. Because pairs with later event times are much more likely to be lost to censoring than pairs with early events, Harrell’s C-index places disproportionate weight on early failures and typically exhibits upward bias as the censoring rate increases.
Uno’s Inverse Probability of Censoring Weighted (IPCW) Estimator
To eliminate dependency on the censoring distribution, Uno et al. (2011) introduced an inverse probability of censoring weighted (IPCW) concordance estimator evaluated over a prespecified clinical truncation horizon $tau$ (such as $tau = 5$ years):
$$hat{C}_{text{Uno}}(tau) = frac{sum_{i neq j} delta_i , {hat{G}(tilde{T}_i)}^{-2} , I(tilde{T}_i < tilde{T}_j, tilde{T}_i < tau) left[ I(hat{eta}_i > hat{eta}_j) + 0.5 , I(hat{eta}_i = hat{eta}_j) right]}{sum_{i neq j} delta_i , {hat{G}(tilde{T}_i)}^{-2} , I(tilde{T}_i < tilde{T}_j, tilde{T}_i < tau)}$$
where $hat{G}(t)$ is the Kaplan-Meier estimator of the censoring time survival distribution $P(C > t)$. By weighting each comparable pair by ${hat{G}(tilde{T}_i)}^{-2}$, Uno’s C-statistic consistently estimates the censoring-free population concordance $P(hat{eta}_i > hat{eta}_j mid T_i < T_j, T_i < tau)$ under independent censoring.
Comparing Harrell’s C-Index, Uno’s C-Statistic, and Time-Dependent AUROC
When validating prognostic models in accordance with TRIPOD+AI reporting standards, biostatisticians pair rank-order concordance metrics with time-dependent ROC curves and calibration slope assessments:
| Metric | Target Parameter | Censoring Sensitivity | Primary Clinical Use Case |
|---|---|---|---|
| Harrell’s C-Index ($hat{C}_{text{Harrell}}$) | Global rank concordance across observed follow-up | Sensitive to study censoring rate and follow-up distribution $G(t)$ | Baseline global discrimination in proportional hazards models with low censoring (<20%–30%) |
| Uno’s IPCW C-Statistic ($hat{C}_{text{Uno}}(tau)$) | $P(hat{eta}_i > hat{eta}_j mid T_i < T_j, T_i < tau)$ up to horizon $tau$ | Consistent and invariant to independent censoring distribution via $hat{G}(t)^{-2}$ weights | Comparing survival models across external validation cohorts with differing follow-up lengths or heavy censoring |
| Time-Dependent AUROC ($text{AUC}(t)$) (Heagerty et al., 2000) | Incident/dynamic or cumulative/dynamic sensitivity vs. specificity at landmark time $t$ | Adjusted via Kaplan-Meier or IPCW estimators at fixed $t$ | Evaluating clinical decision thresholds at specific horizons (e.g., 1-year or 10-year cardiovascular risk) or under non-proportional hazards |
Frequently Asked Questions
What is Harrell’s C-index (concordance index) and how does it evaluate right-censored survival prediction models?
Harrell’s C-index (introduced by Frank E. Harrell Jr. et al., 1982, 1996) measures the rank-order discrimination of a time-to-event survival model in the presence of right-censoring. It estimates the probability $P(hat{eta}_i > hat{eta}_j mid T_i < T_j)$ that, for a randomly selected comparable pair of patients, the patient who experienced the event earlier ($T_i < T_j$) was assigned a higher predicted prognostic risk score ($hat{eta}_i > hat{eta}_j$) or shorter predicted survival time.
How do comparable pairs, tied risk scores, and censoring distribution sensitivity affect Harrell’s C-index?
Under right-censoring, a pair $(i, j)$ with observed follow-up times $tilde{T}_i < tilde{T}_j$ is comparable only if the subject with the shorter follow-up time experienced the event ($delta_i = 1$). Pairs where both subjects are censored or where the earlier subject is censored are excluded. Tied predicted risk scores on a comparable pair receive a weight of $0.5$ in the numerator. Because the set of usable pairs depends on follow-up length, Harrell’s C-index converges to a parameter that depends on the cohort’s censoring distribution and can be upwardly biased under heavy censoring.
When should survival analysis researchers use Uno’s C-statistic or time-dependent AUROC alongside Harrell’s C-index?
Researchers should report Uno’s IPCW C-statistic (Uno et al., 2011) when censoring is moderate to heavy, when comparing survival models across cohorts with different follow-up durations, or when a prespecified clinical horizon $tau$ is required. Time-dependent AUROC (Heagerty et al., 2000) should be reported alongside global concordance when evaluating discrimination at specific clinical landmark times such as 1-year or 5-year survival.
