Skip to main content
v2026.11,986 entries · CC-BY 4.0

Intraclass Correlation Coefficient (ICC): Forms, Interpretation, and How to Report It

What the ICC measures, the ICC(1,1)/(2,1)/(3,1) forms and when to use each, Koo & Li interpretation benchmarks, ICC vs. kappa/Pearson r/Bland-Altman/Cronbach’s alpha, sample size, and how to report results.

Written and maintained by CASRAI Editorial Board

Published · Last updated

On this page

The intraclass correlation coefficient (ICC) is a reliability statistic that gives the share of total score variance due to real differences between subjects. It runs from 0 to 1; under Koo & Li (2016), below 0.50 is poor, 0.50 to 0.75 moderate, 0.75 to 0.90 good and above 0.90 excellent, judged on the 95% confidence interval.

On this page: what the intraclass correlation coefficient (ICC) measures, why the specific form you choose (ICC(1,1), ICC(2,1), ICC(3,1), and their average-measures “k” counterparts) is the single most common source of error in published reliability research, how to interpret the result, how ICC compares to Cohen’s/Fleiss’ kappa, Pearson’s r, Bland-Altman analysis, and Cronbach’s alpha, sample-size guidance for reliability studies, how to report results, and how to compute ICC in R, Python, and SPSS.

What ICC Measures

The intraclass correlation coefficient (ICC) is a reliability statistic for continuous measurements. It expresses the proportion of the total variance in a set of scores that is due to true differences between subjects, as opposed to variance introduced by measurement error, rater disagreement, or occasion-to-occasion inconsistency:

ICC = variance(between subjects) / [variance(between subjects) + variance(error)]

An ICC close to 1.0 means that most of the variability in the data reflects genuine differences between the people or specimens being measured, and very little reflects noise from the measurement process itself. An ICC close to 0 means the opposite: raters, instruments, or occasions disagree with each other by roughly as much as subjects differ from one another, so the measurement is not distinguishing subjects reliably.

ICC is used for two related but distinct purposes, both of which require continuous or ordinal-treated-as-continuous data:

  • Inter-rater reliability — do two or more raters/observers/instruments produce consistent scores for the same subjects?
  • Test-retest reliability — does the same rater/instrument produce consistent scores for the same subjects across repeated occasions? Here the two occasions take the place of raters, and choosing and justifying the retest interval becomes the design decision the coefficient itself cannot recover from.

See CASRAI’s comparison of test-retest vs. inter-rater reliability for how those two reliability questions differ conceptually — ICC is the statistic commonly used to quantify either one when the underlying data are continuous.

A note on the search term: ICC is very frequently typed and searched as “inter class correlation” or “interclass correlation.” This is a common misspelling of intraclass correlation — the “intra” (within) is doing real conceptual work here, since the statistic is built from a within-subject/between-subject variance decomposition, not a correlation “between classes” of anything. If you found this page searching either term, you’re in the right place.

What Is a Good ICC Score?

An ICC score runs from 0 to 1 (negative estimates can occur and signal no usable reliability). Under the widely used Koo & Li (2016) guideline, below 0.50 is poor, 0.50 to 0.75 moderate, 0.75 to 0.90 good, and above 0.90 excellent. Judge the 95% confidence interval, not the point estimate alone, and always name the ICC form used.

If you searched “ICC stats” or “ICC statistics”, this is the intraclass correlation coefficient used to quantify reliability and agreement; the same abbreviation also appears in one other statistical setting and in a non-statistical one, both covered below.

Why the Specific ICC “Form” Is the Single Biggest Source of Error

Unlike Pearson’s r, “the ICC” is not one statistic — it is a family of at least six (and, depending on convention, up to ten) distinct coefficients that can produce meaningfully different numbers from the exact same dataset. Reporting “the ICC was 0.82” without specifying which form was used is functionally incomplete, and choosing the wrong form for a given study design is the most frequently cited methodological error in published reliability research. The two conventions researchers cite are:

  • Shrout & Fleiss (1979) — the original notation, ICC(model, type), e.g. ICC(2,1), ICC(3,k).
  • McGraw & Wong (1996) — a later, more explicit notation, e.g. ICC(A,1) for absolute agreement or ICC(C,k) for consistency, that makes the agreement-vs-consistency choice part of the label itself rather than leaving it implicit in the model.

Three independent decisions combine to define a specific ICC form. Get any one wrong and the number you report answers a different question than the one your study design actually asked.

Decision 1 — Model: One-Way or Two-Way?

  • One-way random effects (Shrout & Fleiss “Case 1”) — each subject is rated by a different, randomly selected set of raters. There is no way to separate rater variance from residual error, because raters aren’t crossed with subjects. Produces ICC(1,1) or ICC(1,k).
  • Two-way random effects (Case 2) — the same set of raters rates every subject, and those raters are themselves treated as a random sample from a larger population of possible raters you want to generalize beyond. Produces ICC(2,1) or ICC(2,k).
  • Two-way mixed effects (Case 3) — the same set of raters rates every subject, but those specific raters are the only ones of interest — you are not trying to generalize to other, hypothetical raters. Produces ICC(3,1) or ICC(3,k).

The practical test: if your raters are a fixed panel you’re stuck with for this study (the two study clinicians, this one instrument), use the two-way mixed model (Case 3). If your raters are meant to represent a broader pool of similarly-trained raters you want your conclusion to generalize to, use the two-way random model (Case 2).

Decision 2 — Type: Single Measures or Average Measures?

This reflects what will actually be used in practice. If a single rater’s single score will be used going forward (a clinician’s one assessment, one lab technician’s one reading), report the single-measures ICC (the “,1” forms: ICC(2,1), ICC(3,1)). If scores from multiple raters will be averaged before use (a consensus score from a 3-rater panel), report the average-measures ICC (the “,k” forms: ICC(2,k), ICC(3,k)) — averaging multiple raters’ scores mechanically increases reliability, so the “k” forms are always numerically higher than their single-measures counterparts for the same data, and the two answer genuinely different practical questions.

Decision 3 — Definition: Consistency or Absolute Agreement?

  • Absolute agreement asks: do raters assign the same numeric value? A systematic offset between raters (one rater consistently scoring two points higher than another) counts against reliability.
  • Consistency asks: do raters rank subjects the same way, even if there’s a systematic offset between them? A rater who is consistently two points higher than another, but tracks their relative ordering of subjects perfectly, still scores well on the consistency definition.

Absolute agreement is almost always the more defensible choice when scores from different raters or occasions will be used interchangeably in practice (e.g., a clinical score that different clinicians will read at different visits) — a systematic offset is a real problem in that use case. Consistency is defensible when only relative ranking matters, or when a known, correctable offset (e.g., a calibration constant between two instruments) is not itself the thing being tested.

Decision Table: Which Form for Which Design

Your design Model Type Definition Form (Shrout & Fleiss)
Each subject rated by a different random rater; single rating used One-way random Single No rater term (agreement and consistency are not separable) ICC(1,1)
Same raters rate all subjects; raters are a random sample you generalize beyond; single rating used, offsets matter Two-way random Single Absolute agreement ICC(2,1)
Same raters rate all subjects; those specific raters are the only ones of interest; single rating used, only ranking matters Two-way mixed Single Consistency ICC(3,1)
Same raters rate all subjects; those specific raters are the only ones of interest; single rating used, offsets matter Two-way mixed Single Absolute agreement ICC(A,1) in McGraw & Wong notation (the form Koo & Li recommend for test-retest and intra-rater studies)
Any of the above two-way designs, but a k-rater average (not a single rating) will be used going forward Two-way random or mixed, as above Average (k) As above ICC(2,k) or ICC(3,k)

Koo & Li give a selection guideline rather than a default: for test-retest and intra-rater reliability they recommend a two-way mixed-effects model with absolute agreement. Software defaults differ and do not always match that design logic (the R irr::icc() defaults, for example, are one-way, consistency, single measures), so set model, type and unit explicitly and state them in the methods section. The data table below this guide lines up the six commonly reported forms with their Shrout & Fleiss and McGraw & Wong labels and with the arguments of irr::icc() and pingouin.intraclass_corr().

Interpreting ICC Values

The most widely cited interpretation benchmarks come from Koo & Li’s 2016 guideline in the Journal of Chiropractic Medicine:

  • Less than 0.50 — poor reliability
  • 0.50 to 0.75 — moderate reliability
  • 0.75 to 0.90 — good reliability
  • Greater than 0.90 — excellent reliability

Two caveats matter more than the thresholds themselves. First, these are conventions adopted for convenience, not statistical laws — a different field or a higher-stakes application (e.g., a diagnostic threshold that determines treatment) may reasonably demand a stricter bar than a general convention supplies. Second, and more important: Koo & Li’s own guideline recommends basing the reliability judgment on the 95% confidence interval of the ICC estimate, not the point estimate alone. A point estimate of 0.82 sounds solidly “good,” but if that estimate came from a small study and its 95% CI actually runs from 0.55 to 0.95, the honest read spans moderate through excellent — reporting only “0.82, good reliability” overstates the precision the data actually support.

ICC Score Meaning at a Glance

ICC value Reliability label (Koo & Li, 2016) What it implies in practice
Below 0.50 Poor Error variance is about as large as, or larger than, true between-subject variance; individual scores are not trustworthy.
0.50 to 0.75 Moderate Usable for group-level comparisons, rarely for individual decisions.
0.75 to 0.90 Good Acceptable for most research uses; check the lower confidence limit.
Above 0.90 Excellent Suited to decisions about individuals, provided the lower confidence limit also clears the bar.

The labels are the guideline’s published bands (also the default in the R effectsize::interpret_icc() function); the “implies in practice” column is interpretive advice rather than part of the guideline. Koo & Li also advise obtaining at least 30 heterogeneous samples and, where possible, at least 3 raters. When one rater is measured twice, the same coefficient applies; see intra-rater reliability for a worked example.

Intracluster Correlation: The Other ICC in Statistics

In cluster-randomised trials, ICC usually stands for the intracluster correlation coefficient. It uses the same variance-ratio idea, but between clusters (clinics, schools, practices) rather than between subjects, and it is used to inflate sample size rather than to judge a measurement. The design effect is 1 + (m − 1) × ICC, where m is the average cluster size. Kerry and Bland (BMJ, 1998) give an example: an ICC of 0.019 with 50 referrals per practice yields a design effect of about 1.93, so the trial needs roughly twice as many participants as one randomised by individual. Even a small intracluster correlation matters when clusters are large. The cluster randomised trials guide covers the design effect in depth, and the power analysis guide covers sample size calculation more broadly.

The third common meaning is non-statistical: searches for “ICC scores” frequently refer to the International Cricket Council’s match scores and rankings, which is a different subject from this guide.

ICC vs. Alternatives: Kappa, Pearson r, Bland-Altman, and Cronbach’s Alpha

ICC is frequently confused with, or substituted for, several other statistics that answer related but genuinely different questions:

  • Cohen’s kappa / Fleiss’ kappa — the categorical-data equivalents of ICC. Use kappa when raters assign subjects to discrete categories (a diagnosis, a pass/fail rating); use ICC when raters assign a continuous or ordinal score. Applying ICC to nominal categories, or kappa to continuous scores, is a category error. CASRAI’s guide to choosing the right inter-rater reliability coefficient works through the full selection decision across kappa, weighted kappa, Fleiss’ kappa, Krippendorff’s alpha and Gwet’s AC1, including the prevalence paradox that can drive kappa near zero at 95% agreement.
  • Pearson’s correlation coefficient (r) — r is not a reliability or agreement statistic, even though it is routinely misused as one. Two raters whose scores are perfectly correlated (r = 1.0) but who are systematically offset by a constant (one always scoring five points higher) would show perfect Pearson correlation while genuinely disagreeing on every subject’s actual value. ICC’s absolute-agreement forms are specifically built to catch that kind of systematic bias; r structurally cannot.
  • Bland-Altman analysis — used to assess agreement between two measurement methods (e.g., a new device against a reference standard), producing a bias estimate and limits of agreement rather than a single coefficient. Bland-Altman and ICC are complementary, not competing: ICC gives a single reliability number and confidence interval, Bland-Altman visualizes where and how two methods disagree across the measurement range. Many methodologists recommend reporting both for a method-comparison study.
  • Cronbach’s alpha — measures internal consistency among multiple items on a scale (do the items on a questionnaire measure the same underlying construct?), not agreement between raters or occasions. See CASRAI’s guide to Cronbach’s alpha for the full treatment. Alpha and the ICC are both variance-based reliability coefficients, but they answer different design questions: alpha asks whether items hang together, ICC asks whether raters or occasions agree on the same subjects.

All four sit within the broader question of whether an instrument measures its intended construct consistently and correctly — see CASRAI’s guide to types of validity in research for how reliability (addressed by ICC, kappa, and alpha) relates to, but is distinct from, validity.

Sample Size for Reliability Studies

ICC-based reliability studies need adequate sample size on two separate dimensions, and underpowering either one produces an unstable estimate even if the other looks generous:

  • Number of subjects — too few subjects, and the between-subject variance term in the ICC formula is itself poorly estimated, producing wide, unstable confidence intervals regardless of how many raters you use. Formal sample-size calculations for reliability studies (based on the expected ICC, the number of raters, and the desired CI width) are available in dedicated reliability-study sample-size methods and software — a generic post-hoc power analysis built for a group-comparison test does not directly transfer to a reliability design and should not be substituted for one.
  • Number of raters (k) — more raters generally narrows the CI and, for average-measures forms, mechanically increases the ICC itself, but only if the additional raters are genuinely representative of the rater population you intend to generalize to (for the two-way random model) rather than added simply to inflate the number.

Koo & Li advise at least 30 heterogeneous samples and, wherever possible, at least 3 raters. Treat 30 as a floor to plan above rather than a target, and prefer a formal calculation based on the expected ICC, the number of raters and the desired confidence-interval width where one is feasible. Their guideline also notes that a low ICC can reflect limited variability among the subjects sampled, too few subjects or too few raters, not only poor measurement.

How to Report ICC Results

The Guidelines for Reporting Reliability and Agreement Studies (GRRAS), published by Kottner and colleagues in the Journal of Clinical Epidemiology (2011), is the most widely cited reporting checklist specific to reliability and agreement studies, alongside the general COSMIN initiative’s guidance on evaluating outcome measurement instruments. Applied specifically to ICC, Koo & Li’s guideline recommends every report state, at minimum:

  1. The model used (one-way random, two-way random, or two-way mixed).
  2. The type (single measures or average of k measures), and the value of k if applicable.
  3. The definition (consistency or absolute agreement).
  4. The ICC point estimate and its 95% confidence interval, not the point estimate alone.
  5. The software (and version) used to compute it, since default forms and CI-computation methods differ across packages.

A minimally complete reporting sentence looks like: “Inter-rater reliability was assessed using a two-way random-effects model with absolute agreement, single-rater type — ICC(2,1) = 0.84, 95% CI [0.71, 0.92] — calculated in R (irr package, v0.84.1).”

An ICC on its own does not tell a clinician how much change in an individual score is more than noise. The reliability coefficient can be converted into a standard error of measurement and then into a minimal detectable change threshold; because that conversion starts from the ICC, the form you chose (and whether it counts systematic rater or occasion bias) carries through to it.

Calculating ICC: R, Python, and SPSS

  • R, irr package — icc(ratings, model = c("oneway", "twoway"), type = c("consistency", "agreement"), unit = c("single", "average"), r0 = 0, conf.level = 0.95). The defaults are model = "oneway", type = "consistency" and unit = "single", and type = "agreement" is not available for the one-way model. The function tests the null hypothesis ICC = r0 against the one-sided alternative ICC > r0, and it deletes subjects with missing ratings listwise. Always set all three arguments yourself.
  • R, psych package — psych::ICC() returns the six Shrout & Fleiss forms in one call so you can compare them side by side.
  • Python, pingouin — pingouin.intraclass_corr() returns six rows, labelled in the Type column as ICC(1,1), ICC(A,1), ICC(C,1) and the average-measures versions ICC(1,k), ICC(A,k) and ICC(C,k). Columns are Type, ICC, F, pval, df1, df2 and CI95. It is ANOVA-based and expects complete, balanced data: the nan_policy argument either raises an error on missing values or omits them. Its results are documented as validated against R’s psych package.
  • SPSS — Analyze → Scale → Reliability Analysis → Statistics → check “Intraclass correlation coefficient,” then choose the Model (One-Way Random, Two-Way Random, or Two-Way Mixed) and Type (Consistency or Absolute Agreement) from the dropdowns that appear.

Whichever tool you use, confirm the model, type and definition it actually computed match the design decision in the decision table above before reporting the number.

Worked Example: One-Way ICC From Mean Squares

For the one-way model, Liljequist and colleagues (2019) give ICC(1) = (MSBS − MSWS) / (MSBS + (k − 1) × MSWS), where MSBS is the between-subjects mean square, MSWS the within-subjects mean square and k the number of ratings per subject. Suppose an illustrative ANOVA of two ratings per subject (k = 2) gives MSBS = 24 and MSWS = 3. These numbers are invented to show the arithmetic and come from no study.

  1. Numerator: 24 − 3 = 21.
  2. Denominator: 24 + (2 − 1) × 3 = 27.
  3. ICC(1) = 21 / 27 = 0.778, which falls in Koo & Li’s “good” band (0.75 to 0.90). The interval still decides the label: if the 95% confidence interval ran from 0.45 to 0.92, the defensible reading would span poor-to-moderate through excellent.

If MSWS were 12 instead, the same calculation gives (24 − 12) / (24 + 12) = 0.333, poor. Only the within-subject noise changed.

Check for Bias Before Choosing the Form

Liljequist and colleagues recommend comparing forms as a diagnostic. ICC(1) should not be used when there is non-negligible systematic bias between raters or occasions, because the one-way model has no rater term to absorb it. When ICC(C,1) (consistency) is clearly larger than ICC(A,1) (absolute agreement) on the same data, that gap signals bias between raters or occasions. A significant F-test for the rater or occasion term (F = MSBM / MSE, where MSBM is the between-measurements mean square) points the same way; in pingouin output this is the F and pval of the relevant row. When bias is present, the authors advise reporting both ICC(A,1) and ICC(C,1) with their confidence intervals, so readers can see how much of the unreliability is systematic offset and how much is random noise.

Why a Low ICC Is Not Always Poor Measurement

The ICC is a ratio of between-subject variance to total variance, so it depends on the sample as well as the instrument. Koo & Li note that a low ICC can reflect limited variability among the subjects sampled, too few subjects or too few raters. A reliable scale tested only on subjects who all score alike will return a low ICC because there is little true between-subject variance to detect. Liljequist and colleagues add that a negative sample ICC(1) is simply a bad (unfortunate) estimate: it arises by chance, especially with small samples, not from a coefficient that can truly be below zero. Before declaring an instrument unreliable, check the spread of subjects, the sample size and the rater count.

ICC vs. Pearson Correlation

The ICC correlates measurements within a class (the same subject measured repeatedly or by several raters), as in Fisher’s original formulation, whereas Pearson’s r is an interclass statistic between two different variables. That is where the name comes from, and why “inter class correlation” is a misspelling of the reliability statistic rather than a second method. The detailed comparison appears in the alternatives section below.

Frequently Asked Questions

Is “inter-class correlation” the same as “intraclass correlation”?

Yes — “inter class correlation” and “interclass correlation” are common misspellings/mishearings of intraclass correlation coefficient. There is no separate statistic called “inter-class correlation” in standard use; if you encountered that phrase, it refers to ICC.

Can ICC be negative?

A sample estimate can be negative. Liljequist and colleagues describe a negative sample ICC(1) as simply a bad (unfortunate) estimate: it arises by chance, particularly with few subjects, and signals no usable reliability. Check the sample size, the spread of the subjects and whether the form matches the design before drawing conclusions.

Do I need to run all six (or ten) ICC forms?

No — report the one form that matches your actual study design (see the decision table above). Running and reporting several forms and highlighting whichever looks best is a form of selective reporting; decide the model, type, and definition from your design before looking at the results.

What’s a good number of raters and subjects for an ICC study?

Koo & Li recommend at least 30 heterogeneous samples and, whenever possible, at least 3 raters.

Is an ICC of 0.7 acceptable?

Under the Koo & Li bands, 0.70 is moderate (0.50 to 0.75). Judge the 95% confidence interval, not the point estimate; whether moderate is acceptable depends on the stakes of the decision.

Why do ICC(2,1) and ICC(3,1) differ?

In McGraw & Wong labels these are ICC(A,1) and ICC(C,1). Absolute agreement counts systematic offset between raters as error, consistency ignores it, so ICC(C,1) is clearly larger when bias is present (Liljequist et al., 2019).

Which ICC applies when different raters score different subjects?

That is the one-way random design, ICC(1,1) or ICC(1,k). The one-way form should not be used when raters introduce non-negligible bias.

What does a significant F in pingouin output mean?

It tests the rater or occasion term and points to systematic bias between raters or occasions, which is why ICC(A,1) and ICC(C,1) may then be reported together.

See also: CASRAI’s guides to descriptive statistics for the variance and standard-deviation concepts ICC is built from, and questionnaire design for instrument-development context where ICC and Cronbach’s alpha are both routinely reported together.

Sources (accessed 2026-11-16)

Data: ICC forms crosswalk: Shrout-Fleiss and McGraw-Wong labels, design conditions, and R irr / Python pingouin equivalents

Compares six commonly reported intraclass correlation forms by model, measurement type, definition, notation, and the arguments or output labels used in R irr::icc() and Python pingouin.intraclass_corr(). Compiled from Koo & Li (2016), Liljequist et al. (2019), the irr documentation and the pingouin source docstring.

ICC forms crosswalk: Shrout-Fleiss and McGraw-Wong labels, design conditions, and R irr / Python pingouin equivalents
ICC form (Shrout-Fleiss)McGraw-Wong / pingouin labelModelTypeDefinitionR irr::icc() argumentspingouin Type value
ICC(1,1)ICC(1)One-way randomSingle raterConsistency only (no rater term)model="oneway", type="consistency", unit="single"ICC(1,1)
ICC(1,k)ICC(k)One-way randomMean of kConsistency only (no rater term)model="oneway", type="consistency", unit="average"ICC(1,k)
ICC(2,1)ICC(A,1)Two-way randomSingle raterAbsolute agreementmodel="twoway", type="agreement", unit="single"ICC(A,1)
ICC(2,k)ICC(A,k)Two-way randomMean of kAbsolute agreementmodel="twoway", type="agreement", unit="average"ICC(A,k)
ICC(3,1)ICC(C,1)Two-way mixedSingle raterConsistencymodel="twoway", type="consistency", unit="single"ICC(C,1)
ICC(3,k)ICC(C,k)Two-way mixedMean of kConsistencymodel="twoway", type="consistency", unit="average"ICC(C,k)

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about Intraclass Correlation Coefficient (ICC): Forms, Interpretation, and How to Report It

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →