Cluster Randomized Trials & the ICC

Not what you were looking for? If you want the ICC used to assess rater agreement or the reliability of a measurement instrument, see Intraclass Correlation Coefficient (ICC): Forms, Interpretation, and How to Report It. This page covers a different concept that happens to share the same abbreviation: the intracluster correlation coefficient, which describes how similar outcomes are among individuals who belong to the same randomised group — clinic, school, ward, village — in a cluster randomised trial. The two ICCs are computed differently, used for different purposes, and confusing them leads to real design and analysis errors, so if your question is about test-retest reliability or inter-rater agreement, that other page is the one you want.

What Is a Cluster Randomised Trial?

In a standard randomised controlled trial, individual participants are randomised to intervention or control. In a cluster randomised trial (CRT), whole groups — general practices, hospital wards, schools, villages, workplaces — are randomised instead, and every individual within a chosen cluster receives the same allocation. Researchers turn to cluster randomisation for a few recurring reasons:

The intervention is inherently delivered at the group level. A staff training programme, a clinical decision-support tool, or a policy change typically cannot be given to some patients in a clinic and withheld from others seen by the same trained staff on the same day.
Avoiding contamination. If individuals were randomised within the same clinic or classroom, the intervention would likely “leak” to control participants through shared staff, shared space, or peer interaction, biasing the comparison toward the null.
Practical and logistical constraints. It is often only feasible to roll an intervention out to an entire site, not to a subset of the people within it.
Ethical or political constraints. Denying an intervention to specific individuals within a community that has agreed to participate can be harder to justify or implement than allocating by whole community.

For the specific staggered-rollout variant of this design, see Stepped-Wedge and Cluster-Randomized Trial Designs, which covers when sponsors choose that design and the regulatory and ethical considerations involved. This page focuses on the statistical mechanics common to both parallel and stepped-wedge cluster designs: the intracluster correlation coefficient, the design effect it produces, and what both mean for sample size and analysis.

The Intracluster Correlation Coefficient (ICC)

Individuals within the same cluster are rarely statistically independent. People who attend the same clinic tend to be treated by the same staff under the same local protocols; people who live in the same village share environment, infrastructure, and social influence. Outcomes measured on two people from the same cluster therefore tend to be more similar to each other than outcomes measured on two people drawn from different clusters.

The intracluster correlation coefficient (ICC, often written rho or ICC) quantifies exactly how much of that similarity exists. Formally, it is the proportion of the total variance in the outcome that is attributable to variation between clusters, rather than variation within clusters:

ICC = variance between clusters / (variance between clusters + variance within clusters)

Equivalently, the ICC can be interpreted as the expected correlation between the outcomes of any two randomly chosen individuals from the same cluster. An ICC of 0 means clusters contribute nothing beyond individual-level noise — the data behave as if individuals had been randomised directly. An ICC of 1 means every individual within a cluster has an identical outcome and the cluster is effectively a single data point.

In practice, ICCs for health services and public health outcomes are usually small — commonly cited ranges fall between about 0.001 and 0.05 — because most of the variation in an individual health outcome is still driven by individual-level factors, not by which clinic or region someone happens to belong to. The consequence that surprises people new to cluster designs is that even an ICC this small can have a large effect on the trial once cluster sizes are moderate to large, which is the subject of the next section.

The Design Effect: Why a Tiny ICC Matters So Much

The statistical price of clustering is captured by the design effect (DEFF), sometimes called the variance inflation factor. For a cluster randomised trial with an average cluster size of m and an assumed intracluster correlation of ICC, the standard formula (Kish, 1965; Donner & Klar, 2000) is:

DEFF = 1 + (m − 1) × ICC

The design effect tells you how many times larger a cluster-randomised sample needs to be than an individually randomised sample would need to be, to achieve the same statistical power. A worked example shows why this matters even when the ICC looks negligible on paper:

Suppose an individually randomised trial would need 400 participants per arm to detect the effect of interest.
The planned cluster design has an average cluster size of m = 50 participants, and a conservative planning ICC of 0.02 — a small value, well within the typical health-services range.
DEFF = 1 + (50 − 1) × 0.02 = 1 + 49 × 0.02 = 1 + 0.98 = 1.98.
The required sample size roughly doubles: about 400 × 1.98 ≈ 792 participants per arm are now needed, not 400.

Notice what drove that near-doubling: not a large ICC, but a large average cluster size multiplying a small ICC. The same 0.02 ICC with clusters of only 10 people would produce DEFF = 1 + 9 × 0.02 = 1.18 — a modest 18% inflation. This is why cluster size, not just the ICC itself, is a central design lever: doubling the number of clusters while halving the average cluster size can reduce the design effect even though total sample size stays the same.

Sample Size: Clusters vs. Cluster Size

Once the design effect is estimated, planning proceeds by inflating the individually-randomised sample size calculation by DEFF, then deciding how to split that inflated total between the number of clusters and the number of participants per cluster. Two points matter more in practice than the arithmetic itself:

The number of clusters drives statistical power more than the total number of participants. Adding more participants within existing clusters has rapidly diminishing returns once cluster size is already moderate, because within-cluster observations are correlated and each additional individual in the same cluster adds less new information than a person in a brand-new cluster would. Adding more clusters, by contrast, adds genuinely independent units.
Very few clusters per arm is a real methodological problem, independent of total sample size. Trials with roughly fewer than 20–30 clusters per arm face unreliable estimation of the between-cluster variance itself, unstable standard errors, and inflated Type I error even when analysed correctly, which is why methodologists generally recommend small-sample corrections (see the analysis section below) whenever the number of clusters is limited, and recommend against proceeding with only a handful of clusters per arm at all where it can be avoided.

For the general mechanics of translating an effect size and power target into a required sample, see Power Analysis and Sample Size Calculation — the CRT-specific step is applying the design effect on top of that individual-level calculation.

Analysing a Cluster Randomised Trial

The central warning in cluster trial methodology: analysing cluster-randomised data as though participants had been individually randomised inflates Type I error — it treats correlated observations as independent, understates the true standard error, and produces p-values that are too small and confidence intervals that are too narrow, making a null result look statistically significant far more often than it should. This is one of the most common and most consequential analysis errors in the cluster-trial literature. Three approaches correctly account for clustering:

Mixed-effects (multilevel/hierarchical) models, which include a random effect for cluster, explicitly partitioning variance between and within clusters and estimating the ICC as part of the model.
Generalized estimating equations (GEE) with robust (“sandwich”) standard errors, which model the population-average effect while adjusting variance estimates for the within-cluster correlation structure. With a small number of clusters, GEE’s standard sandwich estimator is known to be biased downward, so small-sample corrections (for example, bias-corrected sandwich estimators or a t-distribution with cluster-based degrees of freedom rather than a normal approximation) are recommended once the cluster count is limited.
Cluster-level summary analysis, in which a single summary statistic (a mean or proportion) is calculated for each cluster and those cluster-level summaries — not individual-level data — become the unit of analysis. This is simple and robust but discards within-cluster information and is best suited to designs with a reasonably large, roughly equal number of clusters per arm.

Whichever approach is used, the analysis method should be pre-specified in the statistical analysis plan before unblinding, not chosen after seeing the data.

Getting an ICC for Planning Purposes

Because the ICC has such leverage over the required sample size, planning a cluster trial requires a defensible planning estimate before any data exist. Common sources, in rough order of preference, are:

Pilot or feasibility data from the same population and outcome, where a cluster-level analysis of pilot results can produce a study-specific ICC estimate.
Published ICC values from comparable outcomes, populations, and cluster types reported in prior cluster-randomised trials or dedicated ICC compendia in the methodological literature.
A conservative assumed value drawn from the typical range for the outcome type (for example, the low end of the 0.001–0.05 range commonly reported in health services research), used deliberately as an upper bound on uncertainty rather than a best guess.

Because no planning ICC is ever known with certainty, protocols should report a sensitivity analysis showing how the required sample size changes across a plausible range of ICC values, not a single point estimate treated as fact.

Related Designs

Stepped-wedge cluster randomised trials — every cluster starts in the control condition and crosses over to intervention at a staggered, randomly assigned time. See Stepped-Wedge and Cluster-Randomized Trial Designs for when this variant is chosen and its distinct regulatory and ethical considerations; it still requires an ICC-based design effect adjustment, typically a more complex one that also accounts for time.
Individually randomised group-treatment trials, where individuals are randomised individually but the intervention is still delivered in groups (for example, a group therapy session), producing clustering in the intervention arm only.
Matched-pair or stratified cluster designs, where clusters are paired or stratified on baseline characteristics before randomisation to improve balance, at some cost to the degrees of freedom available for analysis when the number of clusters is already small.

Reporting: The CONSORT Extension for Cluster Trials

The core CONSORT Statement was written for individually randomised, parallel-group trials and does not, on its own, adequately cover cluster-level randomisation. The CONSORT 2010 Statement: extension to cluster randomised trials (Campbell, Piaggio, Elbourne & Altman, BMJ 2012;345:e5661, updating an earlier 2004 extension) adds reporting requirements specific to this design, including:

The number of clusters randomised to each arm, not just the number of participants.
How clusters were identified, defined, and recruited, and the level at which randomisation occurred.
The method used to account for clustering in the sample size calculation, including the ICC value used for planning and, once available, the ICC actually observed in the trial data — reporting the observed ICC is one of the extension’s specific requirements, both to aid interpretation of the current trial and to build the evidence base future trials draw on for planning.
Confirmation that the statistical analysis accounted for clustering, and the method used to do so.
Details of any matching, stratification, or restriction applied at the cluster level.

See The EQUATOR Network: Which Reporting Guideline Does Your Study Need? for how this extension fits alongside other design-specific reporting guidelines.

Ethics: Who Consents When a Cluster Is Randomised?

Cluster randomisation raises a consent question that individually randomised trials do not: informed consent is normally sought from, and given by, the individual who will receive the intervention — but in a cluster trial, the allocation decision is made at the group level, before and independent of any individual’s consent to their own participation. Two distinct consent questions get bundled together and need to be kept separate:

Consent (or a comparable gatekeeper agreement) to the cluster’s participation in the trial — typically obtained from a legitimate representative of the cluster (a clinic director, school administrator, or village leader), sometimes called gatekeeper or cluster-level consent. This authorises the site to be part of the trial and, often, to have the intervention or data collection procedures implemented there.
Individual informed consent for a specific person’s own participation, data collection, or exposure to research procedures beyond what would have happened anyway as part of routine care or normal practice.

Whether individual consent is still required, and for what, depends heavily on what is actually being asked of participants. Where the intervention is a change to how a service or system is delivered to everyone regardless of trial participation (for example, a clinic-wide protocol change) and the only additional research activity is analysis of routinely collected, de-identified data, an ethics committee may determine that individual consent for the intervention itself is not required, though it may still be required for any additional data collection, and a waiver of informed consent would need to be formally justified and approved, not simply assumed. Where individuals are asked to do anything beyond that — complete additional surveys, undergo additional procedures, or have identifiable data collected for research purposes — individual informed consent is still generally required. Research ethics committees and IRBs evaluate this on a design-by-design basis; the CONSORT cluster extension and companion methodological guidance both recommend that trial protocols state explicitly what is being consented to, by whom, and at which level.

Frequently Asked Questions

Is the intracluster correlation coefficient the same as the intraclass correlation coefficient?

No, although they share the ICC abbreviation and both describe correlation arising from grouped data. The intraclass correlation coefficient typically measures agreement or reliability between raters or repeated measurements of the same subject. The intracluster correlation coefficient covered on this page measures how similar outcomes are among different individuals who belong to the same randomised cluster in a cluster trial. See Intraclass Correlation Coefficient (ICC): Forms, Interpretation, and How to Report It for the reliability sense.

What is a typical ICC value in a cluster randomised trial?

It varies substantially by outcome and cluster type, but health services and public health outcomes commonly fall in the roughly 0.001–0.05 range. Outcomes that are strongly shaped by local practice, environment, or shared exposure can have higher ICCs; individual clinical outcomes with mostly patient-level determinants tend to sit at the low end.

Why does a small ICC still inflate the sample size so much?

Because the design effect multiplies the ICC by the average cluster size minus one. A small ICC combined with a large cluster size can still produce a design effect well above 1, as the worked example above shows — DEFF depends on both the ICC and how many people are in each cluster, not on the ICC alone.

What happens if I analyse a cluster trial without accounting for clustering?

Treating clustered data as independent observations understates the true standard error, producing artificially small p-values and artificially narrow confidence intervals. This inflates the Type I error rate — the trial will report statistically significant findings more often than the true false-positive rate justifies.

How many clusters do I need per arm?

There is no universal minimum, but methodologists generally caution against fewer than roughly 20–30 clusters per arm, because estimating the between-cluster variance reliably, and applying standard asymptotic statistical methods, both become difficult with very few clusters. Below that range, small-sample corrections to the analysis become important rather than optional.

Cluster Randomised Trials and the Intracluster Correlation Coefficient (ICC)

Ask about Cluster Randomised Trials and the Intracluster Correlation Coefficient (ICC)