A confidence interval is a range of values, calculated from sample data, that is designed to contain the true value of an unknown population parameter with a stated level of confidence. A 95% confidence interval is produced by a procedure that, over many repeated samples, would capture the true parameter in about 95% of those intervals. It conveys both an estimate of the parameter and the uncertainty around that estimate, expressed as the width of the interval.
The correct interpretation
The confidence level is a property of the long-run procedure, not of any single interval. Once a specific interval has been calculated, the true parameter either lies inside it or it does not; there is no probability left to assign. It is therefore incorrect to say there is a 95% probability that the parameter lies within a particular calculated interval. The accurate statement is that if the study were repeated many times and an interval computed each time, about 95% of those intervals would contain the true value. This frequentist interpretation is subtle but important, and misstating it is one of the most common errors in applied statistics.
| Statement | Correct? |
|---|---|
| 95% of intervals from repeated samples contain the true parameter | Yes |
| There is a 95% probability this specific interval contains the parameter | No |
| The interval shows a range of plausible values for the parameter | Yes, a reasonable informal reading |
| 95% of the data fall within the interval | No, that confuses it with a data range |
Width, precision and sample size
The width of a confidence interval reflects the precision of the estimate. A narrow interval indicates a precise estimate; a wide one signals substantial uncertainty. Width depends chiefly on the variability in the data and on the sample size. Larger samples generally produce narrower intervals because the standard error shrinks as the sample grows. Raising the confidence level, say from 95% to 99%, widens the interval, because demanding greater confidence requires admitting a broader range of plausible values.
Relationship to statistical significance
Confidence intervals and significance tests are closely linked. For a comparison such as a difference between two means, if a 95% confidence interval for the difference excludes zero, the result is statistically significant at the 0.05 level; if the interval includes zero, it is not. The interval therefore conveys the same information as a p-value while adding crucial context: the estimated size of the effect and the range of values compatible with the data.
Why intervals are often more informative
Reporting a confidence interval communicates more than a bare p-value because it shows magnitude and precision together. A result may be statistically significant yet have an interval spanning only trivial effects, or be non-significant yet have an interval wide enough to include important ones. Many methodologists, including the authors of the American Statistical Association’s 2016 guidance on p-values, encourage reporting estimates with intervals rather than relying on significance thresholds alone. This practice supports clearer interpretation and stronger reproducibility, themes tracked in our reproducibility category. The underlying methods belong to the broader discipline of statistics, and consistent reporting terminology is documented in the CASRAI dictionary.
Frequently asked questions
What does a 95% confidence interval really mean?
It means that the method used to build the interval would capture the true population value in about 95% of repeated samples. It is not a 95% probability that the true value lies in one particular calculated interval.
Does a narrower interval always mean a better study?
A narrow interval indicates a precise estimate, usually from a large or low-variability sample, but precision is not the same as validity. A precise estimate from a biased study can still be wrong. Width describes uncertainty from sampling, not freedom from bias.
Should I report a confidence interval or a p-value?
Where possible, report an effect estimate with its confidence interval, optionally alongside a p-value. The interval shows both the size and the precision of the effect, which is generally more informative for readers. See the CASRAI author guidance for reporting recommendations.
