Skip to main content
v2026.11,986 entries · CC-BY 4.0

Hypothesis Testing and Confidence Intervals: How They Connect

A two-sided test at level alpha rejects when the matching confidence interval excludes the null value. Worked t-test example with R, Python and Excel.

Written and maintained by CASRAI Editorial Board

Published

On this page

A confidence interval and a hypothesis test answer the same yes-or-no question from the same data. For a two-sided test at significance level α, you reject the null hypothesis exactly when the matching (1 − α) confidence interval does not contain the null value: a 95% interval goes with α = 0.05, a 99% interval with α = 0.01. The interval does more than the test, because it also shows the range of effect sizes the data are compatible with and how precisely they were measured.

Greenland and colleagues describe the link as moving “from tests to estimates”: the effect sizes whose test gives P > 0.05 form a range that corresponds to a 95% confidence interval, and values outside it correspond to P ≤ 0.05. So a confidence interval can be read as the set of null values a test at that level would not reject. The same relationship is what lets you use confidence intervals and hypothesis testing interchangeably for the two-sided decision.

This page is about that connection. For the mechanics of each tool on its own, see confidence intervals explained, the test statistic in hypothesis testing and the comparison p-value vs confidence interval.

Confidence interval in hypothesis testing: the decision rule

Bassett’s course text states the procedure directly: to test at level α, form the confidence interval at the matching level and check whether it contains the null value. If a 95% interval excludes the null value, reject at α = 0.05; if it contains the null value, do not reject. The steps are:

  1. State H0 (for example μ = 52) and H1 (μ ≠ 52), and choose α before looking at the data.
  2. Compute the estimate and its standard error.
  3. Build the (1 − α) interval: estimate ± critical value × standard error.
  4. Reject H0 if the null value lies outside the interval; otherwise fail to reject it.

NIST’s engineering handbook gives the other half of the equivalence: with α = 0.05, the null hypothesis is rejected 5% of the time when it is true, and the test rejects when the test statistic exceeds the critical value, which is the same decision as a p-value below α. The critical value in step 3 is that same critical value, which is why the two procedures cannot disagree for a two-sided test of a mean.

Formula with every symbol explained

For a one-sample t-test of a mean, the test statistic and the interval are built from the same ingredients:

  • Test statistic: t = (x̄ − μ0) / (s / √n)
  • Confidence interval: x̄ ± tcrit × (s / √n)

Here x̄ is the sample mean, μ0 is the null value, s is the sample standard deviation, n is the sample size, s/√n is the standard error, and tcrit is the two-sided critical value of the t distribution with n − 1 degrees of freedom at level α. Rejecting when |t| > tcrit is algebraically the same as μ0 falling outside x̄ ± tcrit × s/√n.

Worked example: one decision, three ways

The ten values below are illustrative numbers invented for this page, not a real study: 52.1, 54.3, 50.8, 55.0, 53.2, 51.9, 56.4, 52.7, 54.8, 53.0. Test H0: μ = 52 against H1: μ ≠ 52 at α = 0.05.

  • n = 10, sum = 534.2, so x̄ = 53.42.
  • Sample standard deviation s = 1.6904; standard error = 1.6904 / √10 = 0.5345; degrees of freedom = 9.
  • Test statistic: t = (53.42 − 52) / 0.5345 = 2.656.
Method Calculation Decision at α = 0.05
Critical value tcrit (9 df, two-sided) = 2.262; |2.656| > 2.262 Reject H0
p-value two-sided p = 0.026, which is below 0.05 Reject H0
95% confidence interval 53.42 ± 2.262 × 0.5345 = 53.42 ± 1.21 = [52.21, 54.63]; 52 is outside Reject H0

All three routes reject. The interval also tells you something the other two do not: the data are compatible with a true mean anywhere from about 52.2 to 54.6, so the shift from 52 is real in this sample but could be as small as 0.2 units.

Matching alpha and confidence level

The interval must be built at the level that matches the test. Using the same data, the table below (also offered as a downloadable data table at the end of the page) repeats the test at three levels. At α = 0.01 the 99% interval, [51.68, 55.16], now contains 52 and the test does not reject, which agrees with the p-value of 0.026 being above 0.01 and with 2.656 being below the 99% critical value of 3.250.

Bassett’s toad-girth example shows the same flip with real data: for a right-sided test against 94, the 95% lower bound is 96.84 (excludes 94) while the 99% lower bound is 93.93 (includes 94). The decision changed because the level changed, not because the data did. Choose α first, then the interval level, never the reverse.

See what alpha means in statistics for how to choose the level.

One-sided tests and one-sided intervals

The two-sided rule above does not carry over unchanged. Bassett’s text matches each one-sided alternative to a one-sided interval: a “less” alternative uses a left one-sided interval, and a “greater” alternative uses a right one-sided interval. In his 5-heads-in-15-flips example, the “less” interval for the proportion runs from 0 to 0.5765, contains 0.5, and so gives no evidence that the proportion is below 0.5. For our data, a right-sided test of μ > 52 at α = 0.05 has lower bound 53.42 − 1.833 × 0.5345 = 52.44, which is above 52, so it rejects. Checking a two-sided interval against a one-sided hypothesis uses the wrong tail. Read more in two-tailed hypothesis testing.

What the interval adds that the test does not

A test returns one bit of information at your chosen α. An interval returns every null value that test would have kept. Bassett’s coin examples show how that matters: 9,850 heads in 20,000 flips gives a 95% interval of 0.4855 to 0.4995, which excludes 0.5 but only because the sample is huge and the departure tiny. Four heads in 10 flips gives 0.137 to 0.726, which contains 0.5 but is so wide that the data say little either way. A bare “significant” or “not significant” hides both situations; the interval width shows which one you are in.

Assumptions and common mistakes

  • Assuming equivalence beyond its limits. The duality is exact for a two-sided test and its matching interval. It does not make an interval a test of every other kind of hypothesis.
  • Reading exclusion as refutation. Greenland et al. list “an effect size outside the 95% confidence interval has been refuted (or excluded) by the data” as a misinterpretation; stronger conditions are needed before values outside an interval can be called excluded.
  • Treating everything inside the interval as equally supported. Values inside the interval have different P values; those nearer the estimate have higher ones.
  • Equating precision with significance. Greenland et al. note that precision is the interval’s width, not a matter of whether it includes the null or any other value.
  • Mismatched levels. Using a 95% interval to judge a test run at α = 0.01 (or the reverse) gives wrong decisions, as the toad example shows.
  • Ignoring the test’s own assumptions. The t-test and its interval share assumptions such as independent observations and roughly normal data or a large enough sample.

How to report it

Report the estimate, the interval and the test together so readers can use either. For the example above, one common APA-style line is: The sample mean (M = 53.42, SD = 1.69) differed from the reference value of 52, t(9) = 2.66, p = .026, 95% CI [52.21, 54.63]. State the level of the interval and the α used for the decision. More patterns are in how to report confidence intervals.

How to calculate it in Excel, R or Python

Excel (data in A1:A10, null value 52):

t statistic:   =(AVERAGE(A1:A10)-52)/(STDEV.S(A1:A10)/SQRT(10))
p-value:       =T.DIST.2T(ABS(t_cell),9)
CI half-width: =CONFIDENCE.T(0.05,STDEV.S(A1:A10),10)
critical t:    =T.INV.2T(0.05,9)

R (the output reports t, df, p and the 95% interval together):

x = c(52.1,54.3,50.8,55.0,53.2,51.9,56.4,52.7,54.8,53.0)
t.test(x, mu = 52, conf.level = 0.95)
t.test(x, mu = 52, alternative = "greater")  # one-sided, lower bound

Python (SciPy):

from scipy import stats
x = [52.1,54.3,50.8,55.0,53.2,51.9,56.4,52.7,54.8,53.0]
res = stats.ttest_1samp(x, popmean=52)
print(res.statistic, res.pvalue)
print(res.confidence_interval(confidence_level=0.95))

The R call reports the test and the interval in one output.

Frequently asked questions

Can I use a confidence interval to do a hypothesis test?

Yes, for a two-sided test. Build the (1 − α) interval and reject the null hypothesis if the null value is outside it.

What confidence level matches α = 0.05?

A 95% interval. In general, the matching level is 1 − α, so α = 0.01 pairs with 99%.

If the confidence interval contains zero, is the result not significant?

When the null value is zero (for example a difference of means) and the interval matches the test level, a zero inside the interval means the test does not reject. That is not evidence the effect is zero; the interval may simply be wide.

Is a confidence interval better than a p-value?

They answer different questions. The p-value measures compatibility of the data with one null value; the interval shows all compatible values and the precision. Reporting both is common practice.

Does the interval work for one-sided tests?

Only with a one-sided interval: a left interval for a “less” alternative and a right interval for a “greater” one.

Sources

  • Greenland S, et al. (2016). Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology. pmc.ncbi.nlm.nih.gov/articles/PMC4877414 (read 2026-11-15).
  • Bassett R. Introduction to Statistics and Data Science, ch. 16, Confidence Intervals and Hypothesis Testing, Naval Postgraduate School. faculty.nps.edu/rbassett (read 2026-11-15).
  • NIST/SEMATECH e-Handbook of Statistical Methods, section 7.1.3. itl.nist.gov/div898/handbook/prc/section1/prc131.htm (read 2026-11-15).
  • Worked-example arithmetic computed for this page from the illustrative data shown.

Data: One-sample t-test decision at three significance levels with matching confidence intervals

Illustrative worked example (ten invented values, null mean 52, n = 10, 9 df): for each alpha, the two-sided critical t, the matching (1 - alpha) confidence interval for the mean and the resulting decision. Computed from the formulas in the cited sources.

One-sample t-test decision at three significance levels with matching confidence intervals
Significance level (alpha)Matching confidence level (percent)Two-sided critical t (9 df)Interval lower boundInterval upper boundContains null value 52?Decision
0.1901.83352.4454.4NoReject H0
0.05952.26252.2154.63NoReject H0
0.01993.2551.6855.16YesFail to reject H0

Sources

Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about Hypothesis Testing and Confidence Intervals: How They Connect

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →