T-Test: One-Sample, Independent, and Paired — When to Use Each

A t-test is a statistical hypothesis test used to determine whether the mean of a continuous variable differs from a specified value, or whether the means of two groups differ from each other, more than would be expected by chance alone. It is one of the most widely used procedures in quantitative research because the question it answers — “is this difference in averages real, or is it noise?” — comes up constantly, from comparing a treatment group to a control group to checking whether a sample mean matches a population benchmark.

This guide covers the three types of t-test, the decision rule for choosing between them, the assumptions each relies on, how to read and report the output, and when a t-test is the wrong tool for the job.

The means and standard deviations a t-test compares are themselves the descriptive statistics covered in CASRAI’s guide to descriptive statistics, which explains how central tendency and dispersion are calculated and reported in the first place.

What a t-test does

A t-test compares an observed difference in means to the amount of variability you’d expect from sampling error alone, expressed as a test statistic (t) with an associated p-value. For a full treatment of what that p-value does and doesn’t mean, see CASRAI’s guide on what a p value is. If the observed difference is large relative to that expected variability, the test returns a small p-value, and the difference is described as statistically significant. A t-test only ever compares means of a continuous outcome across one or two groups (or one group against a fixed value) — for three or more groups, see the note on ANOVA below.

The three types of t-test

All three types share the same underlying logic; they differ in what’s being compared.

1. One-sample t-test

Compares the mean of a single sample to a known or hypothesized value. Example use: testing whether the average turnaround time for IRB review at one institution differs from a published national benchmark.

2. Independent-samples (two-sample) t-test

Compares the means of two separate, unrelated groups — for example, a treatment group versus a control group, or survey respondents from two different institutions. The two samples must consist of different subjects with no pairing between them.

3. Paired (repeated-measures) t-test

Compares two related measurements taken from the same subjects or matched pairs — for example, the same participants’ scores before and after an intervention, or matched case-control pairs. Because each pair shares a common source of variability, the paired test analyzes the differences within each pair rather than treating the two sets of scores as independent.

Decision rule

• One group, compared against a fixed number → one-sample
• Two groups, different subjects in each → independent-samples
• Two measurements, same subjects (or matched pairs) → paired
• Three or more groups or measurements → not a t-test — see ANOVA

Mistaking a paired design for an independent-samples design (or vice versa) is one of the most common analysis errors in applied research — see the “when not to use” section below.

Assumptions

The standard (Student’s) t-test relies on four assumptions:

• Continuous outcome. The dependent variable should be measured on an interval or ratio scale. See types of variables for how this fits into the broader classification of data types.
• Independence of observations. Each observation (or each pair, in a paired design) must be independent of the others. Data collected in clusters — e.g., multiple measurements per patient, or students nested within classrooms — violates this and requires a different approach (see below).
• Approximate normality. The outcome should be approximately normally distributed within each group, or the differences should be approximately normal in a paired design. This matters most in small samples; by the Central Limit Theorem, the sampling distribution of the mean becomes approximately normal as sample size grows even when the underlying data are not, which is why the t-test is considered reasonably robust to mild non-normality at larger n (rough rule of thumb, n ≥ ~30 per group). In small samples with clearly skewed or ordinal data, a non-parametric alternative is safer. A histogram or Q-Q plot of the outcome (or of the differences, in a paired design) is a fast way to check this visually before running the test — see CASRAI’s guide to normality of distribution for the fuller set of checks (Q-Q plots, Shapiro-Wilk, skewness/kurtosis) and what to do if the assumption doesn’t hold.
• Homogeneity of variance (independent-samples test only). The classic Student’s two-sample t-test assumes both groups have equal population variances.

Welch’s t-test: the safer default for two independent samples

When two groups’ variances are unequal — especially combined with unequal sample sizes — the classic Student’s t-test’s Type I error rate becomes unreliable. Welch’s t-test (also called Welch’s unequal variances t-test) does not assume equal variances: it adjusts both the standard error calculation and the degrees of freedom to account for the difference. Because Welch’s test performs essentially identically to Student’s t-test when variances genuinely are equal, but stays valid when they aren’t, a substantial body of methodological literature recommends using Welch’s test as the default for any independent-samples comparison rather than testing for equal variances first and switching tests conditionally. R’s built-in t.test() function reflects this consensus — it uses Welch’s version by default.

Test statistic, degrees of freedom, and tails

Conceptually, the t statistic is a signal-to-noise ratio: the observed difference between means, divided by the standard error of that difference (an estimate of how much the difference would bounce around from sample to sample due to chance). A larger absolute t value means the observed difference is large relative to its expected sampling variability.

See CASRAI’s guide to degrees of freedom for the general concept; here, degrees of freedom (df) reflect how much independent information is available to estimate variability, and determine the exact shape of the reference t-distribution used to compute the p-value:

• One-sample and paired t-test: df = n − 1
• Independent-samples Student’s t-test: df = n₁ + n₂ − 2
• Welch’s t-test: df is calculated via the Welch–Satterthwaite equation, which weights each group’s variance and sample size; the result is typically a non-integer value below n₁ + n₂ − 2, reflecting the extra uncertainty from not assuming equal variances

One-tailed vs. two-tailed: A two-tailed test asks whether the means differ in either direction; a one-tailed test asks whether the difference goes in one pre-specified direction only. Two-tailed is the standard default because it doesn’t require committing in advance to a direction, and using a one-tailed test only to reduce a borderline p-value below a threshold after the fact is a form of p-hacking. A one-tailed test is only appropriate when a directional hypothesis was specified before data collection and there is a substantive reason a difference in the other direction would be treated identically to no difference at all — a rare situation in most research contexts.

Interpreting the output

A t-test report typically includes four pieces of information: the t statistic, degrees of freedom, the p-value, and the mean difference with its confidence interval. All four matter, but the mean difference and its confidence interval usually carry more practical information than the p-value alone: the p-value only tells you whether the difference is unlikely to be due to chance at a given threshold, while the confidence interval tells you the plausible range for the size of that difference, and how precisely it was estimated. A statistically significant result with a very wide confidence interval (e.g., a difference plausibly anywhere from barely-detectable to very large) is a much weaker basis for a practical claim than a significant result with a narrow interval, even if both produce p < .05. See how to report p-values for more on why p-values shouldn’t be interpreted in isolation, and statistical significance vs. clinical significance for why a significant mean difference isn’t automatically a meaningful one.

Effect size

Because statistical significance depends on sample size as well as the size of the difference, a t-test result should normally be reported alongside an effect size that quantifies how large the difference actually is, independent of sample size.

• Cohen’s d expresses the mean difference in standard-deviation units: the mean difference divided by the pooled standard deviation. Conventional benchmarks (Cohen, 1988) are d ≈ 0.2 (small), 0.5 (medium), and 0.8 (large) — but these are rough, field-dependent heuristics, not fixed thresholds. A d of 0.3 can be a large, practically important effect in some fields (e.g., some areas of medicine or education) and a trivial one in others; always interpret effect-size magnitude against the norms of the specific literature, not the generic Cohen benchmarks alone.
• Hedges’ g applies a small-sample correction to Cohen’s d, which is slightly biased (tends to overestimate the true effect) when sample sizes are small. Hedges’ g is generally preferred over Cohen’s d for samples under roughly n = 20 per group.

Planning the sample size needed to detect a given effect size at adequate power is covered in power analysis and sample size calculation.

Reporting a t-test in APA style

APA style reports a t-test result in a standard sequence: the test statistic (t), degrees of freedom (in parentheses), the t value, the p-value, and, where relevant, the confidence interval and effect size.

Worked example sentence (illustrative wording only, not a real dataset or finding):

An independent-samples t-test indicated that response accuracy was higher in the intervention group (M = 82.4, SD = 6.1) than in the control group (M = 76.9, SD = 7.8), t(58) = 3.02, p = .004, 95% CI [1.87, 9.13], d = 0.78.

For a Welch’s test, most style guides expect the same format, simply using the (often non-integer) Welch-adjusted degrees of freedom, e.g., t(41.3) = ….

When not to use a t-test

• More than two groups. Running separate pairwise t-tests across three or more groups inflates the family-wise Type I error rate — each additional comparison adds its own chance of a false positive, so the probability of at least one spurious “significant” result across the full set climbs well above the nominal 5% used for any single test. Use ANOVA (with an appropriate post-hoc correction for pairwise comparisons) instead.
• Small, non-normal samples or ordinal data. When the normality assumption is clearly untenable and the sample is too small for the Central Limit Theorem to help, or the outcome is ordinal rather than truly continuous, use a non-parametric alternative: the Mann-Whitney U test in place of the independent-samples t-test, or the Wilcoxon signed-rank test in place of the paired t-test.
• Confusing paired and independent designs. Analyzing paired data (e.g., pre/post scores on the same subjects) with the independent-samples test discards the information gained by pairing and typically produces an overly conservative (less powerful) test; analyzing independent data as if paired is simply invalid, since there is no meaningful pairing to align.
• Clustered or nested data. If observations are grouped — repeated measures beyond a single before/after pair, patients nested within clinics, students nested within classrooms — the independence assumption is violated and a t-test will understate the true uncertainty. Multilevel/mixed-effects models or repeated-measures ANOVA are the appropriate tools.
• Categorical outcome. A t-test only applies to a continuous outcome. If the outcome is categorical — success/failure counts across groups, for example — see CASRAI’s chi-square test guide instead.

How to run a t-test

R

R’s base t.test() function handles all three types:

t.test(x, mu = 50)                       # one-sample
t.test(x, y)                             # independent-samples, Welch's by default
t.test(x, y, var.equal = TRUE)           # independent-samples, Student's (equal variances)
t.test(x, y, paired = TRUE)              # paired

Note that t.test() uses Welch’s version by default for the two-sample case (var.equal = FALSE is the default) — set var.equal = TRUE explicitly only if the classic Student’s test is specifically required.

Python (SciPy)

from scipy import stats
stats.ttest_1samp(x, popmean=50)               # one-sample
stats.ttest_ind(x, y, equal_var=False)         # independent-samples, Welch's
stats.ttest_ind(x, y, equal_var=True)          # independent-samples, Student's (default)
stats.ttest_rel(x, y)                          # paired

scipy.stats.ttest_ind() defaults to equal_var=True (Student’s test) — set equal_var=False explicitly to get Welch’s test.

SPSS

In SPSS, use Analyze > Compare Means > One-Sample T Test, Independent-Samples T Test, or Paired-Samples T Test depending on design. For the independent-samples procedure, SPSS automatically runs Levene’s Test for Equality of Variances alongside the t-test and prints two output rows — “Equal variances assumed” and “Equal variances not assumed” (the Welch version) — so you can read the appropriate row based on the Levene’s test result.

Related CASRAI resources

• Statistical test — the broader category a t-test belongs to
• ANOVA — for comparing three or more group means
• Effect size — quantifying the magnitude of a difference
• Power analysis and sample size calculation — planning sample size in advance
• P-value and how to report p-values
• Confidence interval
• Types of variables
• Statistical significance vs. clinical significance
• Regression analysis — for modeling relationships beyond a simple two-group comparison

Frequently asked questions

What does a t-test tell you?

It tells you whether an observed difference between a sample mean and a reference value (or between two group means) is larger than would be expected from random sampling variation alone, expressed as a p-value, alongside an estimate of the size and precision of that difference (the mean difference and its confidence interval).

What is the difference between a t-test and a z-test?

Both compare means, but a t-test is used when the population standard deviation is unknown and estimated from the sample (the normal case in real research), while a z-test assumes the population standard deviation is known. In practice, almost all applied mean-comparison tests use the t-test rather than the z-test.

What is the difference between a t-test and ANOVA?

A t-test compares the means of one or two groups. ANOVA (analysis of variance) compares the means of three or more groups in a single test, avoiding the inflated false-positive rate that comes from running multiple separate t-tests across all group pairs. See ANOVA.

Can a t-test be used with a small sample size?

Yes, in principle a t-test can be used with very small samples, but its validity then depends much more heavily on the normality assumption holding, since the Central Limit Theorem provides less help at low n. With small, non-normal, or ordinal data, a non-parametric alternative (Mann-Whitney U or Wilcoxon signed-rank) is usually safer.

Should I always use Welch’s t-test instead of Student’s t-test?

For independent-samples comparisons, using Welch’s t-test by default is widely recommended in the methodological literature: it performs about as well as Student’s test when variances are equal, and stays valid when they are not, so there is little practical downside to defaulting to it.