Bootstrapping is a resampling method for estimating the uncertainty of a statistic — a mean, a median, a regression coefficient, a correlation — when you don’t want to (or can’t) rely on a parametric formula. Instead of assuming a shape for the sampling distribution, you build an approximation of it directly from your own data: draw a new sample, the same size as the original, with replacement, from the data you already have; compute the statistic on that resample; repeat thousands of times; then read the spread of the resulting distribution as your uncertainty estimate. This page works through the mechanics on a tiny worked dataset, the difference between percentile and BCa intervals, how many resamples is actually enough, and the specific situations — small samples, dependent data, extreme quantiles — where bootstrapping breaks down rather than just getting noisier.
What bootstrapping actually does
The core move is called resampling with replacement. If your original sample has n observations, each bootstrap resample also has n observations, but drawn by picking randomly from the original n values, one at a time, replacing each value before the next draw. That means any given resample will typically omit some original observations entirely and duplicate others. The logic is that your original sample is itself an estimate of the population, so treating it as a stand-in population — and resampling from it — approximates what would happen if you could actually draw many new samples from the real population, which in practice you almost never can.
The method was introduced by Bradley Efron in his 1979 paper “Bootstrap Methods: Another Look at the Jackknife” (Annals of Statistics), as a generalization of the older jackknife resampling technique. It is a nonparametric approach: it makes no assumption that the underlying population follows a normal distribution or any other specific shape, which is exactly what makes it useful for statistics — medians, ratios, correlation coefficients, regression coefficients in small samples — where a clean parametric standard-error formula either doesn’t exist or relies on assumptions the data don’t satisfy.
The procedure, worked through on a tiny dataset
The following dataset is a small illustrative set of eight values, used purely to make the arithmetic traceable by hand — it is not drawn from any real study.
Original sample (n = 8): 4, 7, 7, 8, 9, 11, 12, 15 — sample mean = 9.125
- Resample with replacement. Draw 8 values from the original 8, with replacement. One possible resample: 7, 4, 11, 7, 15, 8, 4, 9 (notice 4 and 7 each appear twice; 12 doesn’t appear at all). Compute the statistic of interest — here, the mean: 8.125.
- Repeat. Draw a second independent resample: 9, 12, 7, 8, 11, 11, 4, 15. Mean: 9.625. Repeat this process a large number of times, B (see the section below on how large B needs to be) — each time drawing fresh with-replacement resample of size 8 from the same original 8 values and recording the statistic.
- Build the bootstrap distribution. After B resamples you have B recorded statistics (B bootstrap means, in this example). Sorted, these form an empirical approximation of the sampling distribution of the mean — the distribution the sample mean would actually follow if you could repeat the whole study many times.
- Read off the interval. To get a 95% confidence interval, take the 2.5th and 97.5th percentiles of that sorted bootstrap distribution. If, after 2,000 resamples, the 2.5th percentile of the bootstrap means is 6.4 and the 97.5th percentile is 12.1, your percentile bootstrap 95% CI for the mean is roughly (6.4, 12.1). (These figures continue the worked illustration; they are not the exact output of any specific run and will vary slightly with the random seed and B.)
Every step above generalizes past the mean — the same with-replacement, recompute, repeat procedure works for a median, a trimmed mean, a correlation coefficient, an odds ratio, or a regression coefficient. The only thing that changes is which statistic you compute at step 1-2 on each resample.
Percentile vs. BCa intervals
The percentile method above is the simplest bootstrap confidence interval, but it isn’t the only one, and it isn’t always the most accurate. The bias-corrected and accelerated (BCa) method, also introduced by Efron (1987), adjusts the interval to account for two problems the plain percentile method ignores: bias (the bootstrap distribution’s center may not line up with the original-sample estimate) and skewness (the standard error of the statistic may itself change depending on the true parameter value, which is common for statistics like correlations and ratios). BCa applies a bias-correction factor and an acceleration factor, both estimated from the bootstrap resamples themselves (the acceleration factor via jackknife), to shift and stretch the interval’s endpoints asymmetrically rather than simply cutting at the 2.5th/97.5th percentiles.
| Method | How the interval is built | When to prefer it |
|---|---|---|
| Percentile | Sort the bootstrap statistics; take the 2.5th/97.5th percentiles directly | Simple, symmetric, roughly normal statistics (e.g., means from reasonably large, well-behaved samples); easiest to explain and defend |
| Bias-corrected and accelerated (BCa) | Percentile endpoints adjusted for bias and skewness, estimated from the bootstrap and jackknife distributions | Skewed statistics (ratios, correlations, variances), smaller samples, or whenever the bootstrap distribution itself looks visibly asymmetric |
| Basic (“reverse percentile”) | Reflects the percentile interval around the original-sample estimate | Occasionally used as a quick check against the percentile method; generally superseded by BCa in modern practice |
| Studentized (bootstrap-t) | Bootstraps a pivotal t-like statistic rather than the raw statistic | Can outperform BCa in specific cases but requires a usable standard-error estimate on every resample, which isn’t always available |
In practice: BCa is the more defensible default when you have the computational budget for it (it typically requires more resamples than the percentile method to stabilize — see below), and most modern statistical software (R’s boot package, Python’s scipy.stats.bootstrap) offers it as a named option specifically because plain percentile intervals are known to undercover — produce intervals narrower than their claimed confidence level suggests — when the underlying statistic is biased or skewed.
How many resamples is enough?
The number of bootstrap resamples, conventionally called B, trades computation time against how stable your interval estimate is. There is no single universal number, because the answer depends on what you’re computing and how far into the tails of the distribution you need to be accurate — but the applied-statistics literature converges on a few practical bands:
| Goal | Typical B | Why |
|---|---|---|
| Bootstrap standard error only | ~200–1,000 | The standard error of the bootstrap distribution stabilizes relatively quickly; it doesn’t depend on precisely resolving the tails |
| Percentile confidence interval | ~1,000–2,000 | You need the 2.5th/97.5th percentiles to be stable, which requires more resamples than a standard error alone |
| BCa confidence interval | Several thousand and up (methodological work on choosing B for BCa recommends deriving B from a target tolerance rather than a fixed number) | The bias-correction and acceleration adjustments compound the sampling noise in the percentile endpoints, so BCa needs a larger B to reach the same stability |
The practical rule that follows from this: run your bootstrap at a candidate B, then rerun it with a different random seed at the same B. If the resulting interval endpoints move by an amount that matters for your decision, increase B and check again. This costs nothing but computation time, and computation time is cheap for all but the largest datasets or most expensive statistics (e.g., resampling that requires refitting a complex model on every draw).
When bootstrapping fails
Bootstrapping is not a universal substitute for a parametric method, and treating it as one is a real source of misleading results:
- Small original sample size. The bootstrap resamples from the original data — it cannot manufacture information the original sample doesn’t contain. If n is very small (roughly, single digits to low teens, depending on the statistic), the original sample is a poor stand-in for the population, and every bootstrap resample inherits the same gaps. The bootstrap distribution will look precise without actually being accurate — a false sense of confidence, not a real one.
- Dependent or correlated data. Ordinary with-replacement resampling assumes the original observations are independent. For time series, clustered data (e.g., repeated measures on the same subject, or observations grouped by site), or spatially autocorrelated data, plain bootstrapping breaks the dependence structure and produces intervals that are too narrow. Variants exist for this — the block bootstrap for time series, the cluster bootstrap for grouped data — but the plain method should not be applied to dependent data without one of these adjustments.
- Extreme quantiles and tail estimates. Bootstrapping the median or an interquartile range works reasonably well; bootstrapping something like the 99th percentile, a maximum, or a minimum is much less reliable, because those statistics depend heavily on the most extreme one or two values in the sample, and resampling with replacement from a finite sample cannot generate values more extreme than what’s already there.
- Statistics that aren’t “smooth.” The bootstrap’s theoretical justification relies on the statistic being a reasonably smooth function of the data. Statistics like the sample maximum/minimum, or counts near a hard boundary, violate this and are known cases where the bootstrap performs poorly even asymptotically — not just in small samples.
- Non-representative original sample. If the original sample carries a real bias (systematic non-response, a convenience-sampling design, measurement error concentrated in one direction), the bootstrap reproduces that same bias in every resample. Bootstrapping quantifies sampling variability; it does nothing to correct bias baked into the sampling design itself. See selection bias for the underlying failure mode this can mask if not checked first.
Bootstrapping vs. other resampling methods
Bootstrapping is often confused with two related but distinct resampling techniques:
| Method | What it resamples | What it’s for |
|---|---|---|
| Bootstrap | With-replacement resamples of size n from the original data | Estimating the sampling distribution / standard error / confidence interval of a statistic |
| Jackknife | Leave-one-out subsamples of size n−1 (systematically drop each observation once) | Bias estimation and standard errors; historically the precursor to the bootstrap, still used inside BCa to estimate the acceleration factor |
| Permutation test | Random reassignment of group labels (without replacement) across the observed data | Hypothesis testing — asking whether an observed difference between groups is larger than what relabeling would produce by chance, not estimating a confidence interval |
The practical distinction that matters most: bootstrapping answers “how uncertain is this estimate?” A permutation test answers “could this difference have arisen by chance alone?” They are sometimes used together — a permutation test to establish significance, a bootstrap interval to communicate the size and uncertainty of the effect — but they are not interchangeable, and reporting a bootstrap interval is not the same as reporting a hypothesis test.
Frequently asked questions
Is bootstrapping the same as cross-validation?
No. Cross-validation splits data to evaluate a model’s predictive performance on held-out data; bootstrapping resamples with replacement to estimate the uncertainty of a statistic or estimator. They both involve repeated resampling and are sometimes combined (e.g., the .632 bootstrap used in model-evaluation contexts), but they answer different questions.
Does bootstrapping require a normal distribution?
No — that’s the point of the method. Bootstrapping is nonparametric: it does not assume the population follows a normal distribution or any other specific parametric shape. That’s what makes it useful precisely in cases where a normal-theory formula (like the standard formula for a confidence interval around a mean) would be questionable.
Can bootstrapping fix a small sample size?
No. Bootstrapping quantifies the uncertainty already present in your data; it cannot add information that wasn’t collected. A bootstrap interval from a very small sample will often look deceptively narrow while actually being unreliable, because every resample draws from the same limited set of original values. See the “when bootstrapping fails” section above.
What software runs a bootstrap?
Most general-purpose statistical software supports it: R’s boot package (including BCa via boot.ci), Python’s scipy.stats.bootstrap, and Stata’s bootstrap command are the most commonly used implementations, and all three natively support the percentile and BCa methods described above.
How is a bootstrap confidence interval different from a standard confidence interval?
A standard (parametric) confidence interval is built from a formula that assumes a particular sampling distribution shape (often a normal or t-distribution) and a standard-error formula derived analytically for that statistic. A bootstrap confidence interval is built empirically, from resampling, without assuming that shape — see confidence interval explained for the general concept and how it is defined regardless of method.
For the broader groundwork this method builds on, see law of large numbers and effect size. For the sampling designs bootstrapping is often applied downstream of, see stratified sampling, snowball sampling and the random vs. convenience sampling comparison. For related but distinct methods of quantifying uncertainty, see confidence interval explained and what is a p value.
