Written and maintained by CASRAI Editorial Board
Published · Last updated
On this page
Oversampling means deliberately including more cases from a small group than its share of the population would produce, so that estimates for that group are precise enough to use. In survey research the oversampled group is weighted back down to its true proportion when whole-population figures are computed; in machine learning, oversampling means duplicating or synthesising minority-class cases in the training data to correct class imbalance. Both are controlled imbalances, and both carry a cost that has to be reported.
Definition: what oversampling means in survey design and in machine learning
Pew Research Center describes the survey sense directly. In its Discrimination 2025 methodology, the survey “includes an oversample of non-Hispanic Asian adults in order to provide more precise estimates,” those adults “were selected with certainty,” and the “oversampled groups are weighted back to reflect their correct proportions in the population” (Pew Research Center, 2025). The point of the design is the subgroup estimate; the weight is what keeps the full-sample estimate honest.
The machine-learning sense is different in purpose. The imbalanced-learn documentation defines random over-sampling as “randomly sampling with replacement the current available samples,” and describes SMOTE as generating a new sample on the line between a minority sample and one of its nearest neighbours (imbalanced-learn documentation, 2026). SMOTE was introduced by Chawla, Bowyer, Hall and Kegelmeyer (2002) in the Journal of Artificial Intelligence Research, combining minority over-sampling with majority under-sampling and evaluating with ROC and AUC.
| Question | Survey oversampling | Class-imbalance oversampling |
|---|---|---|
| What is changed | The selection probability of a subgroup, at the design stage | The class counts in the training data, after collection |
| Goal | Precise subgroup estimates | A classifier that does not ignore the minority class |
| Correction afterwards | Weights restore population proportions | Evaluate on untouched, unresampled data |
| Main cost | Lower effective sample size for whole-population estimates | Leakage and over-optimistic scores if done before splitting |
The rest of this page covers survey oversampling first, because that is where the arithmetic is, and then random oversampling and SMOTE.
Oversampling formula: base weights and the design effect
If a unit has selection probability pi, its base weight is wi = 1 / pi. Pew describes the base weight as reflecting the probability of recruitment, followed by calibration to population benchmarks and trimming at the 1st and 99th percentiles, with sampling errors that “take into account the effect of weighting.” Unequal weights cost precision. The Kish approximation, as implemented in Pew’s pewmethods package (read 2026-11-14), is:
deff = 1 + (sw / w̄)², and effective sample size = n / deff
where n is the number of cases, sw is the standard deviation of the weights and w̄ is their mean. An algebraically equivalent form, using the population form of the standard deviation, is deff = n Σw² / (Σw)². The 95% margin of error for a proportion in a simple random sample is 1.96 × √(p(1 − p) / n), and multiplying the variance inside the root by deff gives the weighted version.
Worked example: oversampling a group that is 10% of the population
Take a hypothetical population of 10,000 adults: 9,000 in Group A and 1,000 in Group B. The survey budget allows n = 1,000 interviews. Proportional sampling gives 900 from A and 100 from B. Now oversample B so that it supplies 300 interviews and A supplies 700. These numbers are illustrative, not from a real study.
- Selection probabilities: A = 700 / 9,000 = 0.0778; B = 300 / 1,000 = 0.30.
- Base weights: A = 1 / 0.0778 = 12.857; B = 1 / 0.30 = 3.333. Check: 700 × 12.857 = 9,000 and 300 × 3.333 = 1,000, so the weights reproduce the population.
- Subgroup precision for B at p = 0.5: with 100 cases the margin is 1.96 × √(0.25 / 100) = 9.8 points; with 300 cases it is 1.96 × √(0.25 / 300) = 5.7 points.
- Design effect: Σw² = 700 × 12.857² + 300 × 3.333² = 115,714 + 3,333 = 119,048; deff = 1,000 × 119,048 / 10,000² = 1.19. Effective sample size = 1,000 / 1.19 = 840.
- Whole-sample margin: 1.96 × √(0.25 × 1.19 / 1,000) = 3.4 points, against 3.1 points for a proportional sample of 1,000.
Suppose 60% of Group A and 40% of Group B answered “yes”. The unweighted estimate from the oversampled file is (420 + 120) / 1,000 = 54%, which is biased toward B. The weighted estimate is (12.857 × 420 + 3.333 × 120) / 10,000 = (5,400 + 400) / 10,000 = 58%, the population-correct figure (0.9 × 60% + 0.1 × 40%). The data table below extends the arithmetic to five allocations.
How to interpret an oversample: reading the trade-off
The worked example above is one point on a trade-off, and the dataset attached to this page extends the same arithmetic to five allocations of the 1,000 interviews. Read it as a trade: each step up in the Group B allocation shrinks the Group B margin of error from 9.8 points at 100 interviews to 4.4 points at 500, while the design effect rises from 1.00 to 1.64 and the effective sample size falls from 1,000 to 610. The whole-sample margin widens from 3.1 to 4.0 points.
In practice a design effect of 1.19 means the 1,000 interviews carry the precision of about 840 simple-random-sample interviews, a modest loss that most studies accept to get usable subgroup estimates. A design effect of 1.64 means a third of the sample’s information is spent on the imbalance in weights, which is hard to justify unless Group B results are the main deliverable. Oversampling pays when subgroup estimates are the reason for the study; it is a poor deal when only whole-population results matter.
Real programmes that oversample, and one that stopped
The National Center for Health Statistics reports that NHANES “resumed in August 2021 using a new sample design with no oversampling by race and Hispanic origin,” that examination sample weights account for the complex design, and that some Asian and Hispanic estimates for August 2021 to August 2023 were flagged for wide confidence intervals or reliability (NCHS Health E-Stat 111, read 2026-11-14). That is the oversampling trade-off seen from the other side: without the oversample, small-group estimates lose precision. Related CASRAI guides cover stratified sampling, the usual frame for allocating a boosted stratum, and survey weighting, design weights and raking.
Assumptions and common mistakes
- Forgetting to weight. Analysing an oversampled file unweighted gives the 54% result above, not 58%.
- Ignoring the design effect. Standard errors from software that treats weights as frequencies are too small. Pew states that its sampling errors and significance tests account for weighting.
- Oversampling a group you cannot identify in the frame. The selection probabilities must be known for every sampled unit, which is why oversampling pairs with stratified, multistage or cluster designs rather than with quota sampling, which has no known selection probabilities.
- Resampling before the train/test split (machine learning). The imbalanced-learn pitfalls page warns that resampling the whole dataset first causes leakage: in its experiment the leaky approach scored about 0.724 in cross-validation but about 0.698 on held-out data, while the pipeline approach gave about 0.732 and 0.727 (imbalanced-learn documentation, read 2026-11-14).
- Judging a resampled model by accuracy. Evaluate on the original class distribution, using ROC/AUC as in the SMOTE paper, and see cross-validation for how to keep folds clean.
How to report oversampling (APA example)
State the design, the rationale, the weights and the precision cost. Example wording: “Group B, 10% of the sampling frame, was oversampled to 30% of completed interviews to support subgroup estimates. Base weights equal the inverse of selection probability (Group A = 12.86, Group B = 3.33). The Kish design effect was 1.19, giving an effective sample size of 840 from 1,000 completed interviews. All population estimates are weighted.” For class resampling, report that resampling was applied inside each training fold only, and name the method and its parameters.
How to calculate oversampling weights and design effect in Excel, R or Python
Excel
=COUNT(B2:B1001)*SUMSQ(B2:B1001)/SUM(B2:B1001)^2
With the weights in B2:B1001, this returns deff. Effective n is =COUNT(B2:B1001)/that cell.
R
w = c(rep(9000/700, 700), rep(1000/300, 300)) deff = length(w) * sum(w^2) / sum(w)^2 # 1.190 n_eff = length(w) / deff # 840
Python (weights, then SMOTE inside a pipeline)
import numpy as np
w = np.r_[np.full(700, 9000/700), np.full(300, 1000/300)]
deff = len(w) * np.sum(w**2) / np.sum(w)**2 # 1.190
n_eff = len(w) / deff # 840
from imblearn.pipeline import Pipeline
from imblearn.over_sampling import SMOTE
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
pipe = Pipeline([("smote", SMOTE(random_state=0)), ("clf", LogisticRegression(max_iter=1000))])
scores = cross_val_score(pipe, X, y, scoring="roc_auc", cv=5)
Putting SMOTE inside the imbalanced-learn pipeline means resampling happens only on each training fold, as the pitfalls page recommends.
Frequently asked questions
What is oversampling in statistics?
It is sampling a small subgroup at a higher rate than its population share so subgroup estimates are precise, then using weights to restore the true proportions in whole-population estimates.
What is the difference between oversampling and weighting?
Oversampling is a design decision made before data collection; weighting is the analysis step that undoes its distortion. An oversample without weights is biased toward the boosted group.
Does oversampling reduce the effective sample size?
For whole-population estimates, yes. In the worked example, 1,000 interviews with a 30% oversample of a 10% group behave like 840 for whole-population estimates (design effect 1.19), while the group’s own estimates improve.
What is the difference between random oversampling and SMOTE?
Random oversampling duplicates existing minority cases by sampling with replacement. SMOTE creates new synthetic cases on the line between a minority case and a nearest minority neighbour (imbalanced-learn documentation).
When should I oversample before or after splitting data?
After. Split first, then resample only the training portion, ideally inside a pipeline, so no test information reaches training.
Sources
- Pew Research Center (2025). Discrimination 2025 methodology. https://www.pewresearch.org/politics/2025/05/20/discrimination-2025-methodology/ (read 2026-11-14)
- pewmethods R package, calculate_deff (Kish approximation). https://rdrr.io/github/pewresearch/pewmethods/src/R/calculate_deff.R (read 2026-11-14)
- National Center for Health Statistics. Health E-Stat 111, NHANES August 2021 to August 2023. https://www.cdc.gov/nchs/data/hestat/hestat111.htm (read 2026-11-14)
- imbalanced-learn documentation: Over-sampling. https://imbalanced-learn.org/stable/over_sampling.html and Common pitfalls. https://imbalanced-learn.org/stable/common_pitfalls.html (read 2026-11-14)
- Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321-357. https://doi.org/10.1613/jair.953
Data: Oversampling trade-off: precision gain for a small subgroup versus weighting cost (worked scenarios, n = 1,000)
Computed scenarios for a hypothetical population of 10,000 adults (9,000 Group A, 1,000 Group B) sampled at n = 1,000. For each share of the sample allocated to Group B the table gives sample sizes, inverse-probability base weights, the Kish design effect (n times the sum of squared weights divided by the squared sum of weights), effective sample size, and 95% margins of error at p = 0.5 for Group B alone and for the weighted whole sample. All values were calculated from the formulas in the cited sources; no survey data are involved.
Period covered: 2026.
| Allocation scenario | Group B sample size (respondents) | Group A sample size (respondents) | Group A base weight | Group B base weight | Kish design effect | Effective sample size (respondents) | Group B margin of error (95%, p = 0.5) (percentage points) | Whole-sample margin of error (95%, p = 0.5, with design effect) (percentage points) |
|---|---|---|---|---|---|---|---|---|
| 10% of sample from Group B | 100 | 900 | 10 | 10 | 1 | 1000 | 9.8 | 3.1 |
| 20% of sample from Group B | 200 | 800 | 11.25 | 5 | 1.062 | 941 | 6.9 | 3.2 |
| 30% of sample from Group B | 300 | 700 | 12.86 | 3.33 | 1.19 | 840 | 5.7 | 3.4 |
| 40% of sample from Group B | 400 | 600 | 15 | 2.5 | 1.375 | 727 | 4.9 | 3.6 |
| 50% of sample from Group B | 500 | 500 | 18 | 2 | 1.64 | 610 | 4.4 | 4 |
Sources
- pewmethods calculate_deff (Kish approximation: deff = 1 + (sd/mean)^2 of weights; ess = n/deff) (accessed 2026-11-14)
- Pew Research Center, Discrimination 2025 methodology (oversample weighted back to population proportions) (accessed 2026-11-14)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.







