An effect size is a standardized number describing how large a result is; a p-value only describes how unlikely that result is to be due to chance. The two answer different questions, and a Results section that reports one without the other is incomplete by current reporting standards. The harder problem is upstream of reporting: which effect-size measure to compute depends on your design (two groups vs. many groups, continuous vs. categorical, independent vs. correlational), and Cohen’s small/medium/large labels — the numbers nearly every calculator site repeats — were offered by Cohen himself as a fallback for when a field has nothing better, not a universal yardstick.
This page covers the family: which measure fits which design, why effect size and statistical significance are not substitutes for each other, the specific danger in applying Cohen’s generic benchmarks outside the context he intended, and a reporting template you can adapt directly. For a deep dive on interpreting Cohen’s d specifically — field-calibrated benchmarks, a worked interpretation example, and when to report Hedges’ g instead — see How to Interpret Cohen’s d. For estimating the sample size a target effect size requires before you collect data, see Power Analysis and Sample Size Calculation.
Effect size vs. statistical significance: two different questions
A p-value tests the null hypothesis: assuming there is truly no effect, how likely is a result at least this extreme? It is a function of both the true effect and the sample size — which means the same underlying effect can produce a tiny p-value in a large sample and a large p-value in a small one. A trivially small, practically meaningless difference becomes ‘statistically significant’ with enough observations; a genuinely large, meaningful effect can fail to reach significance in an underpowered study. Effect size is the number that doesn’t move with sample size in that way — it estimates the magnitude of the difference, relationship, or association itself.
The American Statistical Association’s 2016 statement on p-values makes this the second of its six principles, stating directly that ‘smaller p-values do not necessarily imply the presence of larger or more important effects, and larger p-values do not imply a lack of importance or even lack of effect’ (Wasserstein & Lazar, The American Statistician, 70(2):129–133, 2016). The Publication Manual of the American Psychological Association (7th edition, Section 6.5) goes further and makes effect-size reporting a requirement, not a courtesy: for each primary outcome, report an effect size and its confidence interval alongside the significance test. CONSORT 2010, the reporting standard most biomedical journals hold randomized trials to, carries the equivalent requirement for every primary and secondary outcome. A manuscript that reports only ‘p < .05’ with no effect size is, by these standards, an incomplete Results section regardless of how small the p-value is.
Choosing the right effect-size measure by design
There is no single effect size; the correct measure is determined by what kind of comparison your design produces. Using the table below to identify the right family avoids the common mistake of forcing a two-group statistic (like d) onto a multi-group or categorical design where it doesn’t apply.
| Design / comparison | Appropriate effect-size measure | What it standardizes |
|---|---|---|
| Two independent group means (t-test) | Cohen’s d (or Hedges’ g for small samples) | Mean difference in pooled standard-deviation units |
| Two paired/repeated means (paired t-test) | Cohen’s d for paired samples, or dz | Mean of the within-pair differences, standardized |
| Three or more group means (one-way ANOVA) | Eta-squared (η²) or partial eta-squared (ηp²) | Proportion of total variance attributable to the group factor |
| Factorial / repeated-measures ANOVA | Partial eta-squared, or generalized eta-squared (ηG²) for cross-design comparability | Variance attributable to one factor, controlling for the others in the model |
| Association between two continuous variables | Pearson’s r (already standardized — no conversion needed) | Strength and direction of linear association, −1 to 1 |
| Linear regression / multiple predictors | R² (model) or standardized β / semi-partial r² (individual predictor) | Proportion of outcome variance explained |
| 2×2 or larger contingency table (chi-square) | Cramer’s V (or phi for a 2×2 table) | Strength of association between two categorical variables, 0 to 1 |
| Binary outcome, case-control design | Odds ratio (OR) | Odds of the outcome in exposed vs. unexposed group |
| Binary outcome, cohort or trial design with known incidence | Risk ratio / relative risk (RR) | Probability of the outcome in exposed vs. unexposed group |
| Binary outcome, absolute clinical impact | Absolute risk reduction / Number Needed to Treat (NNT) | Real-world patients needed to treat for one additional benefit |
A few of these deserve a note. Pearson’s r is already an effect size in its own right — no separate conversion step is needed when you report a correlation. An odds ratio’s confidence interval is read against 1 (no effect), not 0, which trips up authors used to reading d or r against a zero baseline; because odds ratios diverge from relative risk as baseline event rates rise, they are not interchangeable, and design (case-control vs. cohort/trial) generally dictates which one you can validly compute rather than which one you’d prefer to report. R² and eta-squared are both variance-explained measures and are directly comparable across models built the same way, but partial eta-squared inflates as more factors are added to a factorial design, which is why generalized eta-squared is often preferred when comparing effect sizes across studies with different designs.
The danger in Cohen’s small/medium/large benchmarks
Cohen’s 1988 convention — d = 0.2 small, 0.5 medium, 0.8 large — is the single most-cited effect-size yardstick in the literature, and it was explicitly offered by Cohen as a fallback only, for use when a field has no better empirical basis for judging what counts as a meaningful effect in its own domain. Treated as a universal rule instead, it produces real interpretation errors, because published effect sizes vary substantially by field:
- Psychology: the discipline-wide mean published effect is closer to d ≈ 0.4 — below Cohen’s own ‘medium’ threshold.
- Education research: some methodologists propose an alternate 0.2/0.4/0.6 scale, lower across the board than Cohen’s.
- Gerontology: a proposed field-specific scale runs 0.15/0.40/0.75.
- Clinical medicine: effects in the 0.05–0.2 range are routinely the clinically meaningful ones, given how large a population-level intervention effect needs to be to matter at scale — a d that would be dismissed as ‘small’ by Cohen’s convention can represent a genuinely important clinical result.
The same caution applies to eta-squared, Cramer’s V, and odds-ratio benchmark tables that circulate the same way — they are all convenience defaults, not field-validated thresholds. Wherever possible, interpret a computed effect size against the distribution of effects your specific field actually publishes (a recent meta-analysis or systematic review in the same sub-area is the best available benchmark) rather than a generic textbook cutoff. See How to Interpret Cohen’s d for field-calibrated benchmark tables and a fully worked interpretation example.
How to report effect size: a template
A complete effect-size report states, at minimum: the statistic used, its value, its confidence interval, and a plain-language magnitude statement calibrated to the field rather than a bare textbook label. Four worked examples, one per common design:
- Two-group comparison: ‘Participants in the intervention group scored higher than control, t(98) = 2.85, p = .005, d = 0.57, 95% CI [0.17, 0.97], a medium-to-large effect by conventional benchmarks and consistent with prior trials in this literature.’
- Correlation: ‘Hours of sleep were associated with next-day recall accuracy, r(148) = .42, p < .001, 95% CI [.28, .54].’
- ANOVA: ‘Condition had a significant effect on response time, F(2, 117) = 6.31, p = .003, ηp² = .10, indicating condition accounted for approximately 10% of the variance in response time after controlling for the other factors in the model.’
- Case-control / binary outcome: ‘Exposure was associated with the outcome, OR = 2.3, 95% CI [1.4, 3.8], p = .001. Because the confidence interval excludes 1, the association is unlikely to be due to chance alone; the effect size (OR = 2.3) further indicates the odds of the outcome were more than double among exposed participants.’
Three habits separate an adequate report from an incomplete one: always pair the point estimate with a confidence interval, not just a p-value; state the magnitude in language calibrated to the field’s own published range rather than a bare ‘small/medium/large’ label with no source; and choose the effect-size measure before running the analysis, driven by the design (per the table above), not chosen after the fact from whichever statistic happened to produce the most favorable-looking number.
Effect size and study design: why this belongs upstream of data collection
Effect size is not only a Results-section statistic — it is one of the four inputs to every a priori power analysis, alongside significance level (α), desired statistical power (typically .80), and sample size: fix any three and the fourth is determined. In practice this means a researcher must specify an expected or minimally-important effect size before data collection, in order to calculate the sample size the study needs to reliably detect it. An effect size assumed too optimistically at the design stage is one of the more common, and more consequential, causes of an underpowered study. See Power Analysis and Sample Size Calculation for how the four quantities relate and how to justify a sample-size figure in a protocol or grant application, and Justifying Sample Size in a Manuscript for how to defend that figure to reviewers after the fact.
Frequently asked questions
Is a bigger effect size always better?
Not necessarily. A large effect size from a poorly controlled or biased design is not more trustworthy than a smaller effect from a rigorous one — effect size describes magnitude, not validity. A very large effect size can also be a red flag for a small, underpowered sample, where extreme estimates are more common due to sampling variability alone.
Can I calculate effect size after running the study if I didn’t plan for it?
Yes — effect size can be computed post hoc from the same data used for a significance test (unlike power, which is only meaningfully calculated a priori). What cannot be done validly is retrofitting a target effect size to justify a sample size that was actually chosen for other reasons, sometimes called an observed-power or post hoc power calculation; methodologists generally advise against this specific practice because it is circular with respect to the p-value already obtained.
Why does my statistics software report multiple effect sizes for the same test?
Many procedures support more than one valid effect-size statistic (e.g., both eta-squared and partial eta-squared for the same ANOVA, or both Cohen’s d and Hedges’ g for the same t-test). They are not identical and are not always directly comparable to figures reported elsewhere in the literature; report the one that matches how your comparison field typically reports it, and say explicitly which one you used.
Is effect size the same thing as clinical or practical significance?
Related but not identical. Effect size is a statistical magnitude; clinical or practical significance asks whether that magnitude matters in a real-world context, often assessed against a pre-specified Minimal Clinically Important Difference (MCID) for the specific outcome measure and condition, rather than against a generic statistical benchmark.
See also: Effect Size in the CASRAI Dictionary, p-value, confidence interval, and meta-analysis, where standardized effect sizes are the common currency pooled across studies.
