Power Analysis and Sample Size Calculation: A Complete Guide

Power analysis is the calculation that turns a study’s expected effect into a required sample size, and sample size calculation is the same calculation read the other way round — both start from the same four interlocking quantities and both exist to answer one question before data collection begins: is this study large enough to detect the effect it is designed to find, without being so large that it wastes participants, time, or funding? A study that is underpowered risks a false negative (and, less obviously, inflates the size of any effect it does detect); a study that is needlessly overpowered exposes more participants than necessary to burden or risk for no added scientific value. Both are avoidable with the same arithmetic, done before the study starts rather than after it ends.

The four quantities, and why fixing three determines the fourth

Every power calculation is a relationship among exactly four numbers. Fix any three and the fourth is no longer a free choice — it is determined by the other three:

• Effect size — the magnitude of the difference, association, or relationship the study is designed to detect, expressed in a standardised metric (Cohen’s d for a mean difference, r for a correlation, f² for regression, w or Cohen’s h for proportions).
• Alpha (α) — the significance threshold, i.e. the tolerated Type I error rate (false positive). Conventionally 0.05, sometimes tightened for multiple comparisons.
• Power (1 − β) — the probability of correctly detecting the effect if it truly exists, i.e. one minus the tolerated Type II error rate (false negative). Convention is 0.80, with 0.90 increasingly expected in clinical and high-stakes designs.
• Sample size (N) — the number of observations, participants, or units.

Because these four are mathematically linked through the sampling distribution of the test statistic, a power analysis is really just solving one equation for whichever quantity is unknown. In the design phase, that unknown is almost always N: the researcher supplies a plausible effect size, a conventional alpha, and a target power, and solves for the sample size needed to achieve them. This is why “power analysis” and “sample size calculation” describe the same decision from two directions rather than two different procedures — a guide, protocol, or grant application that says it performed “a power analysis to determine sample size” is using the terms correctly and interchangeably.

The practical consequence of the four-quantity relationship is that every choice has a visible cost. Demanding a smaller effect size be detectable, or demanding higher power, or tightening alpha, all increase the required N, usually non-linearly — halving the effect size you want to detect roughly quadruples the sample size needed, all else equal. This is the arithmetic reviewers and ethics committees are actually checking when they ask a researcher to justify a sample size: not “is bigger better,” but “do these four numbers cohere, and is the effect-size assumption defensible.”

A priori vs. post-hoc power — and why “observed power” is not a fourth thing to report

The much more useful and defensible mode is a priori power analysis: performed before data collection, using an effect size decided in advance, to determine the N the study will recruit. Everything above describes a priori analysis, and it is what funders, IRBs/ethics committees, and pre-registrations expect to see.

Post-hoc (or “observed”) power analysis is performed after a study has already been run, plugging the effect size actually observed in the collected data back into the power formula to compute what power the completed study “had.” This is widely and specifically criticised in the methodological literature as circular and uninformative. Hoenig and Heisey’s frequently cited 2001 paper in The American Statistician, “The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis,” makes the core point precisely: for a fixed alpha, observed power is a deterministic function of the observed p-value alone. A non-significant result will always correspond to a low observed power (below 50%), and a significant one to a higher observed power — the calculation adds no information beyond what the p-value already conveyed, no matter what the true effect actually is. Reporting observed power as if it independently validated (or excused) a null result is a mathematical tautology dressed as an analysis.

That distinction is itself information gain worth stating plainly on a page like this: if a reviewer or manuscript asks you to report the power your completed study “achieved,” the methodologically correct response is not to compute it from your own observed effect size, but either (a) to report the a priori power calculation you used to set the original sample size, or (b) if none was done, to report a confidence interval around the observed effect and discuss what range of true effects the study’s actual N was capable of distinguishing from zero — which is the sensitivity-analysis approach most current methods guidance recommends in place of post-hoc power.

Choosing a plausible effect size with no pilot data

The single hardest input to justify honestly is the effect size, because it is the one number in the equation that is unknown by definition — if you already knew the true effect, you would not need the study. In descending order of defensibility, common approaches are:

1. A systematic review or meta-analysis of prior studies on the same or a closely related question. A pooled effect estimate, ideally with its confidence interval, is the strongest available basis and the one reviewers find most persuasive. If a synthesis of similar interventions or associations already exists, cite it directly rather than a single prior study, which may be an outlier.
2. A single closely comparable prior study, used cautiously — single-study effect sizes are noisy and, per the discussion above, are themselves subject to inflation when the original study was underpowered (a significant result from a small study systematically overestimates the true effect, a pattern sometimes called the “winner’s curse” in this context).
3. Your own pilot or feasibility data, if available — useful for feasibility and protocol refinement, but funders including NIH explicitly caution against treating a small pilot’s effect-size estimate as a reliable input to a full-scale power calculation, precisely because pilot studies are themselves underpowered to estimate an effect size with any precision.
4. Conventional benchmark effect sizes, when no field-specific estimate exists at all. Jacob Cohen’s widely taught conventions define “small,” “medium,” and “large” standardised effects (e.g. d = 0.2, 0.5, 0.8 for mean differences; r = 0.1, 0.3, 0.5 for correlations) as a last-resort anchor, not a substitute for field knowledge. Powering a study to detect only a “medium” effect because no better estimate was available is defensible; doing so when better evidence exists and was ignored is not.
5. The smallest effect size of practical or clinical interest — rather than guessing what effect probably exists, decide what effect would actually matter (clinically, practically, theoretically) if it were true, and power the study to detect that. This sidesteps the guessing problem entirely and is increasingly preferred in pre-registered and clinical designs because it ties the sample size to a substantive judgement rather than a statistical convention.

Whichever approach is used, the justification belongs in the methods section or protocol in enough detail that a reader could reproduce the calculation: the effect size used and its source, alpha, target power, the specific statistical test the calculation was built around (a power calculation for a t-test is not interchangeable with one for a chi-square test or a mixed model), and any adjustment for expected attrition or non-response.

Worked example: powering a two-group comparison

Consider a study comparing a new study-skills intervention against a control group on a continuous outcome (e.g. a validated academic self-efficacy scale), analysed with an independent-samples t-test.

1. Effect size. No pilot data exists. A recent meta-analysis of comparable educational interventions reports a pooled standardised mean difference around d = 0.35 — a small-to-medium effect by Cohen’s conventions. The researcher adopts d = 0.35 as the target, citing the synthesis.
2. Alpha. Set at the conventional two-tailed 0.05.
3. Power. Set at 0.80, the field’s conventional minimum, though the researcher notes 0.90 was considered and rejected only because it would have pushed recruitment beyond feasible capacity — itself a judgement worth stating in the protocol.
4. Solve for N. Entering d = 0.35, α = 0.05 (two-tailed), power = 0.80 into a two-group independent-samples t-test calculation returns a required sample size of approximately n = 129 per group (258 total). (Working the same relationship by hand: for a two-sample t-test the approximate total N is N ≈ 2 × (zα/2 + zβ)² / d²; with z_0.025 ≈ 1.96 and z_0.20 ≈ 0.84, that is 2 × (2.80)² / 0.35² ≈ 2 × 7.84 / 0.1225 ≈ 128 per group, consistent with the exact software output.)
5. Adjust for attrition. The study design anticipates roughly 15% attrition between baseline and follow-up. Inflating 258 by that expected loss (258 ÷ 0.85) gives a recruitment target of approximately 304 participants, split evenly across arms.
6. Justification for the reviewer/ethics committee. The protocol states: the effect size (d = 0.35) and its source (named meta-analysis), alpha (0.05, two-tailed), power (0.80), the resulting analytic N (258) and the attrition-inflated recruitment target (304), and the specific test the calculation assumed (independent-samples t-test). That is a complete, reviewable justification — every number in it can be checked or challenged, which is precisely what distinguishes a defensible sample-size statement from an assertion that “an adequate sample was recruited.”

This illustrative walk-through uses a generic scale and a representative published effect size range rather than a specific named study, to demonstrate the calculation mechanics without implying a specific real trial was conducted as described.

The tools researchers actually use

G*Power is a free, widely used standalone application (developed at Heinrich Heine University Düsseldorf) covering power analysis for t-tests, F-tests (ANOVA, regression), chi-square tests, z-tests, and several exact and correlation-based tests, with a graphical interface that lets a researcher solve for N, for power, or for the minimum detectable effect size interchangeably — useful for exploring how sensitive the required N is to a given assumption before committing to it.

The R package pwr implements the same family of calculations (based on Cohen’s conventions) as scriptable functions — pwr.t.test(), pwr.anova.test(), pwr.r.test(), pwr.chisq.test(), and related functions — which is often preferred where the calculation needs to be reproducible alongside the rest of an analysis pipeline, version-controlled, or adapted programmatically (for example, looping the calculation across a range of plausible effect sizes to build a sensitivity table for a protocol appendix). Both tools implement the same underlying statistical relationships; the choice between them is workflow, not accuracy.

Frequently asked questions

Is a power analysis required for a qualitative study?

No. Power analysis is a feature of frequentist hypothesis testing on quantitative data. Qualitative sample sizes are instead typically justified through concepts like information power or thematic saturation, which answer a different question (has data collection reached the point of diminishing new insight) rather than a statistical detection threshold.

What if the required sample size is not feasible to recruit?

Options include revising the design to increase power without increasing N (a within-subjects or paired design generally requires fewer participants than a between-subjects one for the same effect), narrowing the research question to the smallest effect of genuine practical interest, extending the recruitment window or adding sites, or transparently reframing the study as a pilot/feasibility study rather than a definitively powered confirmatory one — and saying so explicitly, rather than running an underpowered confirmatory study and reporting only the significant results that emerge.

Does a higher sample size ever hurt a study?

Statistically, no — more data narrows confidence intervals and increases power. Practically and ethically, yes: recruiting substantially more participants than the effect of interest requires exposes additional people to any burden or risk of the study, and in clinical research this is a recognised ethical concern (a trial that is unjustifiably overpowered enrols participants who did not need to be enrolled to answer the question). This is one reason ethics committees ask for the calculation in both directions, not only to rule out underpowering.

How does this relate to reliability measures like Cronbach’s alpha?

They address different validity questions in the same study. Power analysis concerns whether the sample is large enough to detect the effect of interest; instrument reliability concerns whether the measurement itself is consistent enough to detect anything at all. A well-powered study built on an unreliable instrument still produces a noisy, hard-to-interpret result — see the CASRAI guide on Cronbach’s alpha for how internal-consistency reliability is assessed and reported.

For the wider sequence of design decisions this fits into — from research question through design, sampling, analysis, and reporting — see the CASRAI Research Methods & Statistics hub.