Sampling & Statistical Power
Before data collection begins, a researcher has to answer two linked questions: how many participants or observations are needed, and how will they be selected. This sub-cluster covers power analysis and sample-size calculation — the a priori estimation of how many observations are needed to detect an effect of a given size at a given significance threshold — alongside the sampling methods (simple random, stratified, cluster, convenience, snowball) that determine whether a sample can support the generalization a study wants to make. Underpowered studies and non-representative sampling are among the most common reasons a study fails peer review or fails to replicate, making this foundational, not optional, groundwork.
Guides
Permuted Block Randomization: How It Works, Block Size, and Reporting
How permuted block randomization works: block size, worked 1:1 example, predictability, variable blocks, reporting wording and code in Excel, R and Python.
Transect Sampling: Line and Belt Methods, Formula and Worked Example
Transect sampling: line vs belt transects, the density formula D = n/2wL with every symbol, a worked example and code to compute it in Excel, R or Python.
Sampling With Replacement: Formula and Example
Sampling with replacement returns each unit before the next draw. See the N^n formula, a worked urn example, and R, Python and Excel code.
2 Sample Z Test (Two-Sample Z-Test): Formula, Example and Code
Two-sample z-test for means: formula, a fully worked example (z = 3.09), one- and two-tailed p-values, assumptions, APA report, and code.
Participant Bias: Types, Worked Example and Reporting
Participant bias explained: volunteer and non-response bias, Hawthorne effect, a worked arithmetic example, and how to report it.
Types of Bias in Statistics: Estimator, Selection and Reporting Bias
Types of bias in statistics explained: estimator bias with the formula, plus selection, information, confounding and reporting bias, with worked numbers.
Survey Bias: Types, Real Examples and How to Reduce Each
Survey bias explained by stage: who is asked, who answers, how they answer. Real Pew examples with the arithmetic, and which fix targets which bias.
Bias in an Experiment: Types, Examples and How Design Prevents Each
Bias in an experiment is systematic error. See the five Cochrane RoB 2 sources, the safeguard for each, and a worked example with the arithmetic.
What Is a Good Sample Size for a Quantitative Study? Formula and Worked Example
A good sample size meets your precision or power target. Formulas, worked arithmetic and code for finding it.
What Is Measurement Bias? Definition, Examples and How to Reduce It
Measurement bias is a systematic gap between measured and true values. Learn the definition, examples, a worked calculation and how to reduce it.
What Is Information Bias? Definition, Types, and a Worked Example
Information bias is systematic error from inaccurate measurement or recording. See examples, types, and a worked odds-ratio example with prevention steps.
Two-Sample t-Test Degrees of Freedom: Pooled vs Welch
How to find degrees of freedom for a two-sample t-test: the pooled and Welch formulas, a step-by-step worked example, APA reporting and code.
What Is a Random Variable? Definition, Examples and Formula
A random variable assigns a number to a chance outcome. Definition, discrete and continuous examples, formulas and a worked example with arithmetic shown.
How to Find Sample Variance: Formula, Worked Example and Excel, R, Python
Find sample variance with the n-1 formula: a worked example (27.2 / 4 = 6.8) and the Excel, R and Python functions that compute it.
Survivorship Bias: When Only the Winners Get Studied
Survivorship bias is the error of drawing conclusions from only the surviving, successful, or completed cases while the eliminated cases go unrecorded. Covers the WWII bomber example, business and financial-data versions, and how it differs from attrition bias and the healthy worker effect.
Sensitivity Power Analysis: Finding the Minimum Detectable Effect Size
How to compute the minimum detectable effect size (MDES) for a fixed, already-collected sample size, using G*Power’s sensitivity mode or R’s pwr package — and why it replaces post hoc power as the defensible calculation to report.
Sample Size Justification for a Grant Application or IRB Protocol
How to justify a sample size to a grant reviewer or IRB, not just calculate one: the NIH Approach criterion, 45 CFR 46 risk/benefit review, the pilot-study rule of thumb when no effect size exists, and the attrition math reviewers expect to see.
Sample Size for Survival Analysis: Why Events, Not People, Drive Power
Survival-analysis power depends on the number of events, not total enrollment. The Schoenfeld formula, converting events to an enrollment target via accrual and dropout, and a worked example for a grant or protocol.
Multistage Sampling: Stages, Design, and When to Use It
What multistage sampling is, PSU/SSU/USU terminology, how it differs from cluster sampling, real national-survey examples, design effect, and when to use it.
How Many Imputations Do You Need? From “Five Is Enough” to the Quadratic Rule
The old “m=5 is enough” convention, the percentage-of-missing-data rule that replaced it, and von Hippel’s more accurate quadratic rule for choosing how many multiple imputations you actually need — with an independently computed relative-efficiency-loss table.
Probability Proportional to Size (PPS) Sampling: Logic, Selection Method, and a Worked Example
PPS sampling gives larger clusters a proportionally higher chance of selection, producing a self-weighting sample in multistage designs. How it works, a fully worked and independently verified selection example, and why it beats simple random sampling of clusters when cluster sizes vary.
Respondent-Driven Sampling: Reaching Hidden Populations Through Peer-Referral Chains
How respondent-driven sampling recruits hidden populations through seed-and-coupon referral chains, and why the network-size weighting is what makes it more than a convenience sample.
Data Saturation and Information Power: Justifying Qualitative Sample Size
Saturation only tells you sample size was enough after data collection. Malterud’s information-power model — study aim, sample specificity, theory, dialogue quality, analysis strategy — lets you justify it in advance.
Maximum Variation Sampling: Choosing Dimensions and Defending the Range
How to choose variation dimensions that do real evidentiary work (not just demographic ones), build and report the sampling matrix, and write the specific methods-section justification that an achieved range counts as maximum variation rather than convenience diversity.
Sampling Frame: Coverage Error, Frame Construction, and What to Report
What undercoverage and overcoverage actually are, how to build and assess a sampling frame, and the exact disclosure items — per AAPOR’s Standards for Disclosure — a methods section needs so readers can judge generalizability.
Statistical Power Analysis & G*Power Sample Size Guide
How to size a study properly: the interlocking relationship between alpha, power, effect size and sample size, the difference between a priori, sensitivity and post hoc power analysis, and step-by-step G*Power settings for the most common designs.
Attrition Bias: When Dropouts Break Your Sample
Attrition bias occurs when study dropouts differ systematically from completers in a way related to the outcome. Learn to distinguish overall from differential attrition, detect it, meet CONSORT reporting requirements, and mitigate it from design through analysis.
Bootstrapping in Statistics: Resampling for Confidence Intervals
Bootstrapping estimates a statistic’s uncertainty by resampling your own data with replacement thousands of times. This guide walks the procedure on a worked example, compares percentile and BCa confidence intervals, gives practical guidance on how many resamples to run, and covers the specific conditions — small samples, dependent data, extreme quantiles — where the method breaks down.
Random Assignment vs Random Sampling: Two Different Randomisations
Random sampling decides who is studied; random assignment decides what happens to them. One builds external validity, the other internal validity — with worked examples for all four combinations.
Selection Bias: How Your Sample Stops Representing Your Population
Selection bias is the umbrella term for systematic distortion in who or what ends up analyzed — sampling bias, self-selection, non-response, attrition, survivorship, and referral bias are its forms. This guide maps where each enters a study and how to detect and mitigate it.
Law of Large Numbers: What Bigger Samples Actually Buy You
What the law of large numbers actually guarantees, the weak vs. strong forms, a worked convergence demonstration, the gambler’s fallacy misreading, and the practical link between sample size and standard error.
Purposive Sampling: Choosing Cases on Purpose
How purposive sampling works, the named variants (maximum variation, homogeneous, typical case, extreme case, critical case, expert), and how to justify the choice in a methods section.
Effect Size: Choosing, Reporting and Interpreting It
A p-value tells you whether a result is unlikely to be chance. It does not tell you whether the result matters. Effect size does — here is how to choose the right measure for your design, avoid Cohen’s benchmark trap, and report it correctly.
Convenience Sampling: When It’s Defensible and When It Isn’t
Convenience sampling is legitimate for pilots, instrument testing, and hard-to-reach populations — but not for population-level claims. Here’s the bias it introduces and how to report it honestly.
The Central Limit Theorem: Why Sample Means Go Normal
The Central Limit Theorem explains why the distribution of sample means approaches normal as sample size increases, regardless of the population’s shape. Covers the formal conditions, a simulation walkthrough, the n≥30 rule of thumb and its limits, and what the theorem does and does not license.
Quota Sampling: Definition, Method, and Examples
How quota sampling works, how it differs from stratified sampling, proportional vs. non-proportional quotas, worked examples, and its advantages and disadvantages.
Cluster Sampling: Definition, Design Effect, and When to Use It
How cluster sampling works, why it trades precision for cost and feasibility, one/two/multistage designs, sampling weight construction, the design effect (DEFF), correct analysis methods, and when to avoid it.
Sampling Bias: Types, Causes, and How to Detect It
Sampling bias occurs when a sample systematically differs from the population it should represent, distorting results in a direction that more data cannot fix. This guide covers selection, non-response, self-selection, survivorship, healthy-worker, referral, and attrition bias, with real examples and how to detect and report each.
Systematic Sampling: Definition, Method, and When to Use It
Systematic sampling selects a random start between 1 and k, then every k-th unit from an ordered population (k = N/n) — simple to run in the field, but vulnerable to periodicity bias if the list has a hidden cyclical pattern.
Simple Random Sampling: Definition, Steps, and Worked Examples
What simple random sampling actually requires (an equal and independent chance of selection for every population member), the sampling frame problem, how to draw a sample step by step in R, Excel, and Python, worked examples at three scales, and how it compares to stratified, cluster, and non-probability methods.
Power Analysis and Sample Size Calculation: A Complete Guide
How to run a power analysis to calculate sample size: the four interlocking quantities (effect size, alpha, power, N), a priori vs. post-hoc power, choosing an effect size without pilot data, a full worked example, and how to justify N to a reviewer or ethics committee.
