Direct comparison
Frequentist vs Bayesian Clinical Trials
Compare frequentist (p-values, fixed sample size) vs. Bayesian (priors, adaptive interim decisions) statistics in clinical trials, plus FDA guidance on each.
Side-by-side comparison
| Dimension | Frequentist | Bayesian |
|---|---|---|
| Core question asked | How likely is data this extreme if the null hypothesis (no effect) were true? | Given the prior and the observed data, what is the probability distribution of the treatment effect? |
| Treatment effect treated as | A fixed, unknown constant | An unknown quantity with its own probability distribution |
| Role of prior information | Not formally incorporated into the primary analysis | Explicit prior distribution, combined with trial data via Bayes' theorem |
| Key output | p-value and confidence interval | Posterior distribution and posterior probability statements (e.g. "92% probability the effect exceeds X") |
| Sample size | Fixed in advance, powered for a target effect size | Can be fixed or formally re-estimated at interim analyses under pre-specified rules |
| Interim looks at data | Require pre-specified group-sequential correction (e.g. O'Brien-Fleming) or Type I error inflates | Built into the design via simulated operating characteristics; supports stop-early-for-efficacy/futility natively |
| Typical use | Confirmatory Phase III trials, most regulatory submissions | Platform/basket/umbrella (master-protocol) trials, device trials, early-phase dose-finding, interim monitoring |
| Key regulatory guidance | ICH E9, ICH E9(R1) estimand addendum, FDA Adaptive Designs guidance (group-sequential provisions) | FDA "Guidance for the Use of Bayesian Statistics in Medical Device Clinical Trials" (2010, CDRH/CBER); FDA Adaptive Designs guidance (Bayesian adaptive provisions) |
| Main practical risk | Unplanned repeated significance testing inflates the real false-positive rate | An overly optimistic or poorly justified prior can bias results toward the desired conclusion |
| Statistical analysis plan (SAP) requirement | Fixed analysis method and alpha-spending strategy specified before unblinding | Prior distribution, its justification, and the full adaptation/decision rules specified and simulated before the trial opens |
Common questions
FAQ
Can a single trial use both frequentist and Bayesian methods?+
Yes. It's common for a platform or adaptive trial to use Bayesian methods for interim monitoring, arm-dropping, or adaptive randomization, while the final confirmatory analysis is reported using a frequentist-calibrated threshold, or a Bayesian posterior-probability rule whose overall Type I error rate has itself been calibrated via simulation to match what regulators expect from a frequentist design.
Is Bayesian statistics acceptable to the FDA?+
Yes, for both drugs/biologics and devices — FDA has published explicit Bayesian guidance for medical device trials since 2010 and covers Bayesian adaptive designs in its Adaptive Designs for Clinical Trials of Drugs and Biologics guidance. The framework choice must be pre-specified and justified in the statistical analysis plan; it is not restricted to one product type.
Why are most confirmatory Phase III trials still frequentist?+
Familiarity, regulatory precedent, and simplicity of review are the main reasons — a fixed sample size and a single pre-specified hypothesis test are comparatively easy for sponsors, statisticians, and regulators to plan around and audit. Bayesian designs are gaining ground fastest where a fixed, single-comparison frequentist design doesn't fit the trial structure, most notably platform trials with an open-ended, evolving set of treatment arms.
Does using a Bayesian prior mean the analysis is less objective?+
Not inherently — a non-informative (vague) prior lets the current trial's data dominate the posterior, producing conclusions very close to a frequentist analysis. The objectivity question arises specifically with informative priors, which is why regulators expect the prior's source and justification documented, and often expect a sensitivity analysis across a range of priors including a non-informative one.
