Written and maintained by CASRAI Editorial Board
Last updated
Short answer
A psychology meta-analysis is a quantitative research synthesis that statistically pools numerical effect sizes across independent empirical studies testing a common psychological hypothesis. Guided by the APA Meta-Analysis Reporting Standards (MARS) and PRISMA 2020 frameworks, it calculates weighted average effect sizes, quantifies between-study heterogeneity, and formally evaluates publication bias.
Definition and scope
In psychological science, individual empirical studies often yield mixed, conflicting, or statistically fragile findings due to sample size constraints, measurement variance, or sampling error. Understanding how evidence accumulates across studies is foundational to psychology research methodology. Rather than relying on qualitative impressions, a meta-analysis extracts standardized quantitative metrics—such as Cohen’s d, Hedges’ g, Pearson’s correlation coefficient (r), or odds ratios—from eligible investigations to determine the direction, magnitude, and boundary conditions of psychological phenomena.
The scope of a psychological meta-analysis encompasses both primary efficacy evaluations (such as comparing cognitive behavioral therapy to control conditions in clinical trials) and observational associations (such as personality traits predicting occupational tenure). Modern meta-analyses in psychology must adhere to rigorous procedural reporting defined by the American Psychological Association (APA) Publications and Communications Board Task Force in their Journal Article Reporting Standards (JARS-MARS; Appelbaum et al., 2018).
How it works or how it is used in practice
Conducting a quantitative meta-analysis in psychological research follows a multi-stage workflow designed to minimize investigator bias and ensure reproducibility:
- Systematic literature search: Investigators formulate explicit inclusion and exclusion criteria and query major bibliographic databases (including PsycINFO, PubMed, and Web of Science) alongside pre-print servers and grey-literature repositories to mitigate publication bias.
- Screening and study selection: Retrieved records are screened by title, abstract, and full text against pre-specified eligibility criteria, and the flow of records through each stage is documented in a PRISMA 2020 flow diagram.
- Data extraction and coding reliability: Independent coders extract sample characteristics, study design features, intervention parameters, and raw statistics required for effect size transformation. Coding reliability is reported so readers can judge how consistently studies were coded.
- Effect size calculation and standardization: Primary outcome data are converted to a common standardized metric. Continuous psychological outcomes typically utilize Hedges’ g, which corrects Cohen’s d for upward sample-size bias in small participant cohorts.
- Statistical modeling: Studies are synthesized using either a fixed-effect model or a random-effects model. A random-effects model fits when true effects may differ across studies, as they often do when participant populations, measures, and conditions vary (Borenstein et al., 2010). Weights are assigned inversely proportional to total variance:
wi* = 1 / (vi + τ2)
where vi is the within-study sampling variance and τ2 (tau-squared) represents between-study heterogeneity. - Heterogeneity evaluation: Investigators calculate Cochran’s Q and the I2 statistic. In-depth analysis often requires heterogeneity modeling in meta-analysis to determine whether variance stems from methodological moderators through subgroup analysis or meta-regression.
- Publication bias diagnostics: Because studies with statistically significant outcomes are more likely to be published, researchers apply funnel plots, Egger’s linear regression, trim-and-fill procedures, and p-curve analysis to evaluate evidentiary value.
Types or related concepts
Meta-analysis sits alongside several other ways of synthesizing evidence in psychological research.
| Synthesis methodology | Primary data input | Synthesis approach | Key limitation |
|---|---|---|---|
| Narrative review | Qualitative study findings and author commentary | Subjective thematic integration without formal search protocols | High susceptibility to author selection bias and confirmation bias |
| Systematic review | Empirical studies meeting pre-registered criteria | Structured qualitative synthesis and risk-of-bias auditing | Does not calculate pooled numerical effect size estimates |
| Quantitative meta-analysis | Standardized numerical effect sizes across studies | Weighted statistical pooling (fixed-effect or random-effects models) | Distorted if underlying literature suffers severe publication bias |
| Individual participant data (IPD) meta-analysis | Raw participant-level trial data from investigators | One-stage or two-stage hierarchical regression modeling | Intensive data-sharing agreements and participant privacy barriers |
Researchers deciding between qualitative and statistical synthesis can consult the comparative framework in systematic review vs meta-analysis.
Where it matters for psychology research and practice
Meta-analysis is used across clinical, experimental, and organizational psychology to combine results from many studies of the same question.
Within experimental and cognitive psychology, meta-analyses resolve longstanding replication debates. In the wake of the broader psychology replication crisis, single laboratory findings no longer command unquestioned authority. Meta-analytic pooling shows whether an effect holds up across laboratories once publication bias is examined. Pooled effect sizes can also inform planning for open-science formats such as registered reports.
Common misunderstandings
- Treating fixed-effect and random-effects models as interchangeable: A fixed-effect model assumes one identical true effect size across all studies and that observed variation is solely sampling error. A random-effects model assumes true effect sizes vary across study populations and protocols. In psychology, assuming a single true effect is rarely realistic.
- Ignoring publication bias: Rosenthal (1979) described the “file drawer problem”: studies with null results often go unpublished. A meta-analysis that synthesizes only published literature risks overestimating psychological effect sizes.
- Combining fundamentally incompatible constructs: Known as the “apples and oranges” criticism, pooling studies that measure theoretically distinct psychological constructs into a single omnibus effect produces misleading conclusions. Clear operational definitions of each construct are part of good reporting under APA MARS.
- Believing meta-analysis fixes poor primary designs: Often called the “garbage in, garbage out” principle, aggregating statistically underpowered or methodologically flawed primary studies cannot generate valid scientific inferences.
Frequently asked questions
What is a meta-analysis study in psychology?
A meta-analysis study in psychology is a secondary empirical research project that uses quantitative statistical procedures to combine numerical findings from multiple published and unpublished studies addressing the same psychological hypothesis.
What is a good number of studies for a meta-analysis?
A meta-analysis can be computed with as few as two studies, but estimates of between-study variance (τ2) and moderator analyses become less stable as the number of studies falls. No single count is a universal rule; the number needed depends on the question and the precision wanted, and the APA JARS-MARS standards ask authors to report how many studies were included and how they were found.
What is another word for meta-analysis?
Meta-analysis is often described as a quantitative research synthesis or a pooled analysis of effect sizes.
What is the purpose of a meta-analysis in psychology research?
The primary purpose of a meta-analysis in psychology research is to establish a precise estimate of an effect size, resolve conflicting empirical findings across individual experiments, identify moderating variables, and assess the robustness of findings against publication bias.
What are the four types of research methods in psychology?
Textbooks group psychology research methods in different ways, so there is no single official list of four. Commonly named approaches include experimental research (manipulating a variable), correlational research (measuring associations), and descriptive or observational research (such as surveys and case studies). Meta-analysis is a different kind of method: it synthesizes results from studies that others have already carried out.
What are the criticisms of meta-analysis?
The main criticisms of meta-analysis include the “file drawer problem” (publication bias favoring significant results), the “apples and oranges” problem (pooling studies with disparate constructs or measures), and the “garbage in, garbage out” critique (synthesizing low-quality or poorly controlled primary studies).
Sources
- Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191 (checked 2026-11-07).
- Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71 (checked 2026-11-07).
- Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638 (checked 2026-11-07).
- Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2010). A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1(2), 97–111. https://doi.org/10.1002/jrsm.12 (checked 2026-11-07).
