Examples
Worked examples
- Is an instance
Adding covariates one at a time until the headline coefficient becomes significant.
- Is an instance
Excluding outliers using a criterion chosen after seeing the result.
Counter-examples
Looks similar, but isn't
- Not an instance
A pre-specified analysis that happens to be non-significant and is reported.
- Not an instance
A robustness check pre-specified in the analysis plan.
Editorial commentary
P-hacking encompasses a spectrum of behaviours from outright fabrication (rare) to subtle, often unconscious selection among analytic forks (common). Indicators include excess of p-values just under 0.05 in literature meta-analyses (Simonsohn's p-curve), and discrepancies between pre-registered and reported analyses. Pre-registration is the principal structural defence.
References
- Simmons, Nelson, Simonsohn, 'False-positive psychology' (Psychological Science, 2011); Simonsohn, Nelson, Simmons, 'P-curve' (Journal of Experimental Psychology: General, 2014).
Also known as
p-fishing · significance chasing
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="P-hacking"
vocab-term-identifier="https://casrai.org/dictionary/term/p-hacking" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"name": "P-hacking",
"identifier": "https://casrai.org/dictionary/term/p-hacking",
"description": "The practice of selectively reporting or adjusting analytical choices in order to obtain a statistically significant p-value, typically below the conventional 0.05 threshold.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/reproducibility-and-computational-research/",
"url": "https://casrai.org/dictionary/term/p-hacking",
"sameAs": [
"p-fishing",
"significance chasing"
],
"license": "https://creativecommons.org/licenses/by/4.0/"
}
