Direct comparison
P-hacking vs HARKing: Key Differences
P-hacking manipulates analysis until p<0.05; HARKing rewrites hypotheses after seeing results. Compare both and how pre-registration defends against each.
Side-by-side comparison
| Dimension | P-hacking | HARKing |
|---|---|---|
| What it manipulates | The data analysis - which tests are run, when data collection stops, which exclusions are applied | The stated hypothesis - rewriting it after the results are known |
| What stays fixed | The hypothesis | The analysis that was actually run |
| Origin of the term | Popularized by Simmons, Nelson & Simonsohn, 'False-Positive Psychology' (2011) | Coined by Norbert Kerr, 'HARKing: Hypothesizing After the Results are Known', Personality and Social Psychology Review (1998) |
| Typical mechanism | Multiple comparisons, optional stopping, selective covariate inclusion, outlier exclusion | Post-hoc hypothesis presented as a priori in the introduction/discussion |
| Where it happens in the paper | Methods/results - the analysis pipeline itself | Introduction/discussion - the narrative framing |
| Detectable via | Excess of p-values just below .05 across the literature (p-curve analysis); discrepancy between pre-registration and reported analysis | Discrepancy between a pre-registered hypothesis and the published introduction; implausibly precise 'predicted' post-hoc findings |
| Primary structural defense | Statistical Analysis Plan (SAP) / pre-specified analysis, Registered Reports | Pre-registration of hypotheses, Registered Reports |
| Classified as | Questionable research practice (QRP), not research misconduct | Questionable research practice (QRP), not research misconduct |
| Can co-occur? | Yes - a p-hacked result is often also HARKed into the introduction as a predicted finding | Yes - see above |
Common questions
FAQ
Can a single study involve both p-hacking and HARKing?+
Yes. A researcher may run multiple undisclosed analyses until one is significant (p-hacking), then write that specific comparison into the paper's introduction as the original hypothesis (HARKing). The two are frequently discussed together for exactly this reason.
Is HARKing always dishonest?+
It is widely treated as a questionable research practice because it misrepresents the confirmatory/exploratory status of a finding, even when the underlying data and analysis are reported accurately. The problem is the false impression of a priori prediction, not fabricated results.
Does pre-registration eliminate both problems completely?+
It substantially reduces both by creating a timestamped, checkable record, but it is not self-enforcing - a pre-registration can be ignored or a study can be run without pre-registering an exploratory arm. Registered Reports, which add independent peer review of the pre-registered protocol, are generally considered a stronger defense than self-filed pre-registration alone.
Is exploratory analysis itself a problem?+
No. Exploratory analysis is a legitimate part of research. The issue is disclosure - exploratory findings should be labelled as exploratory, not presented as confirmatory tests of a hypothesis that was actually formed afterward.
