Tag: p-hacking

  • Questionable Research Practices vs Misconduct: Where Institutions Draw the Line

    Questionable research practices (QRPs) are not research misconduct, and treating them as identical is where institutional triage goes wrong. Misconduct — fabrication, falsification, and plagiarism (FFP) — requires intent and a formal proof standard; QRPs such as p-hacking, HARKing, and selective reporting occupy a wider grey zone that most institutions route through a separate, lower-intensity process.

    A questionable research practice is a research or reporting behaviour that deviates from rigorous methodological or ethical norms without meeting the legal definition of fabrication, falsification, or plagiarism. The distinction matters because it determines which procedural track — and which evidentiary burden — an institution applies to a complaint.

    What counts as a questionable research practice?

    QRPs cluster around the design, analysis, and reporting stages of a study. The most cited taxonomy comes from an incentivised self-report survey published in Psychological Science (John, Loewenstein & Prelec, 2012), which found admission rates for practices such as selectively reporting studies “that worked” approaching 50% — far higher than admitted rates of outright data fabrication, which stayed under 2%. That gap is itself the argument for treating QRPs as a distinct governance problem rather than a rare edge case of misconduct.

    • P-hacking (data dredging): running multiple analyses or exclusion criteria until a result reaches statistical significance, then reporting only that version.
    • HARKing: presenting a post hoc, exploratory finding as though it were the pre-specified hypothesis.
    • Selective/cherry-picked reporting: omitting non-significant outcomes, conditions, or measures from the published record.
    • Salami slicing: splitting one dataset into the minimum publishable units to inflate output count.
    • Honorary or ghost authorship: crediting non-contributors or omitting contributors, which is why standardised contributor role disclosure matters for accountability.
    • Inadequate data retention: failing to keep raw data, code, or protocols available for verification.

    How do QRPs differ from research misconduct?

    Under US federal policy (42 CFR Part 93, administered by the Office of Research Integrity), research misconduct is defined narrowly as fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting results. Three conditions must all be met: a significant departure from accepted practices, intentional/knowing/reckless conduct, and proof by a preponderance of the evidence. The same regulation explicitly excludes “honest error” and “differences of opinion” from the misconduct definition — which is precisely the space most QRPs occupy.

    UK guidance frames the same boundary differently but reaches a similar conclusion. UKRIO’s 2023 guidance by Simon Kolstoe, “Defining the Spectrum of Questionable Research Practices,” describes a continuum running from honest error and sloppiness through to QRPs and, at the far end, fabrication, falsification, and criminality — with no single fixed cut-off point.

    Dimension Questionable research practice Research misconduct (FFP)
    Intent required Not required — may be unintentional, driven by pressure or ignorance Must be intentional, knowing, or reckless
    Governing definition No single legal definition; institutional/disciplinary norms 42 CFR Part 93 (US); institutional misconduct policy (UK/EU)
    Standard of proof Not applicable — assessed for pattern/severity, not adjudicated Preponderance of the evidence
    Typical examples P-hacking, HARKing, selective reporting, salami slicing Fabricated data, altered images, verbatim plagiarism
    Usual institutional track Research governance, retraining, correction, departmental review Formal inquiry and investigation with findings and sanctions

    How do institutions triage an allegation?

    Under the ORI model that most US research institutions adopt, and which UK bodies broadly mirror in structure, a complaint moves through staged gates rather than a single up-or-down decision.

    1. Assessment: a Research Integrity Officer performs an initial review to establish whether, if the allegation were true, it would meet the FFP definition and falls within the institution’s jurisdiction. Concerns that describe sloppiness, honest error, or a QRP without evidence of intent are typically redirected here rather than escalated.
    2. Inquiry: a limited fact-finding step, conducted under 42 CFR Part 93 within 60 calendar days unless circumstances clearly warrant longer, to decide whether a full investigation is warranted.
    3. Investigation: a formal, evidence-gathering process triggered only once the inquiry finds a sufficient basis, applying the preponderance-of-evidence standard and producing a written report with findings.

    QRPs that do not clear the assessment gate are not dismissed outright — the National Academies of Sciences, Engineering, and Medicine’s 2017 report Fostering Integrity in Research proposed the term “detrimental research practices” specifically to argue that institutions need a formal, non-misconduct track for exactly this category, rather than either ignoring it or misapplying the misconduct process to it. COPE’s flowcharts for publishers follow the same logic: suspected fabrication routes to a formal investigation referral, while methodological concerns typically route to correction or expression-of-concern mechanisms first.

    Why does the distinction change institutional response?

    Misapplying the misconduct process to a QRP over-penalises ambiguous conduct and consumes scarce Research Integrity Officer capacity on cases that cannot meet the intent standard. Under-applying it — treating a concealed fabrication as “just” a QRP — lets a genuine breach evade the preponderance-of-evidence process entirely. Accurate triage at the assessment stage is therefore the single highest-leverage decision point in the entire allegation-handling pipeline, and it depends on evaluators being able to name the difference precisely rather than treating “research integrity concern” as one undifferentiated category.

    This is also why standardised terminology for research integrity concepts, and consistent research administration processes for logging and routing complaints, reduce inconsistent outcomes across departments within the same institution.

    Answer-first questions on QRPs and misconduct

    What is the difference between questionable research practices and research misconduct?

    Research misconduct is fabrication, falsification, or plagiarism committed intentionally, knowingly, or recklessly and proven by a preponderance of evidence under frameworks such as 42 CFR Part 93. QRPs are a broader, less severe category — practices like p-hacking or HARKing — that may be unintentional and do not meet the FFP definition.

    What are examples of questionable research practices?

    Common QRPs include p-hacking, HARKing, selective or cherry-picked reporting, salami slicing of datasets into multiple papers, inadequate record-keeping, and honorary or ghost authorship. None of these automatically constitutes misconduct unless deliberate concealment or fabrication can be demonstrated.

    Is p-hacking considered research misconduct?

    Generally no. P-hacking is classified as a questionable research practice because it typically reflects flawed analytical judgement or incentive pressure rather than deliberate fabrication. It can escalate to misconduct only if a researcher knowingly falsifies or conceals the analytic process to deceive reviewers.

    Who decides whether an allegation is misconduct or a questionable practice?

    A Research Integrity Officer (or institutional equivalent) makes this call at the assessment and inquiry stages, evaluating whether alleged conduct could meet the FFP definition before any formal investigation begins. This gatekeeping decision determines which procedural track and evidentiary standard apply.

    Where institutional policy is heading

    Recent scholarship — including a 2026 classification study in a leading research-ethics journal — continues to push toward more granular, ranked taxonomies of QRPs rather than a single undifferentiated label, reflecting growing recognition that “questionable research practice” spans behaviours of very different severity. For institutions, the practical implication is unchanged: allegation-handling policies need an explicit, documented triage step that distinguishes FFP-eligible conduct from the wider QRP category before any case enters a formal investigation track. Institutions that skip this step either over-invest in low-severity concerns or under-investigate the ones that matter.