NIH Rigor and Reproducibility policy

The NIH Rigor and Reproducibility policy is the set of US National Institutes of Health policies, effective from 2016, requiring applicants and grantees to address scientific premise, scientific rigour, biological variables (including sex as a biological variable), and authentication of key biological and chemical resources in grant applications. Unlike the other terms in this cluster, it is a funder compliance requirement, not a type or measure of reproducibility itself — it is the policy mechanism NIH uses to make applicants demonstrate attention to the reproducibility problems those other terms describe.

What it requires, concretely

NIH SF424 application instructions require four specific elements across most institutes and centers: a stated scientific premise (the strength of prior data the proposal builds on, and any known weaknesses); a plan for scientific rigour (adequate sample size, appropriate controls, blinding, and statistical approach); explicit consideration of relevant biological variables, most notably sex as a biological variable in vertebrate animal and human studies unless a strong justification for single-sex study is given; and a plan for authentication of key biological and/or chemical resources (cell lines, antibodies, and other reagents whose identity or purity can drift or be mislabelled). These are now built into standard review criteria at the application stage and are followed up in progress reporting, not just requested once.

Why it exists

The policy was NIH’s direct response to a body of preclinical-reproducibility literature — most prominently Begley & Ellis (2012), which reported that Amgen scientists could not replicate the majority of a sample of “landmark” preclinical cancer studies they attempted to build on, and Freedman et al. (2015), which estimated the cumulative cost of irreproducible preclinical research in the US. The policy targets the empirical and methodological weaknesses (underpowered studies, unblinded scoring, unauthenticated cell lines) that literature identified as recurring causes, rather than mandating any specific reproducibility check on the resulting publication.

Sources

NIH Notice NOT-OD-15-103, “Enhancing Reproducibility through Rigor and Transparency”; Begley & Ellis, Nature (2012); Freedman et al., PLOS Biology (2015).

Who Owns This in a Research Administration Office

Because the four elements are folded into the application itself rather than submitted as a separate compliance form, responsibility for them is usually split across the people already touching the application: the PI drafts the scientific-premise, rigor, and biological-variables content as part of the Research Strategy, while a pre-award research administrator or grants specialist typically checks, before submission, that all four elements are addressed somewhere in the application rather than assuming the PI’s draft covers them. At the institution level, an Office of Research Integrity or a research-compliance office generally does not review individual applications for this policy specifically — the review happens inside NIH’s own peer-review process, at Factor 2 (Rigor and Feasibility) under the current Simplified Review Framework — but the same office often maintains institutional guidance and training materials PIs and their administrators draw on.

What a Reviewer Actually Checks

Study section reviewers are instructed to evaluate rigor and reproducibility as part of the Approach criterion (Factor 2 under the current framework), looking for concrete, specific commitments rather than boilerplate language. In practice this means checking that the scientific premise cites the actual prior data the proposal builds on and is honest about its limitations; that the rigor plan states real sample sizes, controls, and blinding procedures rather than asserting “rigorous methods will be used”; that biological-variables language either addresses both sexes or gives a specific, study-relevant justification for a single-sex design (a generic statement that the disease “primarily affects one sex” is treated differently than a citation-backed rationale); and that the authentication plan names the actual method (e.g., short tandem repeat profiling for cell lines, a specific vendor certificate of analysis for antibodies) rather than a vague assurance that resources will be authenticated.

Checking this against the current guidance

What counts as an adequately “strong justification” for a single-sex study, versus language a reviewer will flag as generic, is a judgment call study sections make case by case rather than something this policy summary can settle in the abstract.

Ask CASRAI: If a study proposes to use only male animals or human subjects, what specific “strong justification” language does NIH expect in the sex-as-a-biological-variable section to avoid a reviewer flag, and does that bar differ between a new competing application and a resubmission responding to that exact critique?

It searches CASRAI’s indexed corpus of research-administration guidance and cites the passage behind each claim, so you can open the source and check it rather than take its word — and it says so when the corpus does not cover something instead of guessing. Two questions a day are free while you are signed out, no account and no card. Everything CASRAI publishes stays free to read.

Frequently Asked Questions

Does this policy apply to non-competing progress reports, or only to new and competing applications?

The rigor and reproducibility elements are evaluated at the application stage, as part of peer review under the Approach/Rigor and Feasibility criterion. Non-competing progress reports (RPPRs) are not separately scored against these four elements the way a competing application is, though an institution’s own internal tracking may still expect a PI’s ongoing work to reflect the rigor commitments made in the original application.

What happens if a reviewer decides the rigor and reproducibility content is inadequate?

There is no separate pass/fail gate distinct from the rest of peer review — weaknesses in scientific premise, rigor, biological variables, or authentication feed into the Approach score (or Factor 2 under the Simplified Review Framework) the same way any other methodological weakness would, and can be cited directly in the summary statement as a concern affecting the overall impact score.

Does the sex-as-a-biological-variable expectation apply to cell-line and in-vitro work, or only whole-animal and human studies?

NIH’s guidance extends the expectation to vertebrate animal and human studies specifically; it does not impose an equivalent requirement on cell lines or other non-vertebrate model systems in the same binding way, though some ICs and study sections still expect applicants to note the sex of origin for human-derived cell lines where it’s known and relevant.

Who at an institution is responsible for confirming a competing application addresses all four elements before submission?

NIH does not assign this to a specific institutional role by regulation — it falls to whatever combination of the PI, a departmental grants administrator, and a central sponsored-programs pre-award reviewer an institution’s own internal submission-readiness process uses. Because NIH does not accept an application missing this content administratively (the check happens through peer review, not a technical-compliance screen), institutions have an incentive to catch gaps before submission rather than rely on NIH to flag them.