SciScore

SciScore is an automated text-mining tool, built and maintained by SciCrunch, that scans the methods section of a submitted research manuscript and checks it against a fixed set of research-rigor and resource-transparency criteria — things like whether the authors state they randomized subjects, whether the experiment was blinded, whether a sample-size or power calculation is reported, whether sex is reported as a biological variable, and whether antibodies, cell lines, model organisms, plasmids, and software are identified with a resolvable RRID (Research Resource Identifier) and catalog/vendor information. It returns a numeric score and a downloadable report rather than a pass/fail verdict, and it is used by a growing number of journals and publishers as a submission-workflow check — not a replacement for editorial or peer review.

Operational definition

A tool or report is properly called “SciScore” (as opposed to a generic rigor checklist, a plagiarism scanner, or a manual reviewer checklist) if it meets all of the following:

• It is generated by SciCrunch’s own automated text-mining system — not a manually completed checklist like the bare MDAR framework forms, and not a general-purpose grammar or plagiarism tool such as iThenticate.
• It scores the manuscript’s methods section specifically against two components: a Rigor Table (randomization, blinding, sample-size/power analysis, sex as a biological variable, and relevant ethical/animal-welfare approvals) and a Key Resources Table (RRIDs and catalog/vendor identification for antibodies, cell lines, organisms, plasmids, oligonucleotides, and software) — each weighted up to 5 points, for a combined score out of 10.
• Its criteria are drawn from named, citable reporting standards — NIH’s rigor and reproducibility principles for preclinical research, the MDAR (Materials Design Analysis Reporting) checklist, the ARRIVE guidelines for animal research, CONSORT reporting for clinical trials, Cell Press’s STAR Methods format, and RRID standards — not an ad hoc or unpublished rubric.
• It is delivered to the author or editor at a specific point in an actual publisher’s submission or revision workflow (at initial submission, at revision, or both), integrated through a platform such as Editorial Manager or eJournalPress, rather than run as a one-off standalone check with no editorial consequence.

A manuscript that has simply been checked against the MDAR checklist by hand, or that carries RRIDs added because a journal’s author guidelines ask for them, has not necessarily been “SciScored” — the term specifically denotes having gone through SciCrunch’s automated scoring tool.

What SciScore checks for

SciScore was trained on sentences from published papers that expert curators tagged as describing a specific rigor criterion (for example, a sentence stating that group allocation was randomized). At submission, it flags methods-section text that does or doesn’t address each criterion, so an author or editor can see, item by item, what’s missing before the paper goes further into review. According to SciCrunch’s own published FAQ for the tool, the average SciScore across journals indexed in PubMed Central was 4.2 out of 10 in 2019 — a benchmark SciCrunch uses to argue that fewer than half of the standard rigor and resource-identification criteria are routinely reported in the literature without prompting.

SciCrunch is the same organization that operates the Resource Identification Portal used to mint and resolve RRIDs, so SciScore’s Key Resources Table and CASRAI’s own RRID entry describe the same underlying identifier infrastructure from two angles — RRID is the identifier itself, SciScore is one automated mechanism that checks whether a manuscript actually used it correctly.

Which publishers and journals use it

SciScore is integrated as a submission or revision-stage check at a number of named publishers, confirmed directly from SciCrunch’s own partner pages and press coverage:

• American Heart Association — announced September 19, 2024, integrating SciScore into the submission workflow of five journals: Arteriosclerosis, Thrombosis, and Vascular Biology, Circulation Research, Stroke, Hypertension, and Stroke: Vascular and Interventional Neurology. A prior one-year pilot with Circulation Research reported that average monthly SciScores for submitted manuscripts rose roughly 4% month-over-month during the pilot period.
• Rockefeller University Press — integrated SciScore into its eJournalPress-based editorial workflow for the Journal of Cell Biology, Journal of Experimental Medicine, and Journal of General Physiology, made available to authors free of charge at the revision stage.
• FASEB — integrated SciScore into its journals’ submission and peer-review system, with authors able to access the tool both at submission and again before publication.

SciScore also integrates with widely used manuscript-submission platforms (Editorial Manager, eJournalPress) and with the bioRxiv/medRxiv preprint servers, which is how it reaches manuscripts before they are formally submitted to a specific journal. Because SciCrunch adds and updates integration partners over time, this list should be treated as a snapshot of confirmed adopters rather than an exhaustive or permanently current one — check SciCrunch’s own site for the current partner roster before citing a specific journal as a current SciScore user.

Scoring and the Rigor and Transparency Index (RTI)

Individual manuscript scores aggregate into SciCrunch’s Rigor and Transparency Index (RTI), described as the yearly average SciScore for a given journal — a proxy for how consistently that journal’s authors report rigor and resource-identification criteria, independent of the journal’s citation-based impact factor. SciCrunch’s own published analysis reports no correlation between a journal’s RTI and its Journal Impact Factor, which is the basis for positioning RTI as a methodological-quality signal distinct from citation-based prestige metrics.

SciCrunch is explicit that SciScore is not a substitute for expert peer review: it cannot judge whether a chosen method was scientifically appropriate for a given study, only whether standard reporting elements are present in the text. Authors can raise their score by stating rigor elements explicitly — including explicitly stating an item is not applicable — and by supplying RRIDs with full catalog and vendor information for research resources, rather than by changing the underlying experimental design.

What counts as a “good” SciScore?

SciCrunch has not published a fixed numeric cutoff for what counts as a “good” SciScore, and CASRAI is not aware of one being defined anywhere in SciCrunch’s own documentation. The tool returns a score out of 10 (up to 5 points from the Rigor Table, up to 5 from the Key Resources Table) rather than a pass/fail grade — it was designed as a completeness signal for a specific methods section, not a verdict on whether the underlying science is well designed.

The only benchmark SciCrunch itself has published is a population-level one: the average SciScore across journals indexed in PubMed Central was 4.2 out of 10 in 2019 (see “What SciScore checks for,” above). That number is a useful rough reference point — a score meaningfully above 4.2 indicates more complete rigor and resource reporting than the median PMC paper had at that time — but it is a historical field-wide average, not a target the tool asks any individual author to hit, and it will have moved as more journals adopted SciScore-style checks since 2019.

In practice, a SciScore is more useful read criterion-by-criterion than as a single number:

• Check which specific items are unaddressed, not just the total. The report lists each Rigor Table and Key Resources Table item individually; a 6/10 that is missing only “sex as a biological variable” — because the study is, say, a plant-tissue experiment where that criterion doesn’t apply — is a different situation from a 6/10 missing randomization and blinding in an animal study where both should have been reported.
• State inapplicable items explicitly rather than leaving them silent. SciCrunch’s own guidance is that authors should explicitly note when a criterion doesn’t apply to their study design — an unaddressed item and a deliberately-not-applicable item can otherwise look the same to a text-mining tool.
• Compare against a target journal’s own reporting norms where that’s visible, not an abstract ideal. Because individual SciScores roll up into a journal’s yearly Rigor and Transparency Index (RTI — see above), a score close to or above a journal’s own published RTI is a more relevant comparison than a universal number, where that figure is available.
• Remember the ceiling is reporting completeness, not study quality. A high SciScore means the methods section states that randomization, blinding, sample-size justification, sex reporting, and RRID-identified resources were addressed — it does not verify that the randomization procedure was sound or that the sample size was adequately powered. SciCrunch is explicit that the tool is not a substitute for expert peer review.

Because no official “good score” threshold exists, treat any specific numeric target quoted elsewhere as an informal rule of thumb rather than a SciCrunch-endorsed standard — the only number SciCrunch itself has published for context is the 4.2/10 PubMed Central average cited above.

How SciScore relates to CASRAI’s broader rigor and reproducibility vocabulary

SciScore is one specific, named implementation of a broader shift toward automated, standards-based rigor checking that CASRAI’s Dictionary tracks more generally:

• RRID (Research Resource Identifier) — the identifier standard SciScore’s Key Resources Table checks for.
• Authentication of key resources — the broader reporting practice SciScore’s Key Resources Table operationalizes at scale.
• NIH Rigor and Reproducibility policy — one of the named standards SciScore’s Rigor Table draws its criteria from.
• Reproducibility — the outcome SciScore’s checks are ultimately intended to support, distinct from replicability or generalizability.
• Reproducibility and computational research (CASRAI Dictionary domain) — browse the wider set of related terms.

For a wider view of how automated rigor tools sit alongside reporting frameworks such as TOP, ARRIVE, CONSORT, and PRISMA, see CASRAI’s news coverage of reproducibility frameworks in practice.