Dictionary domainTrack E
Responsible research assessment
DORA, COARA, R4RI, narrative CVs.
For implementers
Operational deployment checklist for Responsible research assessment: prerequisites, five deploy steps, integration notes for Pure, Symplectic Elements, Worktribe, DSpace, and more, plus the pitfalls that recur in the field.
Terms in this domain
33 terms
Re-evaluation cycle
A scheduled, periodic review by a research-assessment system of its own indicators, criteria and processes to identify drift, unintended consequences and opportunities for reform.
Lifetime contribution lens
An assessment approach that evaluates a researcher's contributions across their whole career, rather than within a fixed recent window, especially for senior recognition such as fellowships, honours and emeritus appointments.
Career diversity
Recognition in research assessment of diverse career paths, trajectories and mobility patterns, including industry experience, intersectoral mobility, career breaks, part-time work and non-linear progression.
Process diversity (in assessment)
Recognition in research assessment of the diverse research processes researchers contribute to, including teaching, mentoring, peer review, leadership, software engineering, data curation, public engagement and policy work.
Output diversification
The recognition in research assessment of a wide range of research outputs beyond journal articles, including datasets, software, protocols, registered reports, policy briefs, creative works and practitioner publications.
Peer review (in assessment)
The use of expert qualitative judgement by relevant peers to evaluate research outputs, researchers, proposals or environments, distinct from peer review of journal manuscripts.
Quantitative-qualitative balance
The principle in responsible assessment that quantitative indicators should support, not supplant, qualitative expert judgement and that both forms of evidence should be combined transparently.
Responsible metrics
An approach to using quantitative indicators in research assessment that emphasises robustness, humility, transparency, diversity and reflexivity, as articulated in the 2015 Metric Tide report.
Twitter mentions (concept)
An altmetric signal counting public posts on Twitter (rebranded to X in 2023) that include a link or identifier referring to a specific research output.
Mendeley readership
An altmetric signal derived from the count of users who have added a research article to their personal library in the Mendeley reference manager.
Relative Citation Ratio (RCR, NIH)
An article-level field-normalised citation metric developed by the US National Institutes of Health, expressing an article's citations per year relative to the median in the article's co-citation network, with a value of 1.0 corresponding to NIH-funded median performance.
Field-Weighted Citation Impact (FWCI)
An article-level citation indicator, published by Elsevier (SciVal/Scopus), that normalises an article's citation count against the average for outputs of the same publication year, document type and subject field, with 1.0 representing the world average.
Article-level metrics
Indicators reported at the level of an individual article rather than the journal in which it was published, typically including citations, views, downloads, saves and altmetric signals.
Altmetrics
A family of indicators that capture attention to and engagement with research outputs across non-traditional sources such as social media, news outlets, policy documents, blogs, Wikipedia and reference managers.
h-index
A researcher-level metric proposed by Jorge Hirsch in 2005, defined as the largest number h such that the researcher has published h papers each cited at least h times.
CiteScore
A journal-level metric published by Elsevier (Scopus) since 2016 that measures the average number of citations received in a given year by documents published in that journal during the previous four years.
Journal Impact Factor (JIF)
An annual journal-level metric, published by Clarivate in the Journal Citation Reports, equal to the average number of citations in a given year to articles published in that journal during the preceding two years.
Personal statement (research assessment)
A short narrative section in a researcher CV, grant application or assessment dossier in which the researcher explains, in their own voice, the context, motivation and significance of their work and contributions.
Biosketch (NIH-style)
A five-page structured CV format mandated by the US National Institutes of Health for grant applications, organised around a personal statement, positions and honours, contributions to science, and a research-support section.
R4RI mandate (UKRI 2024)
The UK Research and Innovation policy, fully effective in 2024, requiring applicants and reviewers in most UKRI funding opportunities to use the R4RI-derived narrative CV ('Resume for Research and Innovation') instead of traditional academic CVs.
R4RI (Resume for Researchers / Royal Society narrative CV)
A narrative-style CV template originally developed by the Royal Society in 2019 that captures a researcher's contributions across four modules rather than relying on a list of publications and metrics.
REF 2029
The next planned cycle of the UK Research Excellence Framework, currently scheduled with a submission deadline in 2028 and results in 2029, with substantial design changes following the Future Research Assessment Programme.
REF 2021
The 2021 cycle of the UK Research Excellence Framework, whose results were published in May 2022, assessing research produced between 2014 and 2020.
REF impact case study
A structured narrative submission to the REF describing how research from a UK institution has produced demonstrable effects, change or benefit beyond academia.
REF (Research Excellence Framework, UK)
The UK's national exercise for assessing the quality of research in higher education institutions, used to inform the selective allocation of quality-related (QR) research funding.
Metric Tide Review
The 2022-2023 re-examination of the original Metric Tide report, also chaired by James Wilsdon for Research England, which reviewed how responsible-metrics principles had been implemented since 2015.
Metric Tide Report
The 2015 independent review of the role of metrics in UK research assessment, commissioned by HEFCE and chaired by James Wilsdon, which introduced the concept of 'responsible metrics'.
Leiden Manifesto
A 2015 set of ten principles for the responsible use of quantitative indicators in research evaluation, authored by Hicks, Wouters, Waltman, de Rijcke and Rafols.
Hong Kong Principles for Assessing Researchers
A set of five principles published in 2020 to guide the assessment of researchers in ways that reward research integrity and trustworthy practice rather than productivity alone.
CoARA Agreement
The 2022 Agreement on Reforming Research Assessment that sets out ten commitments members of CoARA undertake to implement in their evaluation policies and practices.
CoARA (Coalition for Advancing Research Assessment)
A multi-stakeholder coalition launched in 2022 to coordinate the reform of research assessment across institutions, funders and learned societies, primarily in Europe but with global membership.
DORA signatory
An individual or organisation that has formally endorsed the San Francisco Declaration on Research Assessment by registering its commitment on the DORA website.
DORA (San Francisco Declaration on Research Assessment)
A 2012 declaration urging that research assessment stop relying on journal-based metrics, particularly the Journal Impact Factor, as proxies for the quality of individual research outputs or researchers.
