CASRAI Dictionary

Tag: CoARA

  • The case for narrative CVs beyond UKRI

    The UK Research and Innovation Resume for Research and Innovation (R4RI), launched in pilot in 2019 and now the default CV format across UKRI’s seven research councils, has accumulated enough operational experience to draw lessons. The narrative-CV approach has spread internationally: the Dutch Research Council, the Swiss National Science Foundation, the Royal Society’s CV format, parts of the EU’s Horizon Europe evaluation, and several US private funders have adopted variants. Where it has not spread is the major US federal funders. This post argues the case for broader adoption with reference to the UKRI experience.

    What R4RI is

    R4RI replaces the traditional publication-list CV with a structured narrative covering four modules: how the researcher has contributed to the generation of new ideas, tools, methodologies, or knowledge; how they have contributed to the development of others; how they have contributed to the wider research community; how they have contributed to broader research and innovation users and audiences, and the wider environment. Each module has a 250-word limit and the researcher provides specific evidence.

    What R4RI explicitly does not ask for: lengthy publication lists, journal impact factors, h-indexes, citation counts, exhaustive grant histories. The Resume for Research and Innovation can include publications but as evidence of contribution, not as a metric.

    Five years of operational experience

    UKRI’s commissioned evaluations of R4RI (published 2022, 2024) and the broader literature on narrative-CV use have produced a reasonably clear picture.

    First, reviewer time. The early concern was that narrative CVs would take longer to review than conventional ones. The evaluation data show modest increases for first-time reviewers, settling to comparable or shorter review times once reviewers were familiar with the format. Reviewers report that R4RI gives them a clearer picture of the applicant’s actual contribution.

    Second, applicant time. Writing an R4RI takes longer than updating a publication list, and applicants without writing support are at a disadvantage. The equity implication is real: a researcher with access to research-administration support to help draft R4RI does better than one without. UKRI has invested in writing-support resources and several institutions have built internal capacity.

    Third, career-stage equity. Narrative CVs perform better for early-career researchers whose publication record is short but whose contribution is significant; they perform better for researchers with non-traditional career paths; they perform better for researchers in disciplines where high-impact publication is not the norm. They perform less well for researchers with very strong conventional records who feel the narrative format does not adequately recognise their publications. On balance the evaluation suggests narrative CVs reduce systemic bias against under-represented career patterns.

    Fourth, inter-rater reliability. The concern that narrative CVs would produce more variable assessment than metric-based CVs has been partially borne out: inter-rater reliability is somewhat lower for R4RI than for conventional CVs. This is in part a feature, not a bug — different reviewers genuinely weight different contributions differently, and the narrative CV surfaces those judgements. UKRI has responded with reviewer-training resources and structured rubrics.

    Why other funders should adopt narrative CVs

    Four reasons.

    First, narrative CVs operationalise DORA and CoARA commitments in a concrete way. A funder that has signed DORA but continues to use publication-metric CVs is operating in contradiction with its commitment; a funder that adopts a narrative CV format is operationalising it. The CASRAI responsible assessment domain tracks the gap between policy and practice across major funders.

    Second, narrative CVs make CRediT more useful. A CV that reports CRediT roles for the applicant’s recent papers gives the reviewer specific information about contribution; a publication list without CRediT gives only the byline order. The integration is operationally simple: narrative CVs cite specific contributions, CRediT statements describe what the contribution was.

    Third, narrative CVs reduce the metric-feedback loop. The pernicious cycle in which researchers chase high-impact-factor publications because funders weight them and funders weight them because researchers seek them is one of the system pathologies that responsible-assessment reform aims to break. A narrative CV format breaks the funder side of the loop, which gives researchers permission to optimise differently.

    Fourth, narrative CVs encode the broader range of contributions that modern research actually requires. Software, datasets, public engagement, peer review, mentorship, leadership of community-led infrastructure, contributions to open standards — none of these show up well in a conventional CV. They show up in a narrative CV. The CASRAI institutional responsible-assessment guide includes a checklist of contribution types that narrative formats can capture.

    The objections, addressed

    Three objections recur and deserve direct responses.

    Narrative CVs are subjective and unscientific. The metric-based CV is also subjective: someone decided which metrics to weight, what the weighting should be, and what counts as success. Narrative CVs make the subjectivity explicit and reviewable; metric-based CVs hide it behind a number.

    Narrative CVs disadvantage non-English-speakers. The concern is real and the mitigation is to allow CVs in the applicant’s working language with translation support funded by the funder. UKRI does not currently allow non-English R4RIs because UKRI operates in English; an international funder with multilingual operations would need to.

    Narrative CVs are too long for high-volume review. The 250-word-per-module limit and the four-module structure produce a CV that is no longer than a conventional 5-page academic CV; in many cases shorter. The objection is empirically wrong as stated.

    Practical recommendations

    For funders considering adoption, the practical steps are: pilot with one or two grant streams; train reviewers with worked examples; develop a structured rubric for scoring; provide writing-support resources for applicants; commit to an evaluation at year three; iterate the format based on the evaluation. UKRI’s experience suggests this approach yields a usable format within three years.

    For institutions supporting applicants, the practical steps are: build internal capacity to support narrative-CV drafting; offer it equitably across career stages and disciplines; treat it as part of the research-administration support package, not as an exceptional service for senior researchers only.

    For applicants, the practical advice is to start drafting in a narrative format now, even where the funder does not require it. The discipline of articulating contribution rather than enumerating publications produces a richer self-understanding of one’s own research and is useful for promotion, tenure, and personal career planning regardless of funder requirements.

    The trajectory

    We expect narrative-CV adoption to accelerate through 2026-2028. The CoARA commitment to reforming research assessment, combined with the maturity of the R4RI model, gives funders a credible template to adopt. The remaining holdouts are the major US federal funders (NIH and NSF), whose biosketch and current-and-pending-support formats are partial moves toward narrative but retain substantial metric content. The next round of US-funder review-criteria revision will be the test.

    Related dictionary entries

  • DORA, CoARA, and the Hong Kong Principles: the responsible-assessment lineage

    The responsible-research-assessment lineage runs from DORA in 2012 to the Leiden Manifesto in 2015 to the Hong Kong Principles in 2020 to the CoARA Agreement in 2022 to UKRI’s 2024 R4RI mandate and to a steadily lengthening list of similar institutional mandates worldwide. The throughline is consistent: stop using journal impact factors and h-indices as proxies for research quality; assess the research itself, by people qualified to read it, with attention to a wider range of contributions and to the contexts that shape them.

    The throughline has been clear for over a decade. The implementation has been slow. This post traces what each step of the lineage actually committed signatories to, what has held the practical turn back, and what UKRI’s R4RI does that earlier mandates did not.

    DORA: the inflection

    The San Francisco Declaration on Research Assessment, drafted at the ASCB annual meeting in December 2012 and released in May 2013, was the first widely-signed statement that the journal impact factor should not be used as a proxy for individual-researcher quality. DORA’s core recommendation was simple: do not use journal-based metrics in hiring, promotion, or funding decisions; assess scientific content directly.

    The recommendation looked unobjectionable in 2013 and was endorsed within months by major funders (Wellcome, HHMI), publishers (PLOS, eLife, BMC), and societies. The signatory list has since grown past 25,000 institutions and individuals. The problem is that signing DORA and implementing it are not the same thing. Multiple audits over the 2017-2022 period found that most DORA signatories’ promotion-and-tenure committees still relied heavily on journal-prestige proxies, sometimes formally and more often informally.

    The Leiden Manifesto: the operating principles

    Published in Nature in April 2015 by Diana Hicks, Paul Wouters, and colleagues, the Leiden Manifesto for Research Metrics articulated ten principles for the use of bibliometrics: quantitative evaluation supports qualitative expert judgement (not replaces it); measure performance against the research missions of the institution, group, or researcher; protect excellence in locally relevant research; keep data collection and analytical processes open, transparent and simple; allow those evaluated to verify data and analysis; account for variation by field; base assessment of individual researchers on a qualitative judgement of their portfolio; avoid misplaced concreteness and false precision; recognise the systemic effects of assessment and indicators; scrutinise indicators regularly and update them.

    The Leiden Manifesto was more operationally useful than DORA precisely because it described how to use metrics responsibly, not just which metrics to avoid. The principle that quantitative evaluation supports rather than replaces qualitative judgement remains the most cited and the most violated.

    The Hong Kong Principles: linking integrity and assessment

    The Hong Kong Principles for assessing researchers, drafted at the 2019 World Conference on Research Integrity in Hong Kong and published in 2020 by Moher, Bouter, Kleinert and colleagues, took a different angle. The Hong Kong authors argued that current assessment systems actively encourage poor integrity practices (publication bias, salami slicing, p-hacking, gift authorship) because they reward publication count and JIF over rigour. The five Hong Kong principles ask assessors to value: responsible research practices, complete reporting, open science, a diverse range of research types and outputs, and a range of contributions to research.

    The Hong Kong contribution is the explicit link between assessment reform and research-integrity reform. As long as we assess researchers on counts and impact factors, we are paying them to publish more, not better. The Hong Kong frame is now the dominant interpretation among integrity researchers and is referenced explicitly in the CoARA Agreement.

    CoARA: the policy machine

    The Coalition for Advancing Research Assessment was launched in July 2022 by the European Commission, Science Europe, and a wide coalition of European universities, funders, and learned societies. The CoARA Agreement (the founding document) commits signatories to ten reform actions over a defined timetable, with the centrepiece being the move from a quantitative-indicator-led assessment to a portfolio-and-narrative-led assessment.

    What makes CoARA different from DORA, Leiden, and Hong Kong is that CoARA is institutional, has a secretariat, has working groups, and has a peer-pressure mechanism: signatory institutions must publish an Action Plan within one year of joining and report annually on progress. By early 2026 CoARA has over 700 signatory institutions across Europe and a growing footprint in Latin America, Africa, and Asia.

    The CoARA working groups are working through the practical implementation questions that DORA hand-waved: how do you build a hiring committee evaluation rubric that does not collapse back onto JIF; how do you train evaluators; how do you handle international comparisons when home and visiting institutions have different reform stages; how do you preserve fairness across career stages and disciplines.

    The narrative CV turn

    The practical artefact most visibly downstream of this lineage is the narrative CV. Replacing the long-list-of-publications CV with a structured narrative of contributions across several dimensions (research outputs, leadership, team-building, broader impact, and more) was first piloted by the Royal Society (2017), then formalised by the Royal Society of Biology and by UKRI as R4RI (Résumé for Researchers and Innovators).

    The narrative CV asks applicants to articulate, in a defined structure, what they have contributed to their team and field. The Royal Society R4RI structure (since adopted by UKRI and several other funders) has four modules: how you have contributed to the generation of knowledge; how you have contributed to the development of individuals; how you have contributed to the wider research community; how you have contributed to broader research and innovation users and audiences. Each module is several paragraphs of prose with selected outputs cited as evidence.

    UKRI’s 2024 mandate that R4RI be the default CV format across UKRI funding schemes was the first wholesale move of a major funder to mandate a narrative format. The 2025 evaluations of UKRI’s first-cohort R4RI grants are tentatively positive: applicants report finding the format more flexible but more demanding; reviewers report richer assessment material but slower review; outcome diversity (career stage, discipline, institution) appears to have widened modestly. The CoARA working group on narrative formats is using the UKRI experience as its primary case.

    For authors preparing R4RI-format CVs, the CASRAI narrative CV guidance walks through the four modules with examples and common failure modes (over-quantification, under-selection of outputs, generic claims).

    What’s holding implementation back

    Three frictions persist. First, international comparability. A researcher trained in a CoARA-signatory institution applies for a postdoc in a non-signatory institution and has to translate their narrative CV into a publications-and-h-index format, often disadvantageously. Second, evaluator training. The skill of reading a narrative CV well is not innate; evaluators trained on JIF-based assessment default back to it under time pressure. Third, algorithmic ranking. As long as institutions are ranked by university league tables that count high-impact publications, individual hiring committees will be reluctant to fully de-emphasise those publications.

    The work-arounds in 2026 are pragmatic: CoARA’s working groups are producing evaluator training materials; narrative-CV templates are increasingly tool-supported (ORCID’s narrative-CV draft module, the ARDC RAiD-linked R4RI tooling); and at least three European university-ranking systems (the THE Impact Rankings, the U-Multirank framework, the CWTS Leiden Ranking’s responsible-metrics variant) are explicitly excluding JIF-based criteria.

    The 2026 outlook

    The most likely 2026-2027 development is convergence of the European CoARA framework with the North American equivalent that is emerging from NIH’s UNITE initiative and from the OSTP’s open-science memo. The signal that this convergence is real will be a joint funder declaration on assessment reform, probably late 2026 or 2027. The risk is that responsible-assessment policies multiply without converging, leaving researchers to navigate a different framework with each funder.

    For institutions writing or revising assessment policies now, the practical advice is to sign CoARA (or DORA at minimum), commit to a narrative-CV pilot, train your evaluators, and report transparently on outcomes. The responsible-assessment domain at CASRAI tracks signatory institutions’ action plans and the published evaluations.

    Related dictionary entries

    References

    DORA, San Francisco Declaration on Research Assessment (2013). Hicks et al., Bibliometrics: The Leiden Manifesto for research metrics (Nature, 2015). Moher et al., The Hong Kong Principles for assessing researchers (PLOS Biology, 2020). CoARA, Agreement on Reforming Research Assessment (2022). UKRI, Resume for Researchers and Innovators (R4RI) guidance (2024 mandate).