Few decisions shape a research career more than those made at the moment of hiring, tenure and promotion. They determine who enters the profession, who advances, and whose work is rewarded — and in doing so they send a powerful signal about what the system values. For decades, those decisions have leaned heavily on a familiar set of shortcuts: the prestige of the journals a researcher has published in, the impact factor attached to those journals, the raw count of their publications, and the citations they have accumulated. These proxies are seductive because they are quick and appear objective. But a growing movement argues they are also distorting — rewarding the wrong things, overlooking much of what researchers genuinely contribute, and shaping behaviour in ways that harm research itself. This article examines the reform of assessment in hiring and promotion, drawing on the responsible-assessment domain of the CASRAI Dictionary.
What the metric-driven approach gets wrong
The case against journal-based and citation-based assessment is not that numbers are useless but that they are being asked to do work they cannot do. The journal impact factor is a measure of a journal’s average citation performance; it says nothing reliable about the quality or significance of any individual article within it, still less about the researcher who wrote it. Using it to judge people is a category error. Citation counts, meanwhile, are slow, field-dependent and easily distorted, and they capture only one narrow kind of influence. More corrosively, when careers depend on these proxies, researchers are pushed to chase them — prioritising quantity, fashionable topics, more papers, and venues where the metrics are favourable. The metric stops measuring behaviour and starts driving it, and much of what research needs — rigorous replication, careful data curation, mentoring, software, public engagement — goes unrewarded because it does not show up in the count.
DORA and the founding principle
The reform movement’s most influential starting point is the San Francisco Declaration on Research Assessment (DORA). Its central recommendation is simple and far-reaching: do not use journal-based metrics, such as the impact factor, as a surrogate measure of the quality of individual research articles or of the contributions of individual researchers in hiring, promotion and funding decisions. Instead, DORA urges that research be assessed on its own merits and that the full range of research outputs and activities be valued. Thousands of organisations and individuals have signed, committing to change how they evaluate. DORA’s power lies in naming the central abuse plainly and asking institutions to stop it — making the misuse of the impact factor something an organisation must publicly disavow rather than quietly perpetuate.
The Leiden Manifesto and the Hong Kong Principles
Other frameworks complement DORA. The Leiden Manifesto sets out principles for the responsible use of metrics — that quantitative evaluation should support, not supplant, expert qualitative judgement; that measurement be aligned with the mission of institution and researcher; that data and analysis be open and verifiable; and that indicators account for variation by field. It does not reject metrics but disciplines them. The Hong Kong Principles focus on assessing researchers in ways that reward trustworthy research — recognising open, rigorous and reproducible practices and the full range of research activities — so that assessment pulls towards integrity rather than against it. Together these frameworks describe an assessment system that uses judgement supported by responsible indicators, rather than indicators in place of judgement.
CoARA and the move from declaration to action
The most ambitious recent development is the Coalition for Advancing Research Assessment (CoARA) and its Agreement on Reforming Research Assessment. CoARA moves the conversation from principle to implementation: signatory organisations commit to concrete reform, including basing assessment primarily on qualitative judgement supported by responsible use of quantitative indicators, recognising the diversity of research outputs and activities, and abandoning the inappropriate use of journal- and publication-based metrics. Crucially, CoARA asks organisations to actually change their criteria and processes — their hiring rubrics, their promotion frameworks — and to share progress, turning broad agreement into the difficult, practical work of rewriting the rules by which careers are judged.
The narrative CV as a practical instrument
If institutions are to stop counting papers and start assessing contribution, they need a different way of presenting a career — and the narrative CV has become the leading answer. Instead of a long list of outputs and metrics, a narrative CV asks researchers to describe their contributions in prose: what they have contributed to the generation of knowledge, to the research community, to wider society, and to the people and teams they have worked with. This format makes space for exactly the things metrics miss — mentoring, data and software, collaboration, public engagement, contributions to research culture — and it invites qualitative peer judgement of substance rather than mechanical comparison of numbers. The narrative CV is not a panacea; it must be assessed fairly and consistently. But it embodies the reform agenda in a single document: it asks what someone did and why it mattered, rather than where and how often they published.
Contribution, vocabulary and fair judgement
Assessing contribution fairly depends on being able to describe contributions clearly and consistently. This is where structured accounts of who did what become valuable: the CRediT taxonomy, whose full set of contribution types is set out in our overview of the CRediT roles, lets a researcher’s actual role in their work be stated rather than inferred from a byline — supporting the kind of contribution-based assessment these reforms call for, and connecting naturally to the wider conduct of authorship and contributorship. For assessment to be fair across institutions and systems, the terms used to describe outputs, roles and contributions must mean the same thing everywhere; that consistency is what the CASRAI Dictionary provides. Reforming assessment is ultimately about realigning reward with worth — ensuring that the people who do the most valuable research, in all its forms, are the people the system recognises and advances.
Leave a Reply