Metrics are seductive because they are simple. A single number — a journal’s impact factor, a researcher’s h-index, a citation count — promises to compress the messy, qualitative business of judging research into something fast, comparable and apparently objective. And metrics are dangerous for exactly the same reason: their simplicity hides what they leave out, and their apparent objectivity lends unearned authority to comparisons they cannot really support. The response to this tension has not been to abolish metrics but to use them responsibly — to let quantitative indicators inform expert judgement rather than replace it. Two landmark statements from 2015, the Leiden Manifesto and The Metric Tide, set out what responsible use looks like. This article examines both and how they translate into practice, drawing on the responsible assessment domain of the CASRAI Dictionary.
The Leiden Manifesto
The Leiden Manifesto for research metrics, published in 2015, offers ten principles for the responsible use of quantitative indicators. Several of its themes recur throughout the responsible-metrics movement and are worth drawing out. It insists that quantitative evaluation should support, not supplant, qualitative expert assessment — metrics inform judgement; they do not make it. It warns against measuring performance against inappropriate or generic benchmarks, urging that assessment account for the mission and context of the research. It calls for transparency in the data and methods behind any indicator, so that those being assessed can understand and scrutinise how they are judged. It highlights the importance of accounting for variation between fields, since citation behaviour differs enormously across disciplines and naive comparison across them is meaningless. And it cautions against the distortions metrics produce when they become targets — the well-known problem that an indicator, once it is what people are rewarded for, stops measuring what it was meant to.
The Metric Tide
Published the same year, The Metric Tide was an independent review of the role of metrics in research assessment, conducted in the United Kingdom. Its central contribution was the concept of responsible metrics, defined through a set of dimensions that have become a common reference point:
- Robustness — basing indicators on the best available, accurate data.
- Humility — recognising that quantitative evaluation should support, not supplant, expert assessment.
- Transparency — keeping data collection and analytical processes open to scrutiny.
- Diversity — accounting for variation by field and using a range of indicators to reflect the plurality of research.
- Reflexivity — recognising and anticipating the systemic effects of indicators and updating them in response.
The review was notably sceptical of reducing assessment to single numbers and emphasised that metrics work best as a complement to peer review, not a substitute for it. Its framing of responsible metrics as a set of dimensions to be designed for, rather than a checklist to be passed, has proved durable.
What the two have in common
Read together, the Leiden Manifesto and The Metric Tide converge on a consistent message. Metrics are useful but partial; they must be transparent so they can be questioned; they must respect disciplinary difference; they must be used with humility alongside expert judgement; and their users must stay alert to the behaviour they induce, because any metric that becomes a target will eventually be gamed or will distort the work it was meant to measure. Neither document is anti-metric. Both are against the misuse of metrics — against the false precision of a single number standing in for a considered judgement about the quality and significance of research.
From principle to practice
Translating these principles into institutional practice means concrete commitments: assessing research on its own merits rather than on the prestige of its publication venue, using a basket of indicators rather than any single one, being transparent about what is measured and how, contextualising comparisons by field and career stage, and keeping expert peer judgement at the centre with metrics in a supporting role. These commitments connect directly to the broader assessment-reform movement. The principle of not judging research by where it is published is the heart of the comparison in our DORA versus CoARA overview, while the specific hazards of the two most over-used single numbers are examined in our look at the journal impact factor versus the h-index. Responsible metrics is the methodological backbone these reform initiatives share.
Metrics and the recognition of contribution
One reason single-number metrics mislead is that they obscure who actually did the work and what they did. A citation count attaches to a paper, not to the distinct contributions of the people who made it. Structured contributorship through the CRediT taxonomy — whose full set of roles is described in our overview of the CRediT roles — offers a more granular and honest picture of contribution than any aggregate metric can, and is a natural complement to responsible assessment: it supports judging people on what they genuinely contributed rather than on a number that flattens it. The consistent vocabulary that lets assessment frameworks, indicators and contribution records be described and exchanged the same way across systems is maintained in the CASRAI Dictionary, helping ensure that responsible metrics rests on a shared and well-defined foundation.