Responsible Use of Metrics: Comparing UK University DORA Guidance

The responsible use of metrics means applying quantitative research indicators — citation counts, field-weighted citation impact, grant income — only to inform and support expert peer judgement, never to replace it, in line with the San Francisco Declaration on Research Assessment (DORA). Cambridge, Exeter, Edinburgh and UCD have each published DORA-aligned guidance for their own institutions, but a side-by-side reading shows the four documents converge on principle and diverge sharply on governance, prescriptiveness and review discipline.

Responsible research metrics is the umbrella term for institutional policies that constrain how bibliometric and altmetric indicators may be used in hiring, promotion and funding decisions, so that no single number is treated as a proxy for research quality.

• What does “responsible use of metrics” actually require?
• Where Cambridge, Exeter, Edinburgh and UCD guidance converges
• Where the four institutions’ guidance diverges
• What gaps remain for institutions without a dedicated policy?
• Answer-first Q&A on research metrics
• Implications for research administrators

What does “responsible use of metrics” actually require?

DORA’s own guidance on the responsible use of quantitative indicators sets out five criteria that any institutional policy should meet: metrics use should be clear, be transparent, be specific, be contextual, and be fair, according to sfdora.org’s published guidance document. Separately, the UK’s Forum for Responsible Research Metrics — convened by Universities UK following the 2015 Metric Tide report — frames the same territory as five R’s: robustness, humility, transparency, diversity and reflexivity.

Every institutional statement reviewed here traces back to the same three source documents: DORA (2012), the Leiden Manifesto for Research Metrics (2015), and the Metric Tide (2015, updated as Harnessing the Metrics Tide in 2022). What differs is how each university translates those shared principles into binding local policy — and that is where the real variation, and the real risk of inconsistent practice, sits.

Where Cambridge, Exeter, Edinburgh and UCD guidance converges

All four institutions state unambiguously that quantitative metrics must support, not supplant, qualitative expert assessment. All four are DORA signatories and all four explicitly rule out using the Journal Impact Factor as a proxy for the quality of an individual output or researcher.

• Metrics must be applied at the correct level of granularity — never using a journal-level or institution-level number to judge an individual.
• Comparisons between individuals must account for career stage, career breaks and part-time working.
• Any metric used in assessment must be disclosed in advance to the people being assessed.
• Metrics and their underlying datasets must be periodically reviewed for continued fitness of purpose.

Edinburgh and Exeter are also both signatories to the Coalition for Advancing Research Assessment (CoARA), which each joined in 2022, committing them additionally to phasing out inappropriate use of the h-index alongside the Journal Impact Factor.

Where the four institutions’ guidance diverges

Beneath the shared principles, the four documents take genuinely different institutional forms — a distinction that matters more than the principles themselves for anyone trying to replicate or benchmark a policy.

Exeter’s document is the most technically granular of the four, naming specific indicator products — Field-Weighted Citation Impact from Scopus/SciVal, Field Citation Ratio and Relative Citation Ratio from Digital Science, Category Normalised Citation Impact from Web of Science — and warning explicitly against mixing metrics from different bibliometric providers within the same assessment exercise. Edinburgh is the most procedurally binding, with a stated review cycle and a named committee for re-approval. Cambridge is the most devolved, deliberately declining to impose a single university-wide metric policy in favour of discipline-appropriate local rules. UCD is the most EDI-forward, embedding equity language directly into its core commitments rather than treating it as a supporting principle.

What gaps remain for institutions without a dedicated policy?

Cambridge, Exeter, Edinburgh and UCD each have a named committee, a published document and — in Edinburgh’s case — a fixed review date. Many smaller and teaching-intensive institutions have none of this. Several UK universities that rank prominently for “responsible use of metrics” searches — including library subject guides from institutions such as Derby, Plymouth and Sunderland — publish summaries of DORA’s principles rather than institutionally approved governance statements.

That distinction is not cosmetic. A library guide can explain what responsible metrics are; it cannot bind a promotion committee the way a document approved by a Research Policy Group, a Champions Group or an RIIG can. Institutions without a dedicated policy and a named approving body carry a structural gap: staff have no enforceable assurance that a hiring panel or REF preparation exercise will actually follow the principles a library page describes. For research administrators at smaller institutions, the practical route is not to draft new principles from scratch but to adapt an existing framework — Exeter’s document explicitly credits the UCL Principles as its own starting point, and UCD’s statement was built after reviewing three existing peer institutions’ policies, showing that adaptation, not original drafting, is the established norm.

Answer-first Q&A on research metrics

What are the four types of metrics used in research assessment?

Institutional guidance, including Exeter’s, groups research indicators into institutional or discipline-level indicators (rankings, field-weighted citation impact), output-level indicators (citation counts, Journal Impact Factor, altmetrics), research activity indicators (grant income, PGR numbers) and individual-focused indicators (h-index, highly-cited rankings) — each requiring different safeguards against misuse.

What is the use of metrics in responsible research assessment?

Metrics provide contextual, supporting evidence alongside qualitative peer review — never a standalone verdict. Under DORA and the institutional statements reviewed here, quantitative indicators may inform hiring, promotion and funding decisions only when disclosed in advance, appropriately normalised and applied at the correct level of granularity.

What are examples of responsible research metrics?

Commonly cited examples include field-weighted citation impact, altmetrics, grant income and postgraduate research supervision counts, used as part of a discipline-appropriate “basket of measures” rather than in isolation. Journal Impact Factor and raw h-index are explicitly excluded as individual-level proxies by every institution examined here.

Implications for research administrators

For research administrators, the comparison points to a practical hierarchy of maturity: a published statement with no named governance body (the entry point most smaller institutions can reach quickly); a statement with a standing committee (Exeter’s Champions Group, UCD’s Working Group reporting into RIIG); and a statement with a fixed, published review cycle (Edinburgh’s model, next due Spring 2028). Institutions preparing for REF2029 have a direct incentive to close this gap now, since metrics played a limited but real role in informing peer review for REF2021 and several universities’ Codes of Practice explicitly reserve the right to expand that role.

The direction of travel across the sector is unambiguous: DORA and CoARA signatory numbers continue to grow, and the Forum for Responsible Research Metrics gives every institution — large or small — a ready-made template rather than a blank page. The remaining work is not persuasion but implementation: naming a governance body, setting a review date, and publishing the document where staff undergoing assessment can actually find it.