CASRAI Dictionary

Tag: dora research assessment

  • Field-Weighted Citation Impact: Where It Fails

    Field-weighted citation impact (FWCI) is a Scopus-derived metric that divides a publication’s actual citation count by the citation count expected for similar documents in the same subject field, publication year and document type — a result of 1.0 marks the global average, above 1.0 marks above-average impact. Before an institution builds review, promotion or tenure (RPT) criteria around it, the underlying normalisation assumptions need scrutiny.

    Field-weighted citation impact is defined by Elsevier as the ratio of citations actually received by an output to the citations that would be expected based on the average for the global scientific output of the same subject field, publication year and document type. It is calculated using Scopus data and surfaced through SciVal and Pure.

    What is field-weighted citation impact?

    Field-weighted citation impact is a normalised, article-level citation metric built into Elsevier’s SciVal and Scopus platforms. It expresses how a specific output, author, or institution has been cited relative to a global benchmark of comparable publications, rather than in raw citation counts that inevitably favour older papers and citation-heavy fields such as biomedicine.

    An FWCI of 1.48 means a document has been cited 48% more than expected for its field, year and type. An FWCI of 0.6 means it has been cited 40% less than expected. Because the benchmark is fixed at 1.0 by construction, roughly half of all outputs in any given field will sit below that line — a distributional fact that is frequently lost in institutional reporting.

    How is FWCI calculated?

    The field-weighted citation impact formula is simple on its face: FWCI = actual citations received ÷ expected citations for similar documents. The “expected” figure is the average citation count for all Scopus-indexed documents sharing the same Scopus subject classification (ASJC code), publication year, and document type (article, review, conference paper, and so on).

    • A microbiology article published in 2023 that has received 20 citations, against a field average of 10 for similar 2023 microbiology articles, scores an FWCI of 2.0.
    • A humanities article with 3 citations against a field average of 2 scores an FWCI of 1.5 — a superficially similar score built on a far smaller, more volatile citation base.
    • SciVal aggregates FWCI across an author’s or institution’s full output set by summing actual citations and expected citations separately, then dividing the totals — not by averaging individual FWCI scores.

    This matters: a single highly cited outlier can lift a whole portfolio’s FWCI, which is why SciVal documentation recommends reading FWCI alongside output volume and citation distribution, not as a standalone score.

    FWCI vs CiteScore and the Journal Impact Factor

    FWCI is often confused with journal-level metrics because all three numbers look similar — a decimal hovering near 1 to 10. They measure different things at different units of analysis, which is the first source of misapplication in policy documents.

    Metric Unit of analysis Field-normalised? Source and window
    Field-weighted citation impact (FWCI) Article, author, or institution Yes — field, year, document type Scopus data via SciVal; typically a rolling multi-year citation count
    CiteScore Journal No Elsevier/Scopus; launched December 2016; citations in a year to the prior 3 years of documents
    Journal Impact Factor (JIF) Journal No Clarivate Journal Citation Reports; historically a 2-year citation window

    Neither CiteScore nor the JIF adjusts for subject field, so comparing a mathematics journal’s CiteScore to an oncology journal’s compares citation cultures, not quality. FWCI’s field normalisation is what DORA-aligned reformers have asked journal metrics to do and mostly do not — which is also why FWCI is sometimes waved through review committees as the “responsible” metric without further scrutiny.

    Where FWCI breaks down: five assumptions to scrutinise

    FWCI’s field normalisation is a genuine improvement over raw citation counts and journal-level proxies, but it inherits several assumptions that DORA-aligned institutions should test before writing it into criteria.

    • Mean-based benchmarking, not percentile-based. FWCI compares an output to the field average, but citation distributions are heavily right-skewed: a small number of highly cited papers pull the mean upward, so most papers structurally score below 1.0 even when performing typically for their field. This is precisely why the Centre for Science and Technology Studies (CWTS) at Leiden University uses percentile-based indicators, such as the share of a unit’s output in the global top 10% most-cited, in its Leiden Ranking methodology rather than a mean-normalised ratio.
    • Subject classification is assigned to journals, not articles. Scopus’s ASJC subject codes are largely applied at the source-title level. An interdisciplinary article published in a broad-scope journal inherits that journal’s field classification, which can misrepresent the “expected” citation benchmark for a genuinely cross-disciplinary piece of work.
    • Small-sample volatility. For low-citation fields (much of the humanities, parts of engineering and mathematics) or for single articles, a difference of one or two citations can swing FWCI dramatically, because the expected-citation denominator is itself small. A score of 2.0 built on 20 citations is far more stable than one built on 2.
    • Self-citation is not excluded by default. Author, institutional, and journal self-citation inflate the numerator unless a self-citation exclusion is explicitly applied — a configurable option in SciVal, but one that is easy to omit when scores are pulled into a spreadsheet for a committee.
    • A single number cannot represent research quality, originality, or societal value. FWCI measures citation uptake within a fixed window; it says nothing about methodological rigour, reproducibility, data sharing, or the qualitative judgement DORA asks assessors to exercise in its place.

    Should FWCI drive review, promotion and tenure decisions?

    The San Francisco Declaration on Research Assessment (DORA), issued in 2012, recommends that institutions not use journal-based metrics as a surrogate for the quality of individual articles, individual researchers’ contributions, or as inputs to hiring, promotion, and funding decisions. FWCI’s article-level, field-normalised design addresses DORA’s specific objection to journal-level proxies such as the JIF — but it does not exempt FWCI from DORA’s broader principle that quantitative indicators should supplement, not replace, expert reading of the work itself.

    Institutions building RPT criteria around FWCI should require committees to read the underlying subject classification applied to a candidate’s outputs, check whether self-citations are excluded, and treat single-digit-citation scores as statistically unstable rather than definitive. A candidate’s FWCI trend across a full portfolio, read alongside narrative evidence of contribution, is a materially more defensible signal than a single score cited in isolation.

    As UK Research and Innovation and equivalent funders continue to align assessment frameworks with responsible-metrics principles, institutions that document how they weight FWCI against qualitative peer judgement — rather than adopting it as a pass/fail threshold — will be better positioned to defend their research administration processes to auditors, funders, and appeals panels alike.

    Frequently asked questions

    What is the average FWCI?

    The global average FWCI is always 1.0 by mathematical construction, because the benchmark for “expected citations” is itself the average of comparable outputs. A score above 1.0 indicates above-average citation performance for that field, year, and document type; a score below 1.0 indicates below-average performance.

    How do I get my field-weighted citation impact?

    FWCI is retrieved through a SciVal subscription, where institutional users can search an author, publication set, or institution and view the FWCI directly on the metrics dashboard. Some institutions also surface FWCI through Pure, which synchronises the metric from Scopus on a scheduled basis where the integration is enabled.

    What is field-weighted citation impact ranking?

    FWCI is not itself a ranking system — it is a ratio, not a percentile or league-table position. Institutions sometimes rank authors, departments, or outputs by their FWCI scores internally, but this practice inherits all the mean-based and small-sample limitations described above and should be treated cautiously.

    Is field-weighted citation impact the same as CiteScore?

    No. FWCI operates at the article, author, or institution level and is field-normalised; CiteScore is a journal-level average citation rate with no field normalisation. A journal’s CiteScore says nothing about how any single article within it actually performed relative to its field.

    FWCI remains one of the more defensible citation metrics precisely because it was built to correct the field-blindness of journal-level indicators. Its value depends entirely on institutions applying it the way its own documentation recommends: alongside output volume, subject classification checks, and self-citation controls — not as a solitary number standing in for expert judgement in a promotion file.

  • Journal Impact Factor Reform After DORA, CoARA: What the Evidence Shows

    Journal impact factor reform is the shift, led by the San Francisco Declaration on Research Assessment (DORA) and the Coalition for Advancing Research Assessment (CoARA), away from journal-level metrics as proxies for individual researcher quality in hiring, promotion and tenure. Published institutional policy audits show the shift is real but partial: peer-reviewed analysis of promotion documents finds journal impact factor (JIF) language persists at a large minority of research-intensive universities even where DORA has been signed.

    DORA is a 2012 declaration, now signed by more than 25,000 individuals and organisations worldwide, that asks institutions to stop using the JIF as a substitute measure of the quality of individual research articles in funding, appointment and promotion decisions. CoARA, launched in 2022 under the Agreement on Reforming Research Assessment (ARRA), extends the same principle into a formal European coalition with published implementation commitments and a 2029 systemic-reform deadline for signatory organisations.

    What DORA and CoARA Actually Require

    DORA and CoARA are not enforcement bodies; they are voluntary commitments that institutions self-implement. Neither has an audit or sanction mechanism of its own, which is precisely why independent policy audits — not signatory counts — are the only reliable evidence of actual practice change.

    Feature DORA CoARA
    Launched 2012 (San Francisco) 2022, under the ARRA agreement
    Core mechanism Voluntary declaration; no journal-level metrics as a proxy for article or researcher quality Ten formal commitments plus a published multi-year action plan per signatory
    Geographic base Global Predominantly European, growing globally
    Compliance check None — self-reported case studies only National chapters and working groups; EC-commissioned baseline studies
    Reported uptake 25,000+ individual and organisational signatories Around 450 European higher-education institutions, roughly 13% of the sector, per the CoARA-by-numbers uptake study

    Does Signing DORA or CoARA Actually Change Hiring Practice?

    The most direct published audit remains McKiernan et al.’s 2019 eLife meta-research study, which coded the review, promotion and tenure (RPT) documents of 129 US and Canadian universities. The findings are the clearest evidence-based answer available on this question.

    • The JIF was mentioned in RPT documents at 23% of all sampled institutions, rising to 40% among research-intensive (R-type) universities.
    • Of institutions that mentioned the JIF, 87% used it in a supportive context — encouraging its use — and none explicitly prohibited it, even where the parent university had signed DORA.
    • Where JIF appeared, 63% of mentions linked it to “quality”, 40% to impact or significance, and 20% to prestige or reputation — the exact conflation DORA was written to dismantle.

    A follow-up 2024 DORA-commissioned qualitative study of US faculty hiring and tenure assessments reached a similar conclusion: departmental practice frequently lags institutional signature, because RPT criteria are set and applied at department or faculty level, not centrally by the office that signed the declaration. This decentralisation gap is the single most consistent finding across the audit literature and is the primary reason blanket “signed DORA” claims cannot be read as evidence of changed practice.

    Which Institutions Have Implemented Reform, and How

    Where reform has taken hold, it has required a specific policy rewrite, not just a signature. Three documented examples illustrate the range of implementation depth.

    • Utrecht University (Netherlands) signed DORA in 2019 and, by 2022, had formally removed the JIF from all hiring and promotion criteria university-wide as part of its Recognition and Rewards programme, replacing it with team-science and open-science indicators — reported by Nature in July 2021.
    • University of Calgary revised its GFC Academic Staff Criteria and Processes Handbook after signing DORA, explicitly incorporating DORA’s principles into the formal criteria used in tenure and promotion committees, according to the university’s own published case study.
    • CoARA national chapters, including Spain’s, have published sector-wide mapping reports tracking which member institutions have moved from commitment to documented policy change, rather than relying on signature counts alone.

    At the coalition level, the European Commission’s independently commissioned ARRA baseline study — published in mid-2026 — found that engagement is broad and growing but that implementation progress across signatory organisations is uneven and constrained by limited institutional capacity, particularly outside research-intensive universities and outside Western Europe.

    Why Reform Stalls: Barriers Identified in the Audits

    The audit literature converges on a consistent set of structural barriers, distinct from a simple lack of institutional will.

    • Decentralised authority. Central research offices sign declarations; departments and faculties write and apply RPT criteria, creating an implementation gap that persists for years.
    • No enforceable alternative metric. DORA and CoARA prescribe what institutions should stop doing more clearly than what should replace it, leaving evaluators to fall back on familiar journal-based shortcuts under time pressure.
    • Reviewer and panel habit. External referees and appointment panels — often from non-signatory institutions — continue to reference journal prestige informally, even where the host institution’s written policy is silent on the JIF.
    • Absence of external audit. Because neither DORA nor CoARA verifies signatory compliance, self-reported case studies dominate the evidence base, which is why the McKiernan-style document audit remains the field’s methodological benchmark.

    Common Questions on DORA, CoARA and Impact Factor Reform

    Has signing DORA actually changed hiring practice at universities?

    Partially. Published audits of promotion and tenure documents show the journal impact factor still appears at roughly a quarter to two-fifths of institutions, including many DORA signatories, because departmental criteria often lag the institutional-level declaration by years.

    What is the difference between DORA and CoARA?

    DORA is a global, individually signed declaration with no formal implementation mechanism, launched in 2012. CoARA is a European-centred coalition with ten binding commitments and a published multi-year action plan per signatory institution, launched in 2022.

    Do DORA-signatory universities still use the impact factor in tenure decisions?

    Some do. A 2019 eLife audit found 87% of institutions that mentioned the JIF in tenure documents used it in a supportive, encouraging context, regardless of DORA status, showing that signature alone does not remove journal metrics from evaluation practice.

    What replaces the impact factor under responsible research assessment reform?

    Reformed institutions typically adopt narrative CVs, qualitative peer review, and broadened output categories — datasets, software, mentorship, open-science contributions — rather than a single quantitative substitute metric, per CoARA’s core commitments.

    Implications for Research Administrators and Institutional Leaders

    For research administrators, the audit evidence carries a practical conclusion: a DORA or CoARA signature is a governance commitment, not a completed policy change. Verifying reform therefore requires the same document-level audit McKiernan’s team used — checking actual RPT, hiring and promotion wording for JIF language — rather than relying on signatory-list membership as a proxy for compliance.

    Institutions serious about research assessment governance should treat departmental RPT criteria, external referee guidance and appointment panel training as the three concrete levers that determine whether a declaration changes behaviour. Coalition membership sets direction; department-level document rewrites are what the evidence shows actually moves practice.

    The trajectory across both audits and coalition-level reporting points toward continued, uneven reform rather than wholesale abandonment of journal metrics. CoARA’s 2029 milestone and the growing base of institutional case studies mean the evidence base for measuring real change — as opposed to signed intent — will keep expanding over the next several years.

  • Responsible Use of Metrics: Comparing UK University DORA Guidance

    The responsible use of metrics means applying quantitative research indicators — citation counts, field-weighted citation impact, grant income — only to inform and support expert peer judgement, never to replace it, in line with the San Francisco Declaration on Research Assessment (DORA). Cambridge, Exeter, Edinburgh and UCD have each published DORA-aligned guidance for their own institutions, but a side-by-side reading shows the four documents converge on principle and diverge sharply on governance, prescriptiveness and review discipline.

    Responsible research metrics is the umbrella term for institutional policies that constrain how bibliometric and altmetric indicators may be used in hiring, promotion and funding decisions, so that no single number is treated as a proxy for research quality.

    What does “responsible use of metrics” actually require?

    DORA’s own guidance on the responsible use of quantitative indicators sets out five criteria that any institutional policy should meet: metrics use should be clear, be transparent, be specific, be contextual, and be fair, according to sfdora.org’s published guidance document. Separately, the UK’s Forum for Responsible Research Metrics — convened by Universities UK following the 2015 Metric Tide report — frames the same territory as five R’s: robustness, humility, transparency, diversity and reflexivity.

    Every institutional statement reviewed here traces back to the same three source documents: DORA (2012), the Leiden Manifesto for Research Metrics (2015), and the Metric Tide (2015, updated as Harnessing the Metrics Tide in 2022). What differs is how each university translates those shared principles into binding local policy — and that is where the real variation, and the real risk of inconsistent practice, sits.

    Where Cambridge, Exeter, Edinburgh and UCD guidance converges

    All four institutions state unambiguously that quantitative metrics must support, not supplant, qualitative expert assessment. All four are DORA signatories and all four explicitly rule out using the Journal Impact Factor as a proxy for the quality of an individual output or researcher.

    • Metrics must be applied at the correct level of granularity — never using a journal-level or institution-level number to judge an individual.
    • Comparisons between individuals must account for career stage, career breaks and part-time working.
    • Any metric used in assessment must be disclosed in advance to the people being assessed.
    • Metrics and their underlying datasets must be periodically reviewed for continued fitness of purpose.

    Edinburgh and Exeter are also both signatories to the Coalition for Advancing Research Assessment (CoARA), which each joined in 2022, committing them additionally to phasing out inappropriate use of the h-index alongside the Journal Impact Factor.

    Where the four institutions’ guidance diverges

    Beneath the shared principles, the four documents take genuinely different institutional forms — a distinction that matters more than the principles themselves for anyone trying to replicate or benchmark a policy.

    Institution Format Governance body Distinguishing feature
    University of Cambridge High-level institutional principles, devolved Individual Schools and Faculties No single university-wide rulebook; Schools write discipline-specific DORA implementation policies
    University of Exeter Nine enumerated principles (published April 2022) Responsible Metrics Champions Group, plus a DORA Champions network in Colleges and Services Explicitly built on the UCL Principles for the responsible use of bibliometrics as its starting template
    University of Edinburgh Five numbered institutional commitments (“The University will…”) Research Policy Group (2019 approval); Research Strategy Group (2025 re-approval) Only one of the four with a published review cadence — first approved April 2019, reviewed May 2025, next review Spring 2028
    University College Dublin Single institutional statement synthesising three peer frameworks Working Group on the Responsible Use of Research Metrics, reporting to the Research, Innovation and Impact Group (RIIG) Most explicit on equality, diversity and inclusion factors — names career breaks, statutory leave and part-time working directly in the policy text

    Exeter’s document is the most technically granular of the four, naming specific indicator products — Field-Weighted Citation Impact from Scopus/SciVal, Field Citation Ratio and Relative Citation Ratio from Digital Science, Category Normalised Citation Impact from Web of Science — and warning explicitly against mixing metrics from different bibliometric providers within the same assessment exercise. Edinburgh is the most procedurally binding, with a stated review cycle and a named committee for re-approval. Cambridge is the most devolved, deliberately declining to impose a single university-wide metric policy in favour of discipline-appropriate local rules. UCD is the most EDI-forward, embedding equity language directly into its core commitments rather than treating it as a supporting principle.

    What gaps remain for institutions without a dedicated policy?

    Cambridge, Exeter, Edinburgh and UCD each have a named committee, a published document and — in Edinburgh’s case — a fixed review date. Many smaller and teaching-intensive institutions have none of this. Several UK universities that rank prominently for “responsible use of metrics” searches — including library subject guides from institutions such as Derby, Plymouth and Sunderland — publish summaries of DORA’s principles rather than institutionally approved governance statements.

    That distinction is not cosmetic. A library guide can explain what responsible metrics are; it cannot bind a promotion committee the way a document approved by a Research Policy Group, a Champions Group or an RIIG can. Institutions without a dedicated policy and a named approving body carry a structural gap: staff have no enforceable assurance that a hiring panel or REF preparation exercise will actually follow the principles a library page describes. For research administrators at smaller institutions, the practical route is not to draft new principles from scratch but to adapt an existing framework — Exeter’s document explicitly credits the UCL Principles as its own starting point, and UCD’s statement was built after reviewing three existing peer institutions’ policies, showing that adaptation, not original drafting, is the established norm.

    Answer-first Q&A on research metrics

    What are the four types of metrics used in research assessment?

    Institutional guidance, including Exeter’s, groups research indicators into institutional or discipline-level indicators (rankings, field-weighted citation impact), output-level indicators (citation counts, Journal Impact Factor, altmetrics), research activity indicators (grant income, PGR numbers) and individual-focused indicators (h-index, highly-cited rankings) — each requiring different safeguards against misuse.

    What is the use of metrics in responsible research assessment?

    Metrics provide contextual, supporting evidence alongside qualitative peer review — never a standalone verdict. Under DORA and the institutional statements reviewed here, quantitative indicators may inform hiring, promotion and funding decisions only when disclosed in advance, appropriately normalised and applied at the correct level of granularity.

    What are examples of responsible research metrics?

    Commonly cited examples include field-weighted citation impact, altmetrics, grant income and postgraduate research supervision counts, used as part of a discipline-appropriate “basket of measures” rather than in isolation. Journal Impact Factor and raw h-index are explicitly excluded as individual-level proxies by every institution examined here.

    Implications for research administrators

    For research administrators, the comparison points to a practical hierarchy of maturity: a published statement with no named governance body (the entry point most smaller institutions can reach quickly); a statement with a standing committee (Exeter’s Champions Group, UCD’s Working Group reporting into RIIG); and a statement with a fixed, published review cycle (Edinburgh’s model, next due Spring 2028). Institutions preparing for REF2029 have a direct incentive to close this gap now, since metrics played a limited but real role in informing peer review for REF2021 and several universities’ Codes of Practice explicitly reserve the right to expand that role.

    The direction of travel across the sector is unambiguous: DORA and CoARA signatory numbers continue to grow, and the Forum for Responsible Research Metrics gives every institution — large or small — a ready-made template rather than a blank page. The remaining work is not persuasion but implementation: naming a governance body, setting a review date, and publishing the document where staff undergoing assessment can actually find it.

  • Forum for Responsible Research Metrics Explained

    The Forum for Responsible Research Metrics is the UK’s national coordinating body for the responsible use of research metrics. Established in 2016 following the 2015 Metric Tide review, it advises UK higher education funding bodies on metrics use in the Research Excellence Framework (REF) and promotes alignment with DORA and CoARA principles.

    The Forum for Responsible Research Metrics is an independent, sector-wide group of UK research funders, sector bodies and infrastructure experts convened to promote fair and transparent use of quantitative indicators in research assessment. It sits alongside — but is distinct from — the global DORA declaration and the European CoARA coalition, and its guidance shapes how UK institutions design metrics policy ahead of the REF.

    What is the Forum for Responsible Research Metrics?

    The Forum for Responsible Research Metrics is a UK sector body, not a regulator. It has no statutory powers and cannot compel institutions to adopt any metric or policy. Instead it functions as an advisory and convening body, bringing together funders, universities and data-infrastructure providers to agree shared principles for using metrics responsibly in research assessment.

    Its core functions, as set out at its founding, are threefold: advise the UK higher education funding bodies on metrics use in the REF; provide advocacy and leadership on responsible metrics within the UK sector; and establish links with equivalent international initiatives. The Forum is administratively supported by Universities UK (UUK), which convenes its meetings and publishes its outputs.

    When and why was the Forum established?

    The Forum’s origins trace directly to a government-commissioned review. In 2014 the then Higher Education Funding Council for England (HEFCE) — now folded into UKRI’s Research England — commissioned an independent expert group to examine the use of metrics in research evaluation, particularly within the REF. The resulting report, The Metric Tide, was published on 9 July 2015.

    The Metric Tide concluded that metrics could be a useful adjunct to peer review but warned against their use as a substitute for expert judgement, and recommended that institutions consider signing DORA or applying its principles. It also recommended that a UK-wide body be established to advise funders on metrics use in the REF, provide sector leadership, and build international links. The Forum for Responsible Research Metrics was convened in 2016 to fulfil that recommendation.

    • 2014 — HEFCE commissions the independent metrics review
    • 9 July 2015 — The Metric Tide report is published
    • 2016 — The Forum for Responsible Research Metrics is convened, supported by Universities UK
    • 2021 — Forum advice informs metrics use across the three REF2021 assessment elements
    • December 2022 — Sector commentary (Wonkhe) calls for an expanded Forum remit, including holding data providers to account

    How does the Forum relate to DORA and CoARA?

    The Forum, DORA and CoARA are three distinct bodies with overlapping but separate mandates, and conflating them is a common source of confusion for research offices drafting policy. The table below sets out how each operates and how they connect to one another.

    Body Founded Scope Core mechanism Link to the Forum
    Forum for Responsible Research Metrics 2016 UK-wide, advisory to HE funding bodies Advises funders on metrics use in the REF; monitors DORA/CoARA uptake —
    DORA (San Francisco Declaration on Research Assessment) 2013 International, institution/publisher signatory declaration Signatories pledge not to use the Journal Impact Factor as a proxy for individual quality in hiring, promotion or funding decisions The Metric Tide recommended UK institutions sign DORA; the Forum promotes and tracks its adoption
    CoARA (Coalition for Advancing Research Assessment) 2022 Pan-European coalition, member commitments Signatories commit to reforming assessment criteria and procedures over a defined implementation period Complementary European framework; several UK signatories hold both DORA and CoARA membership alongside Forum-aligned institutional policy
    REF (Research Excellence Framework) 2014 (successor to the RAE) UK-wide, all HE institutions Periodic peer-review-led assessment of research quality, impact and environment The Forum’s core client — its advice shapes how funding bodies use metrics within REF criteria

    In practice, the Forum does not ask institutions to sign anything. DORA and CoARA are commitments an institution opts into; the Forum’s guidance is advisory input into how UK funders design and apply metrics within a statutory national exercise.

    What is the Forum’s role in the REF?

    The Forum’s most concrete, recurring function is advising UK higher education funding bodies on how metrics should — and should not — be used within the REF’s three assessment elements: outputs, impact and environment. This advice fed directly into REF2021 guidance and is expected to inform preparation for the next exercise, REF 2029.

    UKRI’s Research England states that the Forum works to improve the data infrastructure underpinning metric use and the broader culture around research metrics, not just the rules for a single assessment cycle. That distinction matters for research offices: Forum guidance is a standing reference point for metrics governance, not a one-off REF submission checklist.

    • Outputs — metrics may inform but must not substitute for peer review of research quality
    • Impact — quantitative indicators supplement, rather than replace, narrative impact case studies
    • Environment — metrics contribute contextual evidence on research culture and infrastructure

    What does the Forum’s guidance mean for research offices?

    For research administrators building or reviewing a metrics policy, Forum guidance sets a de facto national baseline: metrics should be used transparently, contextually, and never as an automatic proxy for quality in hiring, promotion or funding decisions. This mirrors DORA’s core ask but is framed specifically for REF-facing institutional practice.

    Institutional research offices typically apply this in three ways: auditing existing use of journal-level and author-level metrics in internal review processes; documenting which indicators are used for which decisions and why; and aligning local policy statements with DORA and, where relevant, CoARA commitments so REF-facing metrics governance is not developed in isolation. Institutions building or auditing research assessment policy can find related structural guidance in CASRAI’s research administration resources.

    The December 2022 sector call to expand the Forum’s remit — including holding metrics data providers to account — signals that its scope is likely to widen beyond REF-facing advice toward broader accountability for the commercial infrastructure that supplies citation and impact data. Research offices should treat current Forum guidance as a floor, not a ceiling, when drafting policy ahead of REF 2029.

    Frequently asked questions

    Who chairs the UK Forum for Responsible Research Metrics?

    As of Universities UK’s most recently published listing, the Forum is chaired by Professor Max Lu, Vice-Chancellor of the University of Surrey. The Forum itself comprises representatives from UK research funders, sector bodies and data-infrastructure organisations, convened administratively by Universities UK.

    Is the Forum for Responsible Research Metrics the same as DORA?

    No. DORA is a global signatory declaration institutions and individuals opt into; the Forum is a UK sector body advising funders on metrics within the REF. The Metric Tide review recommended UK institutions sign DORA, and the Forum monitors and promotes that uptake without being DORA itself.

    What did the Metric Tide report recommend?

    Published 9 July 2015, The Metric Tide recommended that metrics support but never replace peer review, that institutions consider signing DORA, and that a national Forum be established to advise funders on responsible metrics use in the REF and build international links.

    Does the Forum’s advice apply to REF 2029?

    Yes. The Forum’s advisory role is standing, not exercise-specific, and its guidance on metrics in outputs, impact and environment assessment is expected to inform funding-body preparation for REF 2029 as it did for REF2021.

    What’s next for responsible metrics in the UK?

    With REF 2029 preparation underway and sector pressure to widen the Forum’s remit toward data-provider accountability, UK research offices should expect Forum guidance to evolve rather than stay fixed. Institutions that align internal metrics policy with Forum principles now, rather than at the point of REF submission, will face less rework as that remit expands.

  • DORA vs CoARA: What Administrators Should Know

    DORA vs CoARA are two distinct but connected research-assessment reform initiatives. DORA — the San Francisco Declaration on Research Assessment, launched in 2012 — is a voluntary declaration that asks signatories to stop using journal-based metrics such as the Journal Impact Factor as a proxy for research quality. CoARA — the Coalition for Advancing Research Assessment, formed in 2022 out of the EU-anchored Agreement on Reforming Research Assessment — asks member organisations to go further: sign ten explicit commitments and publish an implementation action plan. For research administrators deciding where to commit institutional resources, DORA sets the principle; CoARA sets the practice.

    DORA is best understood as a global statement of intent. CoARA is best understood as a structured, governed coalition with working groups, national chapters and reporting obligations. Many organisations do both — CoARA itself describes DORA as a foundational influence rather than a competing framework.

    Contents

    What is DORA?

    The San Francisco Declaration on Research Assessment (DORA) is a declaration developed in December 2012 at the American Society for Cell Biology annual meeting in San Francisco. DORA is a set of recommendations against using journal-based metrics — chiefly the Journal Impact Factor — as a proxy for the quality of individual articles, researchers or institutions.

    The declaration sets out one general recommendation plus tailored recommendations for five stakeholder groups: funding agencies, institutions, publishers, organisations that supply metrics, and researchers themselves. Its central instruction is consistent across all five: assess research on its own scientific merits, not on the reputation of the journal it appears in.

    DORA operates as a signature-based declaration hosted at sfdora.org. There is no membership fee, no mandatory reporting cycle and no central secretariat enforcing compliance — an organisation or individual signs, and DORA relies on public accountability and community pressure to drive change.

    What is CoARA?

    The Coalition for Advancing Research Assessment (CoARA) is a coalition of research funders, universities, national academies and other research organisations that formed in 2022 around the Agreement on Reforming Research Assessment (ARRA), finalised that July with support from the European Commission and Science Europe. CoARA is a formal coalition, not a signature-only declaration: joining requires accepting ten explicit commitments and developing a published action plan.

    The ARRA’s ten commitments split into two tiers. Four “core commitments” are mandatory for every signatory, covering peer review as the primary basis of assessment, abandoning inappropriate use of journal- and publication-based metrics including the Journal Impact Factor and h-index, and rejecting the use of institutional rankings in assessment decisions. Six further commitments allow signatories to choose their own pace and approach.

    CoARA is governed through a General Assembly, a Steering Board and thematic Working Groups, and it operates national and regional chapters — including a UK National Chapter co-led by the Universities of Strathclyde, Loughborough and Swansea, according to the Association of Research Managers and Administrators (ARMA).

    DORA vs CoARA: key differences

    The two initiatives address overlapping goals through very different mechanisms. The table below summarises the practical distinctions an administrator needs before choosing where to commit.

    Feature DORA CoARA
    Founded 2012, San Francisco 2022, EU-anchored coalition
    Founding document San Francisco Declaration on Research Assessment Agreement on Reforming Research Assessment (ARRA)
    Form of commitment Voluntary signature Formal membership with ten commitments
    Primary target Misuse of journal-based metrics (e.g. Journal Impact Factor) Whole-system reform of research careers and evaluation
    Governance No central secretariat; community-driven General Assembly, Steering Board, working groups, national chapters
    Follow-up obligation None mandatory Published implementation action plan required
    Geographic anchor Global, unaffiliated Originated in Europe; open to global membership

    How CoARA builds on DORA

    CoARA does not compete with DORA — it explicitly builds on it. Both organisations issued a joint statement in December 2025 describing themselves as “two of a diverse and global group of initiatives that share the aim of driving systemic change towards better research assessment,” according to CoARA’s own news pages. Many CoARA member organisations were DORA signatories first.

    Three concrete extensions distinguish CoARA’s approach from DORA’s:

    • From declaration to action plan. DORA asks for a signature; CoARA requires signatories to publish an action plan implementing the ten ARRA commitments within an agreed timeframe.
    • From one metric problem to whole-system reform. DORA’s scope centres on journal-based metrics. CoARA’s core commitments extend to peer review practice, diversity of research outputs, and the rejection of institutional rankings as an assessment tool.
    • From individual signature to governed coalition. DORA has no membership structure beyond its signatory list. CoARA runs a General Assembly, Steering Board, thematic working groups and national chapters — including the UK chapter co-led by Strathclyde, Loughborough and Swansea — that coordinate implementation and share practice across members.

    Which should your institution commit to?

    For most research organisations this is not an either/or choice. DORA signature carries low administrative overhead and signals a clear public position against metric misuse — a reasonable first step for any institution, funder or publisher. CoARA membership is the heavier commitment: it requires governance capacity to produce and report against an action plan, and it suits institutions ready to reform hiring, promotion and evaluation processes at a systemic level, not just at the level of individual metrics.

    Institutions with limited capacity should sign DORA first and use it to build internal consensus before taking on CoARA’s action-plan obligations. Institutions already running research-culture or REF-adjacent reform programmes are better placed to join CoARA directly, since the ten commitments map closely onto work many UK universities are already doing through national chapters and INORMS-linked evaluation groups.

    Answer-first Q&A

    What does DORA stand for in research?

    DORA stands for the San Francisco Declaration on Research Assessment, launched in December 2012. It calls on funders, institutions, publishers and researchers to stop using journal-based metrics such as the Journal Impact Factor as a substitute for evaluating the actual scientific content of research outputs.

    What is CoARA and how does it differ from DORA?

    CoARA is the Coalition for Advancing Research Assessment, a 2022 coalition built on the Agreement on Reforming Research Assessment. Unlike DORA’s voluntary signature, CoARA requires member organisations to accept ten formal commitments and publish an implementation action plan, coordinated through national chapters and working groups.

    What are the DORA principles?

    DORA’s core principle is that research quality must be assessed on its own merits, not on the venue where it is published. Its recommendations cover funding agencies, institutions, publishers, metrics suppliers and researchers, each urged to eliminate journal-based metrics from funding, hiring and promotion decisions.

    Does DORA apply in the UK?

    DORA is a voluntary global declaration, not a UK legal requirement, but numerous UK universities and funders are signatories. UK institutions increasingly reference DORA and CoARA together in responsible-metrics policies connected to REF-related research culture and assessment reform work.

    Implications for research administrators

    The practical takeaway for administrators is a sequencing question, not a binary choice. DORA signature is fast, low-cost and a credible public marker of intent. CoARA membership is a governance commitment that touches hiring panels, promotion criteria and institutional reporting cycles, and it demands sustained capacity from a research-culture or research-strategy office.

    As responsible research assessment moves from advocacy into funder and institutional policy — with UNESCO, Science Europe and national funders increasingly referencing both frameworks — administrators should expect DORA and CoARA to be treated as complementary layers: DORA the founding principle, CoARA the operational coalition that turns it into an implementation plan.

    For related context on how research contributions are formally recognised alongside assessment reform, see CASRAI’s overview of research administration practice.