A research evaluation framework is a structured system used by a government, funder, or institution to assess the quality, productivity, or impact of research, against defined criteria, at a defined unit of analysis (a national system, an institution, a department, or an individual researcher). The term covers two structurally different families of framework that are often discussed together but serve different purposes: large-scale, government-run performance-based research funding systems that allocate money based on assessed output, and responsible-metrics reform frameworks that set principles for how any evaluation, at any scale, should and should not use quantitative indicators.
This page maps the landscape and links through to CASRAI’s detailed coverage of each specific framework — the UK’s Research Excellence Framework (REF), the San Francisco Declaration on Research Assessment (DORA), and the Coalition for Advancing Research Assessment (CoARA) — rather than duplicating them.
The two families of framework
1. National performance-based research funding systems (PRFS). These are periodic, government-mandated exercises that evaluate research produced across an entire national higher-education sector and tie the result — directly or indirectly — to public funding allocation. Submitting institutions select which outputs and staff to include, and assessment typically combines expert peer review with, in some systems, bibliometric indicators. The UK’s REF is the longest-running example; Italy, Australia, and New Zealand run or have run comparable exercises.
2. Responsible-metrics / research-assessment-reform frameworks. These are not government funding mechanisms. They are voluntary commitments — signed by individual institutions, funders, publishers, and societies — that set principles for how research should be assessed, with a shared core argument: journal-level and article-level metrics (like the Journal Impact Factor) are frequently misapplied as proxies for the quality of an individual article, researcher, or grant application, and evaluation practice should reduce or eliminate that misuse. DORA, CoARA, and the Metric Tide review sit in this family.
A national PRFS and a responsible-metrics commitment are not mutually exclusive — UKRI, which runs the REF, is itself a CoARA signatory, and REF 2029’s own assessment criteria explicitly move away from journal-based proxies. The two families increasingly shape each other rather than operating in isolation.
Core components any evaluation framework defines
- Unit of assessment — what is actually being scored: a whole institution, a subject-based unit of assessment (as in REF), an individual researcher’s narrative CV, or a single grant application.
- Evidence types accepted — peer-reviewed outputs, non-traditional outputs (software, datasets, exhibitions), narrative statements of contribution, and — in REF specifically — impact case studies describing real-world effects beyond academia.
- Indicators used, and how — whether quantitative metrics inform peer-review panels, replace them for some purpose, or are explicitly excluded. This is the exact fault line responsible-metrics frameworks are built around.
- Scoring or rating structure — REF uses a star-rating scale (4* to unclassified) applied by subject-based panels; other systems use different scales or a pass/fail funding-eligibility threshold.
- Consequences — funding allocation (PRFS), reputational signal, or — increasingly — a stated commitment used in hiring, tenure, and promotion decisions.
- Cycle length — REF runs on a roughly 6-7 year cycle (REF 2014, REF 2021, REF 2029); responsible-metrics commitments are ongoing signatory relationships rather than a scored cycle.
National performance-based research funding systems, by country
| System | Country | Current status (2026) |
|---|---|---|
| REF | United Kingdom | Active. REF 2029 is underway, with a sector policy survey planned for July 2026 and submissions due autumn 2028; results expected December 2029. REF 2021 was the prior completed cycle. |
| VQR (Valutazione della Qualità della Ricerca) | Italy | Active. The VQR 2020-2024 evaluation cycle is underway, with results expected by mid-2026; the role of bibliometric indicators in the exercise has been reduced relative to earlier cycles. |
| ERA (Excellence in Research for Australia) | Australia | Discontinued; replacement still undesigned. The Australian Research Council discontinued ERA (and the related Engagement and Impact assessment) in 2023. As of 2026, three years on, no successor evaluation framework has been finalized — the ARC’s proposed concept for a future mechanism remains in consultation, with independent commentators describing it as still lacking a clearly defined design or purpose. |
| PBRF (Performance-Based Research Fund) | New Zealand | Quality Evaluation 2026 cancelled outright. New Zealand’s Tertiary Education Commission cancelled PBRF Quality Evaluation 2026 in March 2024, pending a government-commissioned University Advisory Group review of the university funding system. PBRF Quality Evaluation component funding continues to be allocated based on 2018 results in the interim, with the total fund value unchanged at NZ$315 million. |
Treat the status column as a snapshot — national research-assessment policy changes on a multi-year cycle and specifics (timelines, panel structures, funding formulas) should always be confirmed against the operating body’s own current guidance (for REF: 2029.ref.ac.uk) rather than assumed static.
Responsible-metrics and assessment-reform frameworks
DORA, launched in 2013, is a signatory-based declaration whose core recommendation is that journal-based metrics such as the Journal Impact Factor should not be used as a proxy for the quality of individual research articles, or of an individual researcher, in funding, hiring, or promotion decisions. Institutions, funders, publishers, and individual researchers can become DORA signatories, committing to specific practice changes.
CoARA, launched in 2022, is a broader coalition — concentrated in, but not limited to, Europe — built around a shared CoARA Agreement committing signatory organizations to move toward qualitative, multidimensional research assessment and away from inappropriate use of quantitative indicators, on a defined implementation timeline. Where DORA is a single declaration signed once, CoARA is a membership coalition with ongoing working groups and a structured commitment process.
The Metric Tide review (UK, 2015) and the related Leiden Manifesto for research metrics (published in Nature, 2015) are the two most-cited underlying analyses behind this reform movement — both set out principles for responsible quantitative indicator use rather than proposing a specific assessment exercise of their own, and both are frequently cited within REF and CoARA policy documents. See CASRAI’s dedicated comparison for how DORA and CoARA differ in scope and mechanism: DORA vs CoARA.
Narrative and portfolio-based evaluation
A related, more recent development is the shift toward narrative CV formats that let researchers describe contributions beyond a publication list. The UK’s Résumé for Researchers (R4RI), and the associated 2024 UKRI mandate requiring its use in most UKRI grant applications, is a direct product of the responsible-metrics movement: it replaces a metrics-heavy CV with a structured personal statement format designed to reduce reliance on publication counts and journal prestige as evaluation shortcuts.
How evaluation frameworks intersect with bibliometric infrastructure
Even reform-oriented frameworks still rely on underlying measurement infrastructure to inform peer judgment. The Journal Impact Factor, article-level metrics, and altmetrics (tracked in part through NISO’s Alternative Assessment Metrics work) remain part of the evidence base many evaluators consult — the reform position is not that metrics should never be seen, but that they should never substitute for expert judgment about a specific piece of work. CASRAI’s broader coverage of what a responsible metrics approach looks like in practice sits alongside the two practical guides on finding and interpreting journal-level metrics: Journal Impact Factor and Quartile Rankings and How to Look Up a Journal’s Impact Factor.
Frequently asked questions
Is a research evaluation framework the same thing as the REF?
No. The REF (Research Excellence Framework) is one specific national research evaluation framework, run in the UK. “Research evaluation framework” is the general term for any structured system used to assess research quality or impact — it also covers Italy’s VQR, responsible-metrics commitments like DORA and CoARA, and institution-level tenure and promotion criteria.
What’s the practical difference between DORA and CoARA?
DORA is a single declaration organizations and individuals sign, focused specifically on reducing journal-metric misuse. CoARA is a membership coalition with an agreement, working groups, and a structured multi-year implementation timeline covering a wider range of assessment-reform commitments. See the full DORA vs CoARA comparison for a side-by-side breakdown.
Do national performance-based systems like REF use the Journal Impact Factor to score outputs?
No — REF panels assess outputs through expert peer review by subject-based panels, and REF’s own guidance has explicitly discouraged the use of journal-level metrics like the Journal Impact Factor as a proxy in that review. This is one of the areas where a national PRFS and the responsible-metrics reform position overlap rather than conflict.
Why do institutions sign both a national PRFS and a voluntary framework like DORA or CoARA?
They serve different purposes. A national PRFS determines public research funding allocation and is generally mandatory for institutions receiving that funding. DORA/CoARA signatory status is a voluntary commitment about internal practice — how the institution itself evaluates its own researchers for hiring, tenure, and promotion — and is not tied to any funding formula.
