Tag: OECD policy brief

  • OECD’s Reforming Research Assessment for Better Science: A 2026 Guide for Research Offices

    The OECD’s 2026 report, “Reforming Research Assessment for Better Science,” concludes that research assessment relying on narrow publication metrics and commercial rankings distorts research culture, and it recommends that institutions cut low-value evaluation, adopt open data infrastructures, and use AI in assessment only with caution. For research offices, the report’s six policymaker recommendations translate into concrete changes to how institutional evaluation criteria, data sourcing, and staff training are run.

    Research assessment is the systematic process of monitoring, evaluating and reviewing research inputs, processes, outputs and impacts, carried out by governments, funders, universities and publishers. The OECD reforming research assessment for better science policy brief — OECD Policy Briefs No. 56, published 29 April 2026 — sets out why that process is misaligned with how science now works, and what research-performing organisations should do about it.

    What is the OECD’s 2026 report on reforming research assessment?

    “Reforming Research Assessment for Better Science” is an OECD Policy Brief (No. 56) published on 29 April 2026 that reviews why current research-assessment practices are misaligned with the evolving nature of science, and sets out six actions for policymakers and institutions. It is accompanied by a longer evidence base, OECD Science, Technology and Industry Working Paper No. 2026/7, “New Expectations and Demands from Science: Rethinking Research Assessment Frameworks,” which maps the actors, tensions and drivers behind the reform movement.

    Both documents are credited to the OECD Directorate for Science, Technology and Innovation, with Frédéric Sgard listed as the named contact. The brief carries the persistent identifier DOI 10.1787/f6202159-en; the working paper carries DOI 10.1787/0c685800-en. Neither document proposes a single replacement metric — instead, both argue for a system-level shift in how, and how often, assessment is conducted.

    Why does the OECD say metrics-based assessment needs reform?

    The OECD argues that heavy reliance on publication counts, citation rates and journal impact factors has produced perverse incentives, including a “publish or perish” culture that rewards quantity over quality. The brief cites peer-reviewed evidence — including Fanelli (2010) on publication bias and Öztürk and Taşkın (2024) on how metric-based evaluation fuels questionable publishing — to support this conclusion.

    Three specific harms are named:

    • High-risk, high-reward research is systematically undervalued because standard indicators cannot capture long-horizon payoff.
    • Transdisciplinary and societally engaged research is poorly captured by discipline-bound, publication-and-citation frameworks.
    • Assessment volume has grown faster than institutional capacity to absorb it, creating what the OECD calls research-assessment fatigue among researchers and administrators alike, a burden previously quantified in Technopolis Group’s 2015 REF Accountability Review.

    The report is equally direct about rankings. National and global university league tables, it states, “should not be used in RA” because they rely on non-transparent proprietary methods, are biased toward STEM subjects and English-language output, and — per the UN University’s Independent Expert Group 2023 Statement on Global University Rankings — can accentuate global, regional and national inequalities.

    What alternative evaluation tools and infrastructures does the OECD recommend?

    The OECD does not prescribe one alternative framework; instead, it maps nine existing international initiatives that research offices can draw on, and it names open, non-proprietary databases such as OpenAlex and Redalyc as viable substitutes for closed commercial data sources. The report’s own comparison table — reproduced and dated below — is the clearest single reference point for institutions deciding which framework to adopt or reference in policy documents.

    Initiative Year Core contribution
    DORA (San Francisco Declaration on Research Assessment) 2012 Discourages journal-based metrics as a proxy for quality; spawned the TARA practical-tools project in 2021
    Leiden Manifesto 2015 Principles and best practice for using quantitative indicators responsibly
    INORMS Research Evaluation Group 2018 SCOPE Framework for Research Evaluation and the “More than Our Rank” initiative
    FOLEC-CLACSO 2019 Regionally specific research-assessment guidelines for Latin America
    Hong Kong Principles 2019 Minimising perverse incentives; rewarding trustworthy research practice
    Science Europe Position Statement 2020 Recommendations on research assessment processes for funders
    CoARA (Coalition for Advancing Research Assessment) 2022 Agreement on Reforming Research Assessment, with global signatories
    Barcelona Declaration 2024 Advocates open research information infrastructure
    Global Research Council RRA Working Group 2024 An 11-dimension framework for responsible research assessment

    The OECD’s own recommendation is not to pick a winner among these, but to “promote sustained dialogue” between them and to have governments recognise alignment with these emerging international principles as a criterion within cyclical institutional assessment exercises.

    What should research offices do differently?

    The report’s six policymaker actions each carry a direct operational counterpart for institutional research offices, from auditing evaluation volume to renegotiating data contracts. Research administrators reading the brief should map each national-level recommendation onto an institutional equivalent:

    • Reduce assessment volume: audit which internal reviews, reports and dashboards serve a “clearly defined objective” — and retire those that do not.
    • Diversify data sources: reduce dependency on single proprietary bibliometric platforms by testing open alternatives such as OpenAlex alongside existing subscriptions.
    • Remove rankings from internal criteria: strip commercial league-table position from promotion, tenure and internal funding-allocation rubrics.
    • Govern AI use cautiously: where AI tools are piloted in peer-review triage or portfolio analysis, require transparent, explainable models and documented human oversight rather than opaque large language models.
    • Invest in staff capacity: the brief is explicit that “guidance, training and capacity building will be key” — senior administrators, librarians and peer reviewers all need structured onboarding to new evaluation frameworks, not just a policy memo.
    • Adopt proportionate methods: match the evaluation method (summative for decisions, formative for development, or a blend) to the actual purpose of each assessment exercise.

    Institutions already engaged with CASRAI’s research administration resources will recognise these as extensions of existing responsible-metrics and open-science commitments rather than a wholesale change of direction.

    Answer-first Q&A

    What is responsible research assessment?

    Responsible research assessment refers to evaluation approaches that incentivise, reflect and reward the plural characteristics of high-quality research rather than relying on narrow proxy metrics such as journal impact factor. It combines qualitative judgement with proportionate, context-appropriate quantitative indicators, following principles set out by DORA, the Leiden Manifesto and CoARA’s 2022 Agreement.

    Why does the OECD discourage the use of rankings in research assessment?

    The OECD states that national and global rankings are marketing tools built on non-transparent proprietary data and methods that are not adapted to different institutions’ profiles or purposes. Because they are biased toward STEM subjects and English-language scholarship, their use in funding or hiring decisions can exacerbate global, regional and national inequalities rather than reflect genuine research quality.

    What role should AI play in research assessment, according to the OECD?

    The OECD says AI’s role in research assessment “needs to be carefully examined” rather than adopted by default. It favours transparent, deterministic models over opaque large language models, requires ex-ante risk assessment and human oversight, and warns that AI licensing costs can quietly increase institutions’ dependency on commercial technology providers.

    How can research offices reduce the burden of research assessment?

    Research offices can reduce burden by evaluating “only what and when necessary,” in the OECD’s words — applying assessment solely where a clearly defined objective exists and a less resource-intensive process would not suffice. Matching evaluation type (summative versus formative) to actual purpose, rather than defaulting to full review, is the report’s core proportionality test.

    What happens next for research assessment reform?

    The OECD frames reform as an iterative, long-term structural transition rather than a one-off policy change, pointing to national experiments already under way as evidence. It cites Rushforth’s 2024 analysis of the Netherlands’ “Recognition and Rewards” programme and China’s institutional hybrid responses (Liang, Zhao and Li, 2024) as examples of top-down signals interacting with bottom-up institutional experimentation.

    Concrete pilots are already generating data: Luxembourg’s National Research Fund reports three years of narrative-CV use as of 2026, and UK researchers have begun assessing generative AI’s potential role ahead of the REF 2029 exercise. For research offices, the practical takeaway is that no single framework will be mandated — institutions that start testing proportionate, criteria-linked alternatives now will be better positioned as national funders and assessment bodies converge around the OECD’s six recommendations.