A citation tells you that one researcher read another’s work and found it relevant enough to reference. That is a real signal, but it is a signal about scholarly attention — conversation within the academy. It says almost nothing about whether a study changed a clinical guideline, shifted a piece of legislation, altered how a community manages a resource, or simply helped people understand something they could not before. The change research makes in the wider world is what most funders now mean by ‘impact’, and it lives almost entirely outside the reference list. This article looks at what that broader notion of impact involves, how it is evidenced honestly, and where the vocabulary for describing it sits in the engagement, impact and SDG domain of the CASRAI Dictionary.
Three kinds of impact that citations miss
It helps to separate the strands that ‘impact beyond citations’ bundles together. Policy impact is the influence of research on decisions made by governments, regulators, professional bodies or international organisations — a finding cited in a select-committee report, evidence that shaped a national clinical guideline, or analysis that informed a regulatory threshold. Impact on practice is the change research produces in how things are actually done: a new protocol adopted by a profession, a tool taken up by practitioners, a method that becomes standard in an industry. Public engagement is the two-way interaction between researchers and the public — not broadcasting findings outward, but the dialogue, participation and shared sense-making through which research and society inform one another.
These categories overlap, and a single project may touch all three. What unites them is that none is captured by counting references. A guideline change leaves its trace in the guideline, not in a journal’s citation graph; a public-engagement programme leaves its trace in the people who took part.
The REF and the impact case study
The clearest institutional expression of this shift in the United Kingdom is the Research Excellence Framework (REF), which assesses impact through narrative impact case studies. An impact case study asks an institution to describe a specific, demonstrable effect that underpinning research has had on the economy, society, culture, public policy, health, the environment or quality of life — and crucially to evidence it. The structure is deliberately a story with proof attached: the underpinning research, the pathway by which it reached the people it affected, and the change that followed, each supported by sources that someone other than the author could check.
The REF model is instructive even outside the UK because it forces a discipline that metrics do not: you cannot count your way to an impact case study. You have to articulate a chain of cause and effect and then show, with corroborating evidence, that the chain holds.
Theory of Change: making the pathway explicit
The most useful planning tool in this space is a Theory of Change — an explicit account of how and why a set of activities is expected to lead to a desired outcome. Rather than assuming that good research automatically produces good in the world, a Theory of Change makes the intervening steps visible: these activities lead to these outputs, which produce these intermediate outcomes, which contribute to this longer-term change, under these assumptions about context.
Done well, a Theory of Change is honest about assumptions and about the contribution problem. Research rarely causes a social change single-handedly; it contributes alongside many other factors. A credible impact narrative claims contribution, not sole authorship of an outcome, and a Theory of Change is the structure that lets you describe that contribution without overstating it. It also doubles as an evaluation plan, because it identifies in advance what evidence would show each step actually occurred.
Evidencing impact without inventing it
The cardinal rule of impact reporting is that evidence must be real and verifiable. The temptation to dress up a plausible story with invented reach figures or unattributable quotes is exactly what discredits impact claims. Honest evidence takes recognisable forms: documents that cite or draw on the research (policy papers, guidelines, regulatory submissions); testimony from named users or beneficiaries who can confirm the effect; records of uptake such as downloads of a tool, adoptions of a protocol, or participants in an engagement programme; and independent commentary or review. The standard to aim for is corroboration — could an assessor check this against a source that is not you?
This is also where assessment reform connects. The move away from journal-based proxies toward judging research on its own merits, set out in initiatives compared in our DORA versus CoARA overview, is the same instinct applied to evaluation: assess what the work actually did, not the venue it appeared in. Contribution to impact can itself be recognised at the level of the individual, which is one reason structured contributorship through the CRediT taxonomy matters — impact work, like research work, is rarely the achievement of one person alone.
Impact and the Sustainable Development Goals
A growing number of institutions and funders frame impact against the United Nations Sustainable Development Goals (SDGs), using them as a shared language for the societal challenges research addresses. Mapping a project to one or more SDGs can make its intended contribution legible to audiences far beyond a discipline, and aligning outputs and reporting to the goals helps that signal travel. The SDGs are best treated as a framing and communication device rather than a scorecard: tagging a paper to a goal is meaningful only if the work genuinely speaks to it.
The through-line is that impact beyond citations demands a different kind of rigour, not a lower one. It asks researchers to articulate a pathway, to plan engagement as a two-way relationship rather than dissemination, and to gather verifiable evidence of change as they go. The consistent vocabulary for recording all of this — pathways, outcomes, engagement types and impact categories — is maintained in the CASRAI Dictionary, so that an impact story written in one system can be understood in another.