Funders and institutions increasingly ask researchers to demonstrate impact: the contribution of research to society, the economy, policy, health, culture, and the environment, beyond its contribution to scholarship itself. In the United Kingdom this is most visible in the impact case studies of the Research Excellence Framework. But the narrative case study, powerful as it can be, is only one instrument. A set of more structured frameworks, developed over years of methodological work, offers different ways to assess and evidence impact. Three stand out: the Payback Framework, SIAMPI, and contribution mapping.
Why impact is hard to assess
Research impact resists simple measurement for several reasons. It often unfolds over long and unpredictable timescales, so that the effects of a study may appear years or decades after publication. It frequently results from many contributions rather than a single project, making attribution difficult. And it travels through complex pathways, from a finding to a policy to a practice to an outcome, with each step shaped by factors outside the research itself. Any credible approach to impact assessment has to grapple with these problems of time lag, attribution, and causal complexity.
The Payback Framework
The Payback Framework, developed by Martin Buxton and Stephen Hanney, was among the first systematic attempts to assess the returns from research, originally in health services research. It combines a logic model of how research feeds through to benefits with a set of categories for the different kinds of payback a project can yield. These categories typically span knowledge production, benefits to future research and research use, political and administrative benefits, health and broader sector benefits, and wider economic benefits.
The framework also describes the stages through which research passes, with interfaces where it interacts with the wider world, and feedback loops that recognise impact is rarely a straight line. Its strength is structure: it gives evaluators a consistent way to ask what kinds of benefit a piece of research produced and where, rather than relying on whatever story happens to be most compelling.
SIAMPI and productive interactions
SIAMPI, which stands for Social Impact Assessment Methods through the study of Productive Interactions, takes a different angle. Rather than trying to measure ultimate impacts directly, which is notoriously difficult, it focuses on the productive interactions between researchers and stakeholders that make impact possible. The reasoning is that impact arises through relationships and exchanges, such as collaborations, advisory roles, the use of research in practice, and the contributions can be observed and assessed even when the final outcomes are diffuse or delayed.
SIAMPI distinguishes between direct or personal interactions, indirect interactions through texts and artefacts, and financial interactions. By looking at the quality and reach of these interactions, evaluators can build evidence of the mechanisms through which research influences the world, sidestepping some of the attribution problems that bedevil outcome-focused approaches.
Contribution mapping
Contribution mapping is grounded in the idea of contribution analysis, which asks not whether research caused an outcome on its own, an often impossible claim, but whether and how it plausibly contributed alongside other factors. Applied to research, contribution mapping traces the process from a research project through the people and activities involved to eventual changes, mapping the actors, the linkages, and the alignment of efforts along the way. It treats impact as the result of many converging contributions and seeks to make the research’s part in that convergence visible and defensible.
Choosing and combining approaches
These frameworks are not rivals so much as complementary lenses. The Payback Framework offers a comprehensive taxonomy of benefits and a logic model to organise them. SIAMPI shifts attention to the interactions that generate impact, which is especially useful where outcomes are long-term or shared. Contribution mapping provides a principled way to talk about a research project’s contribution without overclaiming sole causation. In practice, a thoughtful evaluation may draw on more than one, using a logic model to frame the analysis, evidence of productive interactions to show mechanisms, and contribution reasoning to make a measured causal claim.
Evidencing impact responsibly
The common message across these approaches is that impact should be evidenced rather than merely asserted. A persuasive impact narrative is strongest when it rests on a clear account of how the research connected to the changes claimed, supported by documentation of interactions, intermediate steps, and contributions. Linking that account to the underlying outputs, including well-cited datasets and openly shared FAIR data, and recording who did what through frameworks such as CRediT, strengthens the chain of evidence. The standards and vocabularies catalogued in the CASRAI data dictionary help describe these contributions consistently, so that impact assessment rests on a traceable record rather than a well-told story alone.
Leave a Reply