Altmetrics promise something seductive: a near-real-time count of the attention a research output is attracting across news, policy documents, social media, blogs, and reference managers, available within days of publication rather than the years a citation count takes to accumulate. That promise is real, and altmetrics genuinely capture forms of reach that citations miss. But the same speed and breadth that make them useful also make them easy to misread, and the gap between “attention” and “impact” is where most of the trouble lies. This article sets out what altmetrics can and cannot show. It builds on the broader treatment in the engagement, impact and SDG-alignment domain.
What altmetrics actually measure
Altmetrics — short for alternative metrics — are indicators of the online attention and engagement a research output receives, drawn from sources outside the traditional citation databases. Typical sources include mentions in news outlets and policy documents, posts and shares on social media, blog coverage, Wikipedia citations, and saves in reference managers such as Mendeley. They are usually aggregated against a specific output — identified by its DOI — and presented as a score or a breakdown by source.
The honest one-line description is this: altmetrics count attention. They tell you that an output was mentioned, shared, saved, or referenced in non-scholarly venues, and roughly where and how much. That is genuinely valuable information, and it is information that citation counts, by their nature, cannot provide.
What they are useful for
- Speed. Attention accrues within days, so altmetrics can surface early engagement long before citations could exist. For recent outputs they may be the only quantitative signal available.
- Breadth beyond academia. A citation count measures uptake by other researchers. Altmetrics can show reach into policy, news media, and public discussion — audiences a citation count is structurally blind to. For an output whose value is partly its public or policy reach, this is exactly the dimension that matters.
- Qualitative leads, not just numbers. The most useful part of an altmetric record is often not the score but the underlying mentions: which policy document cited the work, which outlet covered it, what the coverage said. Followed up, these point to specific instances of reach that can seed a genuine impact narrative.
- A complement to citations. Used alongside citation data and qualitative evidence, altmetrics add a view that the other sources lack. Their role is supplementary, not substitutive.
What they cannot show
The central caution is simple and must be stated plainly: attention is not impact, and attention is not quality. A high altmetric score means an output was talked about; it says nothing, by itself, about whether the research is sound, whether the attention was positive, or whether any real-world change followed.
- Attention can be negative. A paper widely shared because it is being criticised, debunked, or ridiculed can score highly. The count does not distinguish praise from condemnation.
- Attention is not benefit. Genuine research impact — a changed policy, an improved treatment, an adopted practice — is a downstream outcome that an attention count cannot demonstrate. Altmetrics may flag where to look for impact; they are not evidence of it.
- The numbers are gameable and biased. Social-media-derived metrics can be inflated by coordinated sharing, and they systematically favour topics, languages, and communities that are active online — which is not the same as the topics that matter most.
- Scores are not comparable across contexts. A single composite altmetric number compresses very different kinds of attention into one figure, and that figure means different things in different fields and for different output types. Comparing scores across disciplines is largely meaningless.
The responsible-metrics frame
This is where the wider movement for responsible research assessment provides the discipline that keeps altmetrics honest. The Leiden Manifesto for research metrics (2015) set out principles for the responsible use of quantitative indicators that apply directly here. Three are especially relevant to altmetrics:
- Quantitative evaluation should support, not supplant, expert qualitative judgment. Altmetrics are an input to a human assessment, never a replacement for reading the work and weighing its contribution.
- Account for variation by field. Attention patterns differ enormously between disciplines and output types; a metric must be interpreted in context, not applied as a universal yardstick.
- Avoid misplaced concreteness and false precision. A single score presented to a decimal point invites a confidence the underlying data do not support. The number is an indicator, not a measurement of worth.
The same spirit runs through the broader reform agenda — the Declaration on Research Assessment (DORA) and the Coalition for Advancing Research Assessment (CoARA) — which presses evaluators away from reliance on any single quantitative proxy and toward judging the substance of contributions. Altmetrics sit comfortably inside that frame as one more contextual signal, and sit very badly outside it as a standalone score to be maximised.
Treat an altmetric score the way you would treat a smoke alarm: useful for telling you where to look, useless as a measure of how big the fire is. The value is in the mentions it points you to, not in the number itself.
Using altmetrics well
- Read the mentions, not just the score. The specific policy citation or news item is the evidence; the aggregate number is only a pointer.
- Pair them with citations and qualitative evidence. No single indicator carries an assessment; altmetrics are one strand among several.
- Interpret in context. Field, output type, and audience all change what a given level of attention means.
- Never use a score as a ranking or a target. Optimising for attention corrupts the signal and invites the gaming the metric is most vulnerable to.
Where shared vocabulary fits
“Impact”, “attention”, “reach”, “engagement”, and “altmetric” are used loosely and often interchangeably, which is exactly how attention data gets mistaken for evidence of benefit. A shared, federated vocabulary that defines these terms precisely — distinguishing attention from impact and pointing back to the Leiden Manifesto and the responsible-assessment frameworks for the caveats — is what lets engagement data be used honestly in evaluation. Supplying that definitional layer is the role the CASRAI dictionary is designed to play; the relevant terms sit in the engagement, impact and SDG-alignment domain.