CASRAI Dictionary

Tag: altmetrics

  • Eigenfactor and Altmetrics: Beyond the Impact Factor

    Altmetrics are indicators of the online attention research attracts — mentions, shares, saves and references across the web and social platforms — while the Eigenfactor and its companion Article Influence score weight citations by the standing of the journals that make them. Together they extend evaluation beyond the traditional impact factor, but they measure attention and influence, not the intrinsic quality of any single study.

    Both families of indicator emerged from dissatisfaction with a single citation average. The Eigenfactor refines the citation signal itself; altmetrics capture engagement that citations miss entirely. Neither replaces careful reading, and both invite misinterpretation if treated as scores of merit. They are best thought of as additional lenses on a body of work, each illuminating something a single citation count obscures, rather than as rival verdicts competing to crown a winner.

    The Eigenfactor and Article Influence score

    The Eigenfactor score treats the scholarly literature as a network and ranks journals by the influence of the citations they receive, using an eigenvector method conceptually similar to how web pages are ranked by the importance of the pages linking to them. A citation from a heavily cited, influential journal counts for more than one from a peripheral source. Because the raw Eigenfactor scales with journal size, the Article Influence score normalises it per article, giving a per-paper measure of average influence that is comparable across journals of different sizes. A further refinement is that author self-citations between journals are typically discounted, so a journal cannot inflate its standing simply by citing itself. This network logic is shared with the prestige-weighted journal metrics covered in our guide to CiteScore, SNIP and SJR.

    Why network weighting changes the picture

    Network weighting matters because not all citations are equal. A flat count treats a citation from a marginal, rarely read journal exactly the same as one from a central, heavily cited venue, yet the two clearly carry different evidential weight. The eigenvector approach behind the Eigenfactor and the Article Influence score captures this by letting influence flow through the citation network: a journal cited by influential journals inherits some of that influence, recursively. The effect is to surface journals that are central to the scholarly conversation rather than merely voluminous. It also dampens the impact of citation farming and self-citation, because citations from low-influence sources contribute little. This is the same insight that powers the prestige-weighted journal metrics, and it is one reason network measures are harder to game than raw counts.

    What altmetrics measure

    Altmetrics aggregate diverse online signals: news coverage, policy-document references, social-media mentions, reference-manager saves and blog discussion. Their strengths are speed and breadth — attention accrues within days, long before citations appear, and captures reach into audiences such as practitioners, policymakers and the public that citation counts overlook. A paper influencing clinical guidance or public debate may register strongly in altmetrics while accumulating citations slowly. This timeliness makes altmetrics valuable for spotting emerging work and for evidencing societal reach in ways the slow accrual of citations cannot, particularly for research whose primary audience lies outside academia.

    The risk of gaming and manipulation

    Every metric that carries reward eventually attracts manipulation, and attention-based measures are especially vulnerable. Social-media mentions can be inflated by coordinated promotion, and raw counts can be padded by automated accounts, so a high altmetric score is not by itself evidence of genuine influence. Network-weighted citation measures are more robust, because influence must be conferred by sources that are themselves influential, but they are not immune to citation rings. The practical defence is the same in both cases: never treat a single number as decisive, look at the underlying sources, and combine quantitative signals with expert judgement of the work itself.

    What altmetric signals do and do not capture

    It helps to be precise about which signals carry which meaning. Some altmetric sources hint at scholarly or societal influence; others are pure visibility. The table below sketches the spectrum.

    Signal What it suggests How to read it
    Policy-document citations Uptake into practice or governance Strong societal-impact hint
    Reference-manager saves Scholarly interest from researchers Early engagement signal
    News coverage Public salience Reach, not validity
    Social-media mentions Topical attention Volatile; controversy-prone

    Attention is not impact

    The central caution is that online attention and scholarly impact are different things. A paper can be widely shared because it is controversial, surprising or even flawed; volume of mentions says nothing about validity. Altmetrics are best read as a measure of reach and engagement, complementary to citations rather than a substitute. Conflating the two risks rewarding visibility over rigour, and can even create perverse incentives to court attention rather than do careful work. Authors evidencing the reach of their own work — for example in narrative impact statements — can find guidance in our resources for authors, which encourage describing influence in context rather than leaning on a single attention score.

    Where these metrics complement citation counts

    Used well, the Eigenfactor family and altmetrics fill different gaps left by a simple citation average. The Eigenfactor refines the citation signal itself, distinguishing influential citations from peripheral ones — a logic it shares with the prestige-weighted journal indicators in our guide to CiteScore, SNIP and SJR. Altmetrics, by contrast, capture timely engagement and societal reach that citations record only slowly, if at all. The two are most useful in combination: citations for scholarly influence over time, altmetrics for early and broader attention, neither standing in for a reading of the work.

    Reading these indicators responsibly

    Both the Eigenfactor family and altmetrics should be interpreted within a responsible-assessment framework. The principles of DORA and responsible research assessment, alongside the Leiden Manifesto, stress quantitative indicators as support for — not a replacement of — expert judgement, transparency about what each metric does and does not capture, and avoidance of single-number rankings of people. The longstanding critique of the journal impact factor applies equally here: an indicator’s value depends entirely on using it for the question it can actually answer. Our broader coverage of responsible assessment sets out how these tools fit together.

    Frequently asked questions

    What does the Eigenfactor add over a citation count?

    It weights citations by the influence of the citing journal, so a citation from a highly cited source counts for more, capturing standing within the citation network rather than a flat tally.

    Why normalise to the Article Influence score?

    The raw Eigenfactor grows with journal size. Dividing by the number of articles yields a per-paper average influence that can be compared fairly across large and small journals.

    Do altmetrics show that research is good?

    No. Altmetrics show attention and engagement, not quality. A paper may attract mentions because it is controversial or flawed, so altmetrics complement rather than replace careful evaluation.

    How should these metrics be used responsibly?

    Use them as context alongside expert judgement, be transparent about what each measures, and avoid reducing researchers or papers to a single number — the core of DORA and the Leiden Manifesto.

  • Altmetrics and research impact: what attention data can and cannot show

    Altmetrics promise something seductive: a near-real-time count of the attention a research output is attracting across news, policy documents, social media, blogs, and reference managers, available within days of publication rather than the years a citation count takes to accumulate. That promise is real, and altmetrics genuinely capture forms of reach that citations miss. But the same speed and breadth that make them useful also make them easy to misread, and the gap between “attention” and “impact” is where most of the trouble lies. This article sets out what altmetrics can and cannot show. It builds on the broader treatment in the engagement, impact and SDG-alignment domain.

    What altmetrics actually measure

    Altmetrics — short for alternative metrics — are indicators of the online attention and engagement a research output receives, drawn from sources outside the traditional citation databases. Typical sources include mentions in news outlets and policy documents, posts and shares on social media, blog coverage, Wikipedia citations, and saves in reference managers such as Mendeley. They are usually aggregated against a specific output — identified by its DOI — and presented as a score or a breakdown by source.

    The honest one-line description is this: altmetrics count attention. They tell you that an output was mentioned, shared, saved, or referenced in non-scholarly venues, and roughly where and how much. That is genuinely valuable information, and it is information that citation counts, by their nature, cannot provide.

    What they are useful for

    • Speed. Attention accrues within days, so altmetrics can surface early engagement long before citations could exist. For recent outputs they may be the only quantitative signal available.
    • Breadth beyond academia. A citation count measures uptake by other researchers. Altmetrics can show reach into policy, news media, and public discussion — audiences a citation count is structurally blind to. For an output whose value is partly its public or policy reach, this is exactly the dimension that matters.
    • Qualitative leads, not just numbers. The most useful part of an altmetric record is often not the score but the underlying mentions: which policy document cited the work, which outlet covered it, what the coverage said. Followed up, these point to specific instances of reach that can seed a genuine impact narrative.
    • A complement to citations. Used alongside citation data and qualitative evidence, altmetrics add a view that the other sources lack. Their role is supplementary, not substitutive.

    What they cannot show

    The central caution is simple and must be stated plainly: attention is not impact, and attention is not quality. A high altmetric score means an output was talked about; it says nothing, by itself, about whether the research is sound, whether the attention was positive, or whether any real-world change followed.

    • Attention can be negative. A paper widely shared because it is being criticised, debunked, or ridiculed can score highly. The count does not distinguish praise from condemnation.
    • Attention is not benefit. Genuine research impact — a changed policy, an improved treatment, an adopted practice — is a downstream outcome that an attention count cannot demonstrate. Altmetrics may flag where to look for impact; they are not evidence of it.
    • The numbers are gameable and biased. Social-media-derived metrics can be inflated by coordinated sharing, and they systematically favour topics, languages, and communities that are active online — which is not the same as the topics that matter most.
    • Scores are not comparable across contexts. A single composite altmetric number compresses very different kinds of attention into one figure, and that figure means different things in different fields and for different output types. Comparing scores across disciplines is largely meaningless.

    The responsible-metrics frame

    This is where the wider movement for responsible research assessment provides the discipline that keeps altmetrics honest. The Leiden Manifesto for research metrics (2015) set out principles for the responsible use of quantitative indicators that apply directly here. Three are especially relevant to altmetrics:

    • Quantitative evaluation should support, not supplant, expert qualitative judgment. Altmetrics are an input to a human assessment, never a replacement for reading the work and weighing its contribution.
    • Account for variation by field. Attention patterns differ enormously between disciplines and output types; a metric must be interpreted in context, not applied as a universal yardstick.
    • Avoid misplaced concreteness and false precision. A single score presented to a decimal point invites a confidence the underlying data do not support. The number is an indicator, not a measurement of worth.

    The same spirit runs through the broader reform agenda — the Declaration on Research Assessment (DORA) and the Coalition for Advancing Research Assessment (CoARA) — which presses evaluators away from reliance on any single quantitative proxy and toward judging the substance of contributions. Altmetrics sit comfortably inside that frame as one more contextual signal, and sit very badly outside it as a standalone score to be maximised.

    Treat an altmetric score the way you would treat a smoke alarm: useful for telling you where to look, useless as a measure of how big the fire is. The value is in the mentions it points you to, not in the number itself.

    Using altmetrics well

    1. Read the mentions, not just the score. The specific policy citation or news item is the evidence; the aggregate number is only a pointer.
    2. Pair them with citations and qualitative evidence. No single indicator carries an assessment; altmetrics are one strand among several.
    3. Interpret in context. Field, output type, and audience all change what a given level of attention means.
    4. Never use a score as a ranking or a target. Optimising for attention corrupts the signal and invites the gaming the metric is most vulnerable to.

    Where shared vocabulary fits

    “Impact”, “attention”, “reach”, “engagement”, and “altmetric” are used loosely and often interchangeably, which is exactly how attention data gets mistaken for evidence of benefit. A shared, federated vocabulary that defines these terms precisely — distinguishing attention from impact and pointing back to the Leiden Manifesto and the responsible-assessment frameworks for the caveats — is what lets engagement data be used honestly in evaluation. Supplying that definitional layer is the role the CASRAI dictionary is designed to play; the relevant terms sit in the engagement, impact and SDG-alignment domain.

    Related reading