CASRAI Dictionary

Tag: SciVal bibliometrics

  • SciVal Bibliometrics vs the Leiden Ranking: Benchmarking Under DORA

    SciVal is Elsevier’s Scopus-based platform for benchmarking research output; the CWTS Leiden Ranking is Leiden University’s field-normalised ranking that deliberately avoids one composite score. Institutions increasingly run both together, but DORA warns that any league-table framing can reduce research quality to a single misleading number.

    SciVal bibliometrics refers to the citation and output metrics — including Field-Weighted Citation Impact (FWCI) — that Elsevier’s SciVal platform generates from Scopus data to support institutional research evaluation. Research offices now routinely pair this proprietary layer with the CWTS Leiden Ranking’s open, transparent indicators, creating a benchmarking workflow that sits in direct tension with the San Francisco Declaration on Research Assessment (DORA).

    What is SciVal and what does it measure?

    SciVal is Elsevier’s research-analytics platform, built on Scopus abstract-and-citation data, that lets subscriber institutions benchmark output, impact, and collaboration against named peer groups. It does not produce publicly indexed rankings; access is by institutional subscription, and outputs are configured per user for internal decision-making rather than public comparison.

    Core SciVal modules include:

    • Overview — publication and citation summaries for an entity over time
    • Benchmarking — side-by-side comparison against selected competitor or aspirational institutions
    • Collaboration — network maps of co-authorship at institutional, national, and international level
    • Trends — topic-level growth signals used for strategic investment decisions

    Its signature indicator is Field-Weighted Citation Impact (FWCI), the ratio of citations a set of publications actually received to the citations expected for publications of the same type, year, and subject field. A FWCI of 1.0 represents the world average for that field; values above 1.0 indicate above-average citation impact.

    How does the CWTS Leiden Ranking differ from SciVal?

    The CWTS Leiden Ranking, produced annually since 2007 by the Centre for Science and Technology Studies at Leiden University, is a free, publicly available ranking that explicitly refuses to combine indicators into one overall score. Instead it publishes separate, field-normalised tables — including MNCS (mean normalised citation score) and PP(top 10%), the proportion of an institution’s output among the world’s most-cited 10% of papers in its field.

    Where SciVal is a private diagnostic tool tuned to whatever comparator group an institution chooses, the Leiden Ranking is a public, methodologically documented instrument built for cross-institutional transparency. The distinction matters for governance: SciVal data informs internal strategy conversations, while Leiden Ranking data is citable externally by journalists, funders, and prospective students.

    Dimension SciVal CWTS Leiden Ranking
    Underlying data source Scopus Web of Science (Classic edition) or OpenAlex (Open Edition)
    Access model Institutional subscription Free and publicly browsable
    Composite score Configurable dashboards, no single mandated score Explicitly none — indicators kept separate by design
    Level of analysis Author, department, institution, custom groups Institution-level only
    Signature indicator Field-Weighted Citation Impact (FWCI) MNCS and PP(top 10%)
    Governing body Elsevier (commercial) CWTS, Leiden University (academic)

    Why does DORA caution against benchmarking with league tables?

    DORA, the San Francisco Declaration on Research Assessment published in 2012, calls on institutions to stop using journal- and rank-based proxies as substitutes for assessing the actual content of research. Its core recommendation is definitive: evaluators must not treat a journal impact factor, or by extension a university’s league-table position, as a surrogate measure of the quality of an individual researcher’s contribution.

    The UK’s Research Excellence Framework reinforces the same principle domestically — REF guidance instructs assessment panels not to rely on journal impact factors or bibliometric rankings when judging individual outputs. A single Leiden Ranking position or SciVal FWCI score, DORA argues, compresses genuinely multidimensional research performance into one figure that is easy to misuse in hiring, promotion, and funding decisions.

    How are research offices combining SciVal and Leiden in practice?

    A DORA-conscious workflow uses SciVal for granular internal diagnostics and the Leiden Ranking for transparent, external context — never letting either stand alone as a judgement on individual quality. In practice this looks like a two-stage process rather than a single dashboard export.

    1. Research offices first use SciVal to identify departmental strengths, emerging topics, and collaboration gaps against a self-selected comparator set.
    2. They then check institutional standing against the Leiden Ranking’s published, field-normalised indicators to see how that internal picture holds up against an independently governed, public dataset.
    3. Neither output is applied directly to an individual researcher’s promotion or tenure case, consistent with DORA’s requirement that assessment be based on the substance of the work.

    This “basket of metrics” approach — pairing a proprietary analytics tool with an open, non-composite ranking — is increasingly the model that DORA-signatory universities describe in their own research-assessment policies.

    What does the OpenAlex-based Leiden Ranking Open Edition change?

    Since 2023, CWTS has published a Leiden Ranking Open Edition built entirely on OpenAlex data, run alongside the long-standing Web of Science-based Classic edition. OpenAlex, launched by OurResearch in 2022 as a free successor to the discontinued Microsoft Academic Graph, indexes a broader and more open set of scholarly outputs than either Scopus or Web of Science.

    Because the Open Edition and Classic edition draw on different underlying databases, the same institution can show a materially different position depending on which edition is consulted — a fact rarely mentioned in library guidance on SciVal or Leiden alone. This is itself a practical argument for DORA’s caution: even among ostensibly objective, field-normalised rankings, the choice of data source alone can shift an institution’s apparent standing, before any interpretive judgement is applied.

    Common questions about SciVal bibliometrics

    Is SciVal the same as Scopus?

    No. Scopus is Elsevier’s underlying abstract-and-citation database; SciVal is a separate analytics layer built on top of Scopus data. Scopus supplies the raw publication and citation records, while SciVal turns them into benchmarking dashboards, Field-Weighted Citation Impact scores, collaboration maps, and trend reports for institutions and funders.

    What is SciVal used for?

    Research offices use SciVal to benchmark departments against named peers, track Field-Weighted Citation Impact and output trends, identify emerging research strengths, map collaboration networks, and build evidence for grant applications — functions distinct from external, public rankings such as the Leiden Ranking.

    What are the limitations of SciVal?

    SciVal’s field-normalisation depends on how Scopus classifies each publication’s subject field, which can misclassify interdisciplinary work. Coverage is limited to Scopus-indexed output, under-representing books and some social-science and humanities journals — a gap DORA cites when warning against treating any single metric as definitive.

    What metrics does SciVal provide?

    Core SciVal indicators include Scholarly Output, Citation Count, Field-Weighted Citation Impact (world average equals 1.0), Outputs in Top Citation Percentiles, and Collaboration metrics. These sit alongside Leiden-style indicators such as MNCS and PP(top 10%) used for external, field-normalised comparison.

    What this means for research administrators

    For research administration teams, the practical guidance is to treat SciVal and the Leiden Ranking as complementary diagnostic inputs, not verdicts. Any institutional report that cites either should disclose the comparator group, data source (Scopus, Web of Science, or OpenAlex), and the field-normalisation method applied, so that governance committees can judge the figures in context rather than as a rank alone.

    Where SciVal or Leiden data feeds into funding, hiring, or strategic planning, DORA-aligned institutions pair the quantitative output with qualitative peer assessment — a practice increasingly documented in the research-assessment policies of DORA-signatory universities.

    Where institutional benchmarking is heading

    As open bibliographic sources such as OpenAlex mature alongside proprietary platforms, expect research offices to triangulate across multiple data sources rather than anchor decisions to one dashboard or one ranking position. The direction of travel — visible in the Leiden Ranking’s own move to publish a parallel OpenAlex edition — is toward more transparent, multi-source benchmarking, precisely the “basket of metrics” model DORA has argued for since 2012.

    Research offices that document their methodology and keep SciVal, Leiden, and open datasets in dialogue with each other will be better placed to withstand scrutiny than those relying on any single proprietary score.

  • Field-Weighted Citation Impact: Where It Fails

    Field-weighted citation impact (FWCI) is a Scopus-derived metric that divides a publication’s actual citation count by the citation count expected for similar documents in the same subject field, publication year and document type — a result of 1.0 marks the global average, above 1.0 marks above-average impact. Before an institution builds review, promotion or tenure (RPT) criteria around it, the underlying normalisation assumptions need scrutiny.

    Field-weighted citation impact is defined by Elsevier as the ratio of citations actually received by an output to the citations that would be expected based on the average for the global scientific output of the same subject field, publication year and document type. It is calculated using Scopus data and surfaced through SciVal and Pure.

    What is field-weighted citation impact?

    Field-weighted citation impact is a normalised, article-level citation metric built into Elsevier’s SciVal and Scopus platforms. It expresses how a specific output, author, or institution has been cited relative to a global benchmark of comparable publications, rather than in raw citation counts that inevitably favour older papers and citation-heavy fields such as biomedicine.

    An FWCI of 1.48 means a document has been cited 48% more than expected for its field, year and type. An FWCI of 0.6 means it has been cited 40% less than expected. Because the benchmark is fixed at 1.0 by construction, roughly half of all outputs in any given field will sit below that line — a distributional fact that is frequently lost in institutional reporting.

    How is FWCI calculated?

    The field-weighted citation impact formula is simple on its face: FWCI = actual citations received ÷ expected citations for similar documents. The “expected” figure is the average citation count for all Scopus-indexed documents sharing the same Scopus subject classification (ASJC code), publication year, and document type (article, review, conference paper, and so on).

    • A microbiology article published in 2023 that has received 20 citations, against a field average of 10 for similar 2023 microbiology articles, scores an FWCI of 2.0.
    • A humanities article with 3 citations against a field average of 2 scores an FWCI of 1.5 — a superficially similar score built on a far smaller, more volatile citation base.
    • SciVal aggregates FWCI across an author’s or institution’s full output set by summing actual citations and expected citations separately, then dividing the totals — not by averaging individual FWCI scores.

    This matters: a single highly cited outlier can lift a whole portfolio’s FWCI, which is why SciVal documentation recommends reading FWCI alongside output volume and citation distribution, not as a standalone score.

    FWCI vs CiteScore and the Journal Impact Factor

    FWCI is often confused with journal-level metrics because all three numbers look similar — a decimal hovering near 1 to 10. They measure different things at different units of analysis, which is the first source of misapplication in policy documents.

    Metric Unit of analysis Field-normalised? Source and window
    Field-weighted citation impact (FWCI) Article, author, or institution Yes — field, year, document type Scopus data via SciVal; typically a rolling multi-year citation count
    CiteScore Journal No Elsevier/Scopus; launched December 2016; citations in a year to the prior 3 years of documents
    Journal Impact Factor (JIF) Journal No Clarivate Journal Citation Reports; historically a 2-year citation window

    Neither CiteScore nor the JIF adjusts for subject field, so comparing a mathematics journal’s CiteScore to an oncology journal’s compares citation cultures, not quality. FWCI’s field normalisation is what DORA-aligned reformers have asked journal metrics to do and mostly do not — which is also why FWCI is sometimes waved through review committees as the “responsible” metric without further scrutiny.

    Where FWCI breaks down: five assumptions to scrutinise

    FWCI’s field normalisation is a genuine improvement over raw citation counts and journal-level proxies, but it inherits several assumptions that DORA-aligned institutions should test before writing it into criteria.

    • Mean-based benchmarking, not percentile-based. FWCI compares an output to the field average, but citation distributions are heavily right-skewed: a small number of highly cited papers pull the mean upward, so most papers structurally score below 1.0 even when performing typically for their field. This is precisely why the Centre for Science and Technology Studies (CWTS) at Leiden University uses percentile-based indicators, such as the share of a unit’s output in the global top 10% most-cited, in its Leiden Ranking methodology rather than a mean-normalised ratio.
    • Subject classification is assigned to journals, not articles. Scopus’s ASJC subject codes are largely applied at the source-title level. An interdisciplinary article published in a broad-scope journal inherits that journal’s field classification, which can misrepresent the “expected” citation benchmark for a genuinely cross-disciplinary piece of work.
    • Small-sample volatility. For low-citation fields (much of the humanities, parts of engineering and mathematics) or for single articles, a difference of one or two citations can swing FWCI dramatically, because the expected-citation denominator is itself small. A score of 2.0 built on 20 citations is far more stable than one built on 2.
    • Self-citation is not excluded by default. Author, institutional, and journal self-citation inflate the numerator unless a self-citation exclusion is explicitly applied — a configurable option in SciVal, but one that is easy to omit when scores are pulled into a spreadsheet for a committee.
    • A single number cannot represent research quality, originality, or societal value. FWCI measures citation uptake within a fixed window; it says nothing about methodological rigour, reproducibility, data sharing, or the qualitative judgement DORA asks assessors to exercise in its place.

    Should FWCI drive review, promotion and tenure decisions?

    The San Francisco Declaration on Research Assessment (DORA), issued in 2012, recommends that institutions not use journal-based metrics as a surrogate for the quality of individual articles, individual researchers’ contributions, or as inputs to hiring, promotion, and funding decisions. FWCI’s article-level, field-normalised design addresses DORA’s specific objection to journal-level proxies such as the JIF — but it does not exempt FWCI from DORA’s broader principle that quantitative indicators should supplement, not replace, expert reading of the work itself.

    Institutions building RPT criteria around FWCI should require committees to read the underlying subject classification applied to a candidate’s outputs, check whether self-citations are excluded, and treat single-digit-citation scores as statistically unstable rather than definitive. A candidate’s FWCI trend across a full portfolio, read alongside narrative evidence of contribution, is a materially more defensible signal than a single score cited in isolation.

    As UK Research and Innovation and equivalent funders continue to align assessment frameworks with responsible-metrics principles, institutions that document how they weight FWCI against qualitative peer judgement — rather than adopting it as a pass/fail threshold — will be better positioned to defend their research administration processes to auditors, funders, and appeals panels alike.

    Frequently asked questions

    What is the average FWCI?

    The global average FWCI is always 1.0 by mathematical construction, because the benchmark for “expected citations” is itself the average of comparable outputs. A score above 1.0 indicates above-average citation performance for that field, year, and document type; a score below 1.0 indicates below-average performance.

    How do I get my field-weighted citation impact?

    FWCI is retrieved through a SciVal subscription, where institutional users can search an author, publication set, or institution and view the FWCI directly on the metrics dashboard. Some institutions also surface FWCI through Pure, which synchronises the metric from Scopus on a scheduled basis where the integration is enabled.

    What is field-weighted citation impact ranking?

    FWCI is not itself a ranking system — it is a ratio, not a percentile or league-table position. Institutions sometimes rank authors, departments, or outputs by their FWCI scores internally, but this practice inherits all the mean-based and small-sample limitations described above and should be treated cautiously.

    Is field-weighted citation impact the same as CiteScore?

    No. FWCI operates at the article, author, or institution level and is field-normalised; CiteScore is a journal-level average citation rate with no field normalisation. A journal’s CiteScore says nothing about how any single article within it actually performed relative to its field.

    FWCI remains one of the more defensible citation metrics precisely because it was built to correct the field-blindness of journal-level indicators. Its value depends entirely on institutions applying it the way its own documentation recommends: alongside output volume, subject classification checks, and self-citation controls — not as a solitary number standing in for expert judgement in a promotion file.