Responsible-assessment frameworks (DORA, Leiden Manifesto, Metric Tide, CoARA) converge on the position that no quantitative indicator can replace expert peer review for evaluating the quality of individual research or researchers, but indicators can usefully inform context and pattern-spotting at larger scales (e.g., portfolio analysis). Achieving an appropriate balance means: (1) being explicit about which questions are answered quantitatively versus qualitatively; (2) reporting numbers with uncertainty and context; (3) giving final weight to qualitative judgement in decisions about individuals; and (4) auditing the balance regularly.
References
- Hicks et al., Nature, 2015. Wilsdon et al., HEFCE, 2015. CoARA Agreement, 2022.