Making sense of altmetrics

Citations Over TimeTop 14% of 2016 papers

Abstract

Altmetrics have arisen as a result of the requirement, by employers and funders, to measure individual researchers’ productivity and impact. Before the advent of social media, the impact of a paper was measured solely by the number of times it was cited by others (citation index). Altmetrics (alternative metrics) is a term that first emerged in 2010 to describe a less formal approach to measuring the impact of a paper 1. This editorial describes how assessment of an individual researcher's outputs is evolving and aims to demystify altmetrics. A journal's impact factor is the average number of citations received per paper published in that journal during the preceding 2 years. For example, the Equine Veterinary Journal's (EVJ’s) 2014 impact factor was 2.374 (i.e. on average, papers published in EVJ during 2012 and 2013 received just over two citations each in 2014). Journal impact factors, which were first published in 1961, enable journals to be ranked (e.g. EVJ ranked 9th in the list of 133 journals in the field of veterinary sciences in 2014) and were originally intended to help librarians select which journals were worth purchasing. Researchers also use impact factors to help decide where to submit their research; the higher the impact factor, the more exposure a paper is likely to receive. A lot of value has been placed on the impact factor of the journals in which an individual researcher has published. However, there is dubious correlation between the merits of a given article and the impact factor of the journal. Therefore, alternative metrics that measure impact and quality of individual papers are required. This is emphasised in the San Francisco Declaration on Research Assessment, which states that journal-based metrics should not be used as ‘a surrogate measure of the quality of individual research articles’ 2. The San Francisco Declaration on Research Assessment also propounds considering a broader range of research outputs (including datasets and software) and impact measures. Increasingly, more qualitative indicators of research impact, such as influence on policy and practice, are being taken into consideration, for example in the new system (the Research Excellence Framework) for assessing the quality of research in UK higher education institutions, which was first implemented in 2014. Almetric.com is just one aggregator that collects data on these alternative metrics and is used by Wiley, publishers of the EVJ (Fig 1). The colourful doughnut graph generated by Altmetric.com contains coloured threads each representing a different type of online attention. The number in the centre is the overall altmetric score; this is calculated based on how many times an article is mentioned in social media (e.g. Twitter = blue thread) and mainstream news media (red thread). Clearly, older articles will have had more time to accumulate a higher score; therefore, data are included to allow comparison with articles of a similar age (i.e. published within 6 weeks of publication of the article in question). In addition, the score is placed in context of other articles from the same source. Altmetrics also tracks article views and downloads from online reference managers such as Mendeley and CiteULike, but these do not contribute to the score. Altmetrics attract some criticism, particularly as the score can clearly be manipulated – for example, open access articles receive more tweets than those that only journal subscribers can access. In addition, not only can social media-savvy authors generate online interest in their articles with a bit of effort, it is possible to buy exposure. As a result, whether the extent to which an article is discussed in social media correlates with subsequent citations in primary scientific literature is dubious 3 and the value of altmetrics for estimating impact is controversial. However, these are still early days for altmetrics; increasing engagement with social media by a new generation of researchers is likely to dilute the effect of a few keen individuals promoting the impact of their research. The altmetric score for any published article with a digital object identifier (DOI), Pubmed ID, arXiv ID or Handle can be obtained by installing the Altmetric.com browser ‘bookmarklet’ (http://www.altmetric.com/bookmarklet.php). Altmetrics can provide useful information such as who is engaging with your research, favourably or otherwise; for example, the individual tweets that contribute to the altmetric score can be viewed. Research funders are increasingly demanding evidence that their investment has resulted in impact, and the altmetric score represents a readily obtained indicator. It is likely, therefore, that altmetrics are here to stay (at least for a while) and will increasingly be incorporated as a measure of individual performance alongside traditional citation impact metrics.

Related Papers

• → What do impact factors tell us?(1999)78 cited
• → What Is Societal Impact and Where Do Altmetrics Fit into the Equation?(2019)34 cited
• → Examining the Correlation between Altmetric Score and Citations in the Plastic Surgery Literature(2020)33 cited
• How to make altmetrics useful in societal impact assessments: shifting from citation to interaction approaches(2017)