The term “altmetrics”, coined in 2010,1 refers to all measurements of scientific output not included in conventional bibliometric indicators such as the impact factor (IF). Given the impossibility of closely monitoring the exponential growth of scientific output, altmetrics quantify the use of the Internet and social networks. Since this concept was introduced, it has captured the attention of higher impact scientific journals, which report altmetrics alongside their usual indicators. Since altmetrics do not measure the quality of a publication but concentrate rather on the attention paid to an article in the media,2 they can access, filter, and highlight articles that generate interest from the very moment they are published at a staggering rate of approximately 2.5 million articles per year — a figure that is doubling every 9 years, and is humanly impossible to follow.3

Archivos of Bronconeumologia (AB), the official organ of the Spanish Society of Pulmonology and Thoracic Surgery (SEPAR) and the Latin American Chest Association (ALAT), is the Spanish language respiratory journal with greatest impact worldwide, with an IF 4.214 that ranks it in the first quartile according to the 2018 edition of the Journal of Citation Reports (JCR). In line with other respiratory journals with an impact factor, such as the American Thoracic Society Journals,4 AB has decided to record its altmetrics, which are quantified using specific programs known as aggregators. The aggregator used by AB is PlumX Metrics, created in 2011, but another important aggregator, created in 2012, is Altmetric. The main objective of this study was to conduct an analysis of altmetrics using the Altmetric and PlumX Metrics aggregators, and of AB website traffic during a 5-year period.

We included 273 documents between the years 2014–2018: 186 (68.1%) original articles, 41 (15%) reviews, 26 (9.5%) special articles, and 20 (7.3%) guidelines. Thus, review articles (guidelines, reviews, and special articles), with 87 articles, accounted for 32% of the total. COPD was the area that generated most documents, with 52 articles (19%), followed by tuberculosis and respiratory infections with 39 (14.3%), and sleep-disordered breathing/mechanical ventilation/critical respiratory care with 36 (13.2%). Table 1 shows the contribution of the remaining areas, by document type. Pulmonology was the specialty of the first author of 175 (64.1%) papers, and specialists in 22 different Spanish Health System areas were listed as the first author of the remaining articles. Overall, 70 (25.6%) papers were funded, and more than one institution participated in 226 (82.8%). Median time since publication of all papers was 50.9 months (interquartile range 47.2–54.5), and there were no significant differences in any of the 5 comparisons studied.

The documents whose first author was a pulmonologist received more media attention in most of the PlumX Metrics and Altmetric categories and, in PlumX Metrics, multicenter publications received more attention with more captures (16.4 ± 25.8) and more views in social networks (24.4 ± 62.1) (Table 2). Table 3 shows the PlumX Metrics media impact by SEPAR area, where smoking received the most mentions (1.5 ± 3.7) and physical therapy the most usage (234 ± 159.5) (Fig. 2). Table 4 shows that the relative rate of citation in Altmetric was greater for the area of COPD (2.2 ± 5.8).

Fig. 2.

Distribution of usage by paper according to SEPAR areas in the PlumX Metrics platform. COPD: chronic obstructive pulmonary disease; DILD: diffuse interstitial lung disease; OELD: occupational and environmental lung diseases; SEPAR: Spanish Society of Pulmonology and Thoracic Surgery; SDB: sleep disordered breathing; TRI: tuberculosis and respiratory infections.

(0.13MB).

With respect to document type, guidelines received more media attention in almost all categories analyzed (Table 5). With respect to the website, publications with a pulmonologist as first author, without funding, and reviews or guidelines received more views (Table 6). Single-center or multicenter participation did not influence the amount of traffic, and no differences were found in terms of SEPAR area, regardless of the language of the article viewed (Appendix B Table 2, additional material).

Mean AAS for all publications was 1.9 ± 4.4 (range 0–59). A total of 77 publications (28.2%) had a score of 0 and the most repeated value was 1, documented in 97 (35.5%) papers. The SEPAR area with the highest AAS was COPD with 3.6 ± 8.8, and the lowest AAS was in the area of respiratory physical therapy with 0.6 ± 0.5, with no statistically significant differences between all areas (P = 0.74). In terms of specialty, pulmonology papers showed a more statistically significant AAS of 2.2 ± 5.3 compared to other specialties which had 1.3 ± 2.1 (P = 0.04). Funded publications had an AAS of 1.7 ± 3.0, and those without funding had 1.9 ± 4.9 (P = 0.59). Multicenter studies presented a score of 2.1 ± 4.8 in comparison to single-center studies with 1.2 ± 1.1 (P = 0.71). Finally, by document type, guidelines achieved the highest AAS with 5.1 ± 12.9 and original articles scored 1.3 ± 2.1 (P = 0.002).

Subanalysis of SCI-E citations

A moderate statistically significant correlation between SCI-E citations and AB website traffic was determined, with a Spearman’s correlation coefficient of 0.598 (95% CI: 0.497–0.678; P = 0.01) (Fig. 3). In the remaining domains, a very weak significant correlation was observed with AAS (ρ = 0.157; 95% CI: 0.005–0.306; P = 0.036), a weak correlation with PlumX Metrics captures (ρ = 0.239; 95% CI: 0.118–0.358; P = 0.0001), moderate with PlumX Metrics usage (ρ = 0.420; 95% IC: 0.304–0.533; P = 0.0001), and very strong with PlumX Metrics citations (ρ = 0.832; 95% CI 0.778–0.874; P = 0.0001). Social networks in PlumX Metrics were not significantly correlated (ρ = −0.015; 95% CI: −0.157 to 0.123; P = 0.815).

Fig. 3.

Correlation of citations obtained by the Science Citation Index Expanded with Archivos de Bronconeumologia website traffic expressed by logarithmic transformation.

(0.07MB).

The most significant finding of our study was that the documented AAS was above average for the date of the publication, although approximately a quarter of publications did not generate media impact. It was also interesting to note that, in general terms, publications with a pulmonologist as first author, and guidelines, reviews, and special articles, received more media attention in the aggregators analyzed and according to AB website traffic.

As these types of articles are more numerous, they could be more likely to obtain media impact and conventional impact. Pulmonology is the specialty that has historically been most published in AB, accounting for more than 60% of all papers.6 Although time since publication is a highly important variable in bibliometric studies for estimating the true impact of an article in the scientific community and in a given discipline,7,8 we did not find any significant differences in our study, possibly due to the relatively short follow-up period. Like previous authors, we found in our study that review and consensus articles attracted more media attention than original articles, although they do not contain original content and are possibly over-representated in this type of analysis.9,10 Articles of this type accounted for 32% of the papers included in our study, a percentage similar to that of other recent bibliometric analyses of the historically most cited articles in biomedical disciplines and topics, such as dermatology (25%),11 oral leukoplakia (37%),12 and stroke (36%).13

Multicenter studies achieved higher media attention in the usage and social network domains of PlumX Metrics and in Facebook usage in Altmetric. In an analysis of the 101 neurointervention articles with the greatest media impact, papers in which more disciplines and centers participated had higher altmetric scores.14 Although multicenter does not necessarily mean multidisciplinary, the greater impact observed might be explained by the involvement of more centers with their own channels of dissemination. Finally, the COPD area showed a higher relative citation rate in Altmetric. That said, it should be noted that the 2017 update of the GesEPOC guidelines15 is probably responsible for the AAS impact of 59, nearly three times that of the next highest AAS score of 21 for asthma.

Another important finding was that altmetrics behave differently from conventional metrics. With the exception of PlumX Metrics citations, the correlation with other domains ranged from very weak to moderate. The correlation documented for the 200 most cited articles on emergency medicine (0.31)16 was also weak, and moderate for the 100 most cited articles in nursing research (0.438).17 This may mean that these articles are consulted, but not published, so citations are reduced. Citations documented by PlumX Metrics warrant special attention because, as expected, they showed a very strong correlation with conventional metrics, as they pursue common objectives, but PlumX Metrics include non-IF journals, for which reason the correlation is lower than the recorded figure of 0.832. The fact that these metrics did not correlate closer with conventional metrics does not mean that they do not have sufficient impact. For a start, IF is not the perfect bibliometric indicator, and moreover, we might still not know how to analyze the true effect of altmetrics among the scientific community and the general public. In any case, the growth of almetrics in recent years has been exponential and their potential is almost unlimited, so care must be taken to avoid giving them too much weight to the detriment of quality in scientific production. They should instead be agents of change and responsible scientific communication.

Similarly to a JCR, Altmetric published the list of 100 articles with the greatest media impact in 2018. The spectrum is highly diverse, ranging from topical issues, such as climate change or gender differences, to the fact that the wine glasses in England have doubled in size over the past 300 years,18 highlighting the complexity of both scientific production and consumption. Indeed, one of these publications states that scientists are interested in disseminating the results of their research through Twitter, but that beyond a threshold of 1000 followers, the recipients of the information do not necessarily have extensive scientific training. Thus, reaching such a wide audience can on the one hand be exciting, but at the same time a matter of concern.19 Researchers are therefore responsible for the social impact of the results of their publications and the exponential dissemination offered by social networks,20 and must aim to combine the social impact of their results with the publication of qualitatively and quantitatively accurate evidence in their dissemination and communication strategies.21

The potential of immediate disclosure via media and social networks is undeniable, and previous studies have explored the sustainability of the process in which consultation was not in fact the first option.22 When talking about numbers, whether website traffic or general consultations using search engines, the difference with conventional citations, where growth is more sluggish, is enormous.23 It is clear that the higher impact journals are paying increasing attention to these indicators and dedicate considerable space to them in their websites, but platforms such as Open Access (OA) journals may generate a greater number of consultations or higher readership, although in the case of OA, there is no guarantee that the highest quality scientific content is being accessed. It has been suggested that OA journals (AB is not OA) have a greater media impact for the obvious reason that they are available to any Internet user, even those who are not health professionals.16,17 Be that as it may, in this age of fake news and post-truth, to which medical science is unfortunately not immune, we scientists are obliged to compare and critically evaluate the enormous amount of information to which we are exposed.24

It has been suggested that altmetrics should not be used because they are more easily manipulated than conventional metrics: they can be swayed by the strategies used by journals to publicize their articles on social networks,25 and they are not exempt from the issues that plague conventional citations, such as self-citation.26 Between 2014 and 2016, AB published 1 tweet for each article, and these were counted by the aggregators used in this study, but now it publishes 1 tweet per number, instead of per article. We do not believe or suppose this to be common practice among scientific journals, although a recent study of altmetrics in microbiology found that more than half of the Twitter accounts disseminating publications in the field were “bots”, software that automatically performs repetitive tasks via the Internet.27 As mentioned above, human beings are complex creatures, not only as producers and consumers of science, but also, in the case of social networks, as disseminators. A recent logarithmic analysis of Twitter found that only 41% of recommended publications are effectively read, or to put it another way, 6 of every 10 suggestions we make are not even read,28 with the result that headline-grabbing titles achieve a high media impact, even though the verification of their content might be limited.

In our study we found that consensus articles and those published by pulmonologists as the first author had the greatest media impact. We cannot conclude that one aggregator is superior to another because their differences make them complementary. We do not believe that these altmetrics are superior to conventional metrics, although they will probably become synchronous methods for assessing the impact of scientific publications on a large and small scale. These aggregators are important because they offer the opportunity of determining the desires and preferences of Internet and social network users, which includes nearly all of us in the 21st century. Archivos de Bronconeumologia is not immune to this trend, and we must monitor closely this interesting direction in scientific production which has revolutionized the conventional approach to publications.

Daniel López-Padilla participated in data collection and is responsible for the integrity and accuracy of the analysis. He also contributed to the study design, interpretation, analysis and drafting of the final document. Marta Pérez-Gallan contributed to data collection and interpretation, and drafting the final document. Luis Puente Maestú and Gonzalo Segrelles-Calvo contributed to the interpretation and drafting of the final document. Francisco García-Río, Adolfo Alonso-Arroyo and José Ignacio Granda-Orive contributed substantially to the study design, interpretation, analysis, and drafting of the final document.

The authors state that they have no conflict of interests.