Introduction
Scientometric indicators, and particularly journal impact factor, are widely misunderstood and often used inappropriately. This reflection serves as the starting point for this paper on both the nature of journal impact factor and on international bibliographic indexes. Beyond doubt, the year 2001 was a landmark in the history of Archivos de Bronconeumología because of our journal's inclusion in the main indexes maintained by the Institute for the Scientific Information (ISI), namely Science Citation Index (SCI) Expanded, ISI Alerting Services and Current Contents/Clinical Medicine.1-3 As a result, our journal will finally have an official impact factor, provided of course that we attain a sufficient number of citations to be considered a source journal by ISI and therefore to be taken into account for impact factor calculation.4
Databases: the Science Citation Index
Bibliographic indexes are the main source of information used in bibliometric studies. Specialized databases are available for all branches of science, and the validity of a bibliometric study will depend on appropriate selection of a database to adequately cover the field being studied.5 Bibliometric analyses of biomedical fields should be performed using specialized medical indexes (MEDLINE or Excerpta Médica) or interdisciplinary ones such as the SCI.6
The SCI, which belongs to the ISI, located in the United States of America, began in the 1960s, although it includes entries dating from 1955. The ISI covered nearly 5200 journals in 31 different languages in the 1970s, although its best known products--the SCI, the Social Science Citation Index and the Arts & Humanities Citation Index--covered 5700 source journals.7 From those 3 indexes and from each one separately come the Journal Citation Report (JCR), with its famous "impact factors." Currently the ISI covers some 8500 journals in 35 languages.7
The importance of citations lies in their use as key indicators of the frequency with which researchers actually use scientific journals. The advantages of the SCI can be summarized as follows:
--It is interdisciplinary, indexing journals in both science and technology, although half the journals are of interest to biomedical researchers. Journals are selected on the basis of scientific quality, formal quality, and scientific recognition.5,6,8
--Full indexing is provided, which is to say all articles in the journal are included.
--All authors on the documents are listed, as are the affiliations for each, including city and country--key information for studies of collaboration.
--No less important is the fact that all references in the documents are entered, allowing citations to be searched.5
--A search can be based on a specific author or an article read, allowing related papers that cite the same authors to be found, and the impact factor can be determined. Thus, an author can find out who has cited an article he or she has written and in which journals citations have appeared.
The SCI is therefore a useful tool for bibliographic searches or alerting services,6 although its limitations and drawbacks are well known. It is important to mention that documents have not been indexed following stable criteria over time and entries can be uneven. Another limitation involves the SCI disciplinary classification system by which a document is indexed according to the journal in which it has been published and it may even be classified simultaneously in several disciplines, such that any comparisons made should be on the same types of documents and within the same subject field. Various types of errors have also been found, some arising from the manner of processing information and others from conceptual problems.7 Authors, when including a reference to another study, often commit citation errors, and such errors are not corrected by journal editors. Moreover, the ISI is inconsistent in the way it counts citations of articles signed by a group. Therefore, we are looking at 2 sources of loss of specific data and it is estimated that Spanish authors may lose between 7% and 20% of their real citations because of the aforementioned errors.7 Errors also occur in the adjudication of affiliations and this is aggravated by problems related to author names.7
Among the limitations that can be attributed to the SCI is that of bias in favor of English language journals, such that non-English language journals are less well represented. Additionally, those who publish in English may be monolinguals, meaning that it will be difficult for them to cite articles in other languages.6 Spanish, nevertheless, is a language whose importance is rising both economically and culturally, and we should remember that not all journals that are published in English manage to achieve adequate international circulation.9 Along the same lines, it is also useful to know that bibliometric studies of the effect of adopting English as the language of publication have shown that conversion does not affect impact factor.10 As the early 20th century Spanish histologist Pío del Río Hortega said, "No basta con publicar en un idioma importante para tener trabajos importantes."* The ISI presently gathers information from over 200 Spanish language journals, but only a very few of them are considered "source" journals for the purposes of computing bibliometric indicators.4,7 Yet another limitation is bias in favor of the basic sciences, which are better represented than applied or clinical sciences5; in this situation, it may be more appropriate to include interpretive research as a vehicle for establishing sufficient flow between basic and clinical sciences.11
Nevertheless, the SCI facilitates bibliographic searching and alerting services far better than the other indexes of scientific literature, with great ease of access and grouping of authors, articles, and journals in function of the topic that interests the researcher.6 Furthermore, it is the only database that indexes citations. The necessary information for calculating indicators is obtained by analyzing the scientific repercussion of the publications in question.12
Bibliometric Indicators: the Impact Factor
Bibliometric analysis seeks to compile and study quantitative data from scientific publications, which is to say, it attempts to quantify scientific activity. The science of bibliometrics studies the nature and evolution of a discipline (provided it expresses itself through publication) by computing and analyzing various facets of written communication.9,13,14 Bibliometrics uses indicators, which are parameters that reflect social aspects of scientific activity related to the production, transmission, consumption and repercussion of information (Table 1).15-17
Indicators of repercussion (Table 2) are constructed by looking at citations, which are the mentions a paper receives in later articles. The most widely used indicators for evaluating repercussion are the visibility index (logarithm of the number of citations received), the influence index (ratio of the number of citations received to the number of citations made), the median number of citations received over the lifetime of a journal, and, of course, the index of impact (ratio of the number of citations received to the number of articles published).13,18,19
Calculating an index of impact required great expense until Eugene Garfield's 1964 founding of the SCI, which brought that indicator to the forefront under the name of impact factor. This factor has been used indiscriminately, at times diminishing the real function of citation, and users have failed to take into account that it is a relative index that should not be used to compare journals. Nevertheless, in spite of the SCI's aforementioned limitations and biases,18 the truth is that its impact factor, with all its limitations, is an objective, quantifiable parameter that offers a relatively consistent way to judge a journal's position on the field of international science.12 I believe that even if the position of the "impact idolaters,"20 is inadmissible, the position of "impactophobes" is too, the middle ground being the most advisable position to occupy.11
The impact factor is the ratio of the number of citations in 1 year of all articles published in the journal in the previous 2 years to the number of citable articles published in those previous 2 years. Consequently, a journal's impact factor refers to its influence on similar research at a moment in time.21 Although we would like to have indicators able to assess the quality of publications, we have to make do with indirect reflections of that quality, such as the impact that papers have on investigators in their field.5 The journals that publish more articles have more chances to be cited, and that is one of the reasons for introducing the impact factor, which normalizes the number of citations in function of a journal's size. An author's citing of another work might either affirm or repudiate it, use its content to underpin a premise, provide additional evidence or serve to make a comparison, or to rule out its interest, or to reflect the relation between the fields of the citing and cited authors.9 Citations to the basic sciences are well known to be more numerous than those to clinical research and epidemiology, with review articles and methodological discussions being cited most.22,23 Lately, clinical and epidemiological papers with dozens of authors and multiple international collaborating centers are becoming highly cited documents.20 It is not easy to know the extent to which a citation is due to the intrinsic quality of the cited work or to other aspects such as author prestige, institution prestige, fashion, phobias, or other tendencies.5,9 We can assume, therefore, that the number of citations a paper receives may not be a measure of its quality but rather an indicator of its visibility, use, diffusion, or impact, although various investigators report a positive correlation between the number of citations received and scientific quality.9 The impact factor should not therefore be considered an infallible measure of an author's scientific quality, nor should it be applied as an unchallenged rating scale, although that does not diminish its importance and little by little it is gaining wider acceptance.12 Nor is the impact factor a constant value; rather, it varies over time as a result not only of intrinsic quality but also, at other times, due to events unrelated to science itself.
Spanish biomedical journals--such as Medicina Clínica and Revista Clínica Española--which were included in the SCI in 1988, have maintained their scores. After inclusion, the impact factor of Medicina Clínica rose through 1993, when it reached 0.909, and it remained stable until 1995.24 Impact factors also increased in other Spanish biomedical areas, increasing 2-fold in some cases.25-28 No information is available on the impact of Archivos de Bronconeumología before 1995, but we do have estimated figures for 199629 (impact factor 0.069 and weighted impact factor 0.014), and those can be compared with the estimated figures for the years between 1997 and 20003,15 using a selection of possible source journals. The estimated impact factors for Archivos de Bronconeumología over that period were 0.107 in 1997, 0.089 in 1998, 0.105 en 1999 and 0.119 in 2000, thus showing acceptable improvement over time at a rate similar to that of other Spanish biomedical journals. It is to be expected that more recent increases in impact factor will be even greater given the importance of the respiratory system in biomedicine.7 The citation pattern has been characterized by a dispersion of source journals, with little weight from JCR publications in respiratory medicine, a clear predominance of citations by Spanish authors, with limited self-citation, and greater impact in the areas of tuberculosis and respiratory infections and chronic obstructive pulmonary disease.3,15
To interpret the estimated impact factor of Archivos de Bronconeumología, we can compare it to that of other respiratory medicine journals or to biomedical journals in general. In the first instance, our journal's impact factor is quite similar to the 1999 impact factor of 0.103 of the JCR-included journal Applied Cardiopulmonary Physiology.30 In the second instance, we can consider impact factors from a 1992 paper by Baños et al31 covering Spanish journals in the SCI-JCR. Four such journals published basic science research (Methods and Findings in Experimental and Clinical Pharmacology, Histology and Histopathology, Inmunología and Revista Española de Fisiología) and 5 covered areas of clinical research (Revista Española de las Enfermedades de Aparato Digestivo, Allergologia et Inmunopathologia, Medicina Clínica, Revista Clínica Española and Nefrología). Impact factors were found to be higher for basic science journals. In respiratory medicine, impact factors have risen in recent years even though the 1999 JCR listed only 30 journals in this category. The scarce representation of this area is aggravated by the fact that this category includes publications that are more related to cardiology than to pulmonology or chest surgery. It is striking that up to 15% of articles published by Spanish authors in international respiratory system journals between 1987 and 1998 were signed by cardiologists or chest surgeons.32 It must be emphasized when looking at Spanish productivity in biomedical publication in the 1994-2000 period, that the respiratory system is currently among the clinical disciplines with over 1000 citable documents in Spain (1476 to be exact), according to a superb recent analysis.7 This confirms that our disciplinary productivity is tending to increase, placing us now among the top 15 (in eighth place). Catalonia, with 41% of the citable documents and 71.6% of the citations, is the most productive Spanish autonomous community for respiratory system research; the community of Madrid is not outstandingly productive, although productivity there has clearly increased.7,15,17,34
Most citations of Archivos de Bronconeumología are to be found in Spanish journals of internal or general medicine. Furthermore, a considerable number of international journals unrelated to respiratory medicine cite our journal, another observation that underlines the limitation of the JCR's respiratory system classification.
Impact, Relevance, and Quality
Describing the scientific contribution of research requires us to distinguish between quality, importance or relevance, and impact (Table 3).20 Quality indicates how well-received a study is (originality, methodological adequacy, design, etc.); relevance refers to the potential influence of the research; and impact reflects its repercussion.20 Thus, a proper assessment of research requires an appropriate simultaneous combination of these approaches to evaluation.
Impact factor is one of the first measures to address when raising the quality of a journal, for as mentioned above, there is a positive correlation between the number of citations received and scientific quality. The fact of publishing in a journal with impact itself indicates quality, as does publishing in a journal indexed in international databases. Quality reflects excellence to the extent that impact reflects actual influence on a wider scientific field. Other features that reflect a journal's quality are those based on perceptions, such as expert reviews or peer reviews, which are subjective but considered the foundation for valid scientific publication, an indicator of a scientific journal's quality.35,36 An indirect aspect of quality that should be interpreted cautiously is the fact that, in general, there is a high correlation between scientific eminence (author notoriety) and productivity (although in general there is no correlation at all between the most productive and the most cited37), following Lotka's law.9 Yet another indirect aspect of scientific quality is the presence of a larger number of coauthors (up to a reasonable limit). This is a positive phenomenon because it indicates that more writers want to communicate the results, probably increasing intellectual excellence, and shows that groups of collaborators are being formed to work on multidisciplinary teams; surely the scientific quality of the final product can be assumed to be greater.9,18,38 Likewise, it is useful to point out that the self-citation of Spanish authors and journals not only increases impact factor but also increases Spain's insularity index. Other indices of scientific quality are a journal's compliance with national and international formal guidelines (formal quality),39 journal production quality, and stability or regularity of periodic publication, and, of course, time of uninterrupted publication. An important consideration for a medium of diffusion of scientific knowledge, and a feature that is quickly growing more important, is simultaneous (parallel) paper and electronic publication. Yet another measure of biomedical journal quality is the increasing tendency to apply ever more complex statistical analyses, particularly in certain fields; along these lines, Archivos de Bronconeumología has been progressing satisfactorily, although we lag somewhat behind other national and international journals.41
As mentioned, scientific quality improves considerably if certain qualitative criteria are applied. This can be seen in the great strides in quantity and quality made by prestige journals when they adopt peer review for scientific research and publication.7 However, research evaluation should also cover the planning phase, the training of researchers, or the structuring of institutions.42 In the effort to improve research policies, knowledge of scientific results and their quality is of extraordinary value and should take place in the context of other policy making related to health care, the environment, industry, or education. To that end, a thorough understanding of the real impact on society of results reported by scientists is essential. The specifically health-related issues that we should study are the improvement in the delivery of care in our hospitals and in the system at large, the efficacy of treatments, and better public health management, without forgetting basic research on new therapies, drugs, or surgical procedures.42 Achieving those goals requires us to remember that the interests of researchers do not necessarily coincide with those of society, that determining social or economic impact for health care research should be based on objective indicators of impact with additional information regarding quality. Quantitative evaluation will be called for, and experts must be trained to carry it out.42
Impact Factor: Constraints on Its Use
Although in 1927 Gross and Gross43 had already pointed out the importance of counting the citations an article receives as a measure of its scientific usefulness, their importance as bibliometric indicators was not revealed until the work of Garfield44 appeared to justify establishing comprehensive indexes of citations classified in alphabetical order as a bibliographic tool capable of bringing together those who search for knowledge and those who publish research. Accordingly, proper indexing of citations requires an alphabetically ordered coding system to facilitate, when necessary, a list of original research papers that cite an article in question. Working from the opposite direction, an author can obtain a list of articles that have cited his or her own paper as well as discover how any paper has been received by the scientific community. The first use is key because it allows the importance and impact of an article in the scientific community to be evaluated independently of the size of a journal, the resulting "impact factor" being more useful and indicative of importance than a simple count of publications.44 This is how, as I have previously indicated, the SCI was founded in 1963 by the ISI in Philadelphia. Later, Garfield himself explained what impact means, stating that a clinician or biomedical investigator's citation of a paper indicates the influence it has exercised on him or her, and that therefore the more a work is cited the more influence or impact it will have had on the scientific community.45 The reason for creating an impact factor, according to Garfield, was to assure that the most significant publications, citations, and journals were indexed.46 While monitoring impact factor over a long--or short--period of time and being able to make it public was not a priority when the ISI was founded, the data were nevertheless made available to investigators.46 Although the system cannot be compared with peer review, it nevertheless bears witness to the importance of a life sciences journal. Undoubtedly, nearly all SCI-indexed articles in high-impact journals are cited more than once, which is to say, citation correlates positively with impact.47 Nevertheless, it is inappropriate to compare journals; perhaps such comparison is only valid, although never perfect, within a specialty or subspecialty given that few publications have exactly the same orientation and mix of original and review articles.47 Therefore, since the advent of the impact factor, it has taken on a weighty role in decision making among scientists, from authors to members of Nobel Prize juries to foundation officers adjudicating grants.
When the impact factor was created, no one ever thought it would become the object of widespread controversy.48 At first it was expected to be used constructively, although it was recognized that it might be abused in the wrong hands. Besides impact factor, citation density (mean number of references per article) and the median number of citations over the lifetime of a journal have also become important variables, and an impact factor will not provide sufficient information in specialties that vary little over time and in which the citation half-life is long.8 Another relevant consideration is the fact that prestigious journals publish articles that are neither reports of research nor reviews, such as letters to the editor, editorials, and so on. Such documents do not enter into the calculation of the JCR impact factor, even though they are known to receive citations and are contained in the numerator when an impact factor is calculated but not in the denominator, a fact that favors high-impact journals given that if their denominators are smaller than the real number of total documents their impact factors will be higher.20,48 For this reason, many editors have argued that the numerator in impact factor calculations is more relevant than the denominator and is itself a good indicator of a journal's relevance and cost-effectiveness.49 Therefore, and given that articles are coded for type based on simple human judgment, it might be wiser to define the nature of a substantive article differently or to make an effort to identify and distinguish clinical or laboratory studies or practice-based studies from investigative studies. A mistaken notion is that the size of the scientific community a journal serves affects its impact factor, which is to say, the more researchers a field has the higher the impact factor will be. This assertion does not take into account the fact that the more authors and articles there are to be cited, the more there will be to share those citations. Therefore, the key is not in the number of authors and articles in a specialty, but rather in the number of citations and their duration.48 This can also be linked to the time taken in reviewing and accepting manuscripts, given that if review time is long, various citations that would have affected the impact factor may have been lost, and therefore the impact will be lower.48
Although the drawbacks and shortcomings of impact factor use are known,50 and it must be admitted that it is an imperfect tool for measuring the quality of articles, no better method is currently available and the impact factor has the advantage of having been studied for some time and being an appropriate approach to scientific evaluation.48 The impact factor is valid, therefore, for determining the quality of scientific journals in itself, but should not be used to compare articles, scientists, or groups of scientists (Table 4).51
How Can the Impact Factor of Archivos de Bronconeumología Be Improved?
At this point we can ask how we can improve our journal's impact factor. Several aspects have to be taken into account. Recently, it has been pointed out that exhaustive and scrupulous peer review would certainly increase a journal's impact factor,52 although quite likely it would reduce its content by over 30%. The usefulness and relevance of using international as well as national peer reviewers for a journal have also been discussed, given that when a referee is of the same nationality, the article tends to be assessed too highly. Analyzing all the possible combinations of the preceding suggestions, it has been observed that the differences are not significant between groups when a large number of manuscripts are studied, but that the differences can be considerable in some countries.52 I have already mentioned that our journal's impact factor can be raised by publishing reviews and guidelines, insisting that Spanish-speaking authors should send their best articles to Spanish-language journals, including a larger number of references per article, recommending that authors include self-citations in their research papers. However, whether or not it is acceptable to "play the numbers game" is controversial,53 given that it would be an artificial strategy. This issue, which should not turn into a quarrelsome ethical exercise, was raised some years ago in Spain,54 and it was admitted that attracting articles from Spanish-speaking authors to Spanish-language journals could produce the opposite effect, given that the citations that our medical journals receive abroad are to be found in articles by such authors published there. If we attract those authors back to our journals, our impact factors may decrease. Encouraging increased self-citation is also not easy, given that peer reviewers and editors abroad are reluctant to accept references in other languages, both because the reviewers have difficulty in verifying their accuracy and the agreement between reference and text and because the editors wish their readers to have easy access to cited articles.55 An important exercise for Spanish-speaking authors who send their manuscripts to international journals would be to review the content of Archivos de Bronconeumología looking for possible papers to cite.
It has already been pointed out that a paper is more likely to be cited if it is published in English.6,9,10,12,20,50,53,55,56 The only part of an article published in a non-English language journal that is available whole and in English in the main databases is the abstract. Because potential citations arise from the abstract's inclusion in such indexes, the correct structuring, completeness and writing of that document is essential.55
I must also insist on the importance of maintaining a journal's formal quality and editorial policy over time, as discussed elsewhere.39 A journal's adherence to international publication standards and ease of access to its scientific content are signs of its status as a privileged vehicle for channeling the results of scientific research and evaluating such aspects has been called for from various perspectives--from social, political, and economic as well as scientific ones. Thus, lag times between acceptance and publication play a role, in case of conflict, by establishing priority in publishing discoveries and they reveal the immediacy of a study. Lag times therefore are a measure of the editorial agility that is so important for a journal that aspires to be at the forefront of respiratory system research, and the application and publication of editorial process dates show a journal's level of compliance with international standards for the formal presentation of periodicals, providing an indication of editorial quality.39,55 The importance of editorial agility must be mentioned, but if a journal uses the peer review system it will be difficult to shorten times until publication. Garfield48 recognized the fact that 2 articles about the same topic in the same issue of a journal have positive effects on impact factor. This is difficult to achieve, however, given the diversity of respiratory system topics, which will even make it difficult to cite articles from the same journal in the same year. Because the immediacy index is important, it can be sustained by the references to editorials that accompany one of the original or other featured papers in each issue.48,55 It is also licit to stir up controversy, discussion, and commentary by way of letters to the editor whose period of acceptance is known to be much shorter. Such letters improve the immediacy index and their citations become part of the numerator when calculating the impact factor, thereby contributing to its improvement indirectly.48,55
It will always be necessary to insist that authors should guarantee that references be impeccably written. We should also take special care when writing our own names and addresses because Spanish-surnamed authors can lose up to 20% of their data in the SCI due to errors of transcription of personal data and references, and such errors will logically lower our impact factor in the end.
Aspects that should be taken into account to improve a journal's impact factor were discussed by Garfield in an article that has become a classic.57 Thus, the publication of review articles generally increases impact factor, although reviews must be chosen for their quality and be up to date. Articles that stir up controversy are also recommended, as are papers on methodology, as all of these receive more than the average number of citations and, as a result, increase impact factor. Another option a journal editor can consider is to select authors by analyzing their prior scientific and citation history, a strategy that would undoubtedly increase impact. Additionally, the number of citations can be increased by including articles that arise from work done at several research centers, both local and, better yet, international ones, and I would add that studies should be interdisciplinary. We should not forget that sometimes achieving a greater number of citations depends on the specialty in question, given that some specialties by their very nature require more time to produce articles with impact. Yet another point to insist on is the need for abstracts to be impeccably written and for errors in author affiliations and personal data to be as few as possible. Finally, authors should cite all relevant works on a topic in order for a journal's impact factor to be raised.57
Currently, technological developments have made it imperative for a journal wanting to take a position of leadership to have a web page where tables of contents, abstracts, or even full texts are available. Many journals are now published in both print and electronic media (parallel publication) and a considerable number are issued only on the web. The speed and immediacy of electronic publishing is rapidly becoming an important way to make scientific knowledge available. A scientist interested in a subject can access electronic texts faster than paper texts, favoring citation of electronic articles and, consequently, such availability can increase impact factor.
Many authors have recently inquired into the way impact factors are calculated and have proposed changes or have even suggested eliminating them.58-60 Fassoulaki et al58 proposed adjusting an impact factor by an index of self-citation but they pointed out that the position of the journals they analyzed would not change substantially. Bloch et al,59 on the other hand, proposed abandoning the impact factor altogether and returning to the fundamental bases of evaluation, publishing only a small list of articles that have been identified after a highly selective peer review in which previously specified merits are assessed; in this manner the role of peer review would be crucial for determining the quality of an article but, as we have seen previously, this would increase the impact factor even if that is not the intention.52 In a letter to Nature, Brunstein60 mentioned the possibility that the growth of the Internet and of on-line journals may spell the end of the impact factor or perhaps lead to its redefinition. Garfield61 replied that he doubted that the Internet would be the end of print publication and that in any case, even if it were so, a new type of impact factor would be invented. He asserted that provided references exist, an impact factor can be calculated, since on-line journal citation practices would be standardized sufficiently to allow such calculation.
When Callaham et al62 analyzed the features of scientific articles that are associated with greater citation, they observed that journal impact factor was the variable that determined frequency. Therefore, it is more important than any other analyzed variables they considered, among them such relevant features as the creation or not of a control group, randomization, double blinding, prospective or retrospective design, or explicit statement of a hypothesis, or achievement on a scale of merits and qualities for scientific articles. This leads authors to believe that the journal in which an article is published is as important as the traditional measures of quality.
As a result, until other tools for evaluating scientific quality are found, the impact factor, even with its imperfections, continues to be a good means of evaluation. Experience demonstrates that the best journals are those where it is harder to have an article accepted and those are the journals with higher impact factors. As has happened for other Spanish journals, the impact factor of Archivos de Bronconeumología will increase over the previously estimated one because of the journal's inclusion in the SCI in 2001, and toward the end of 2004 the journal's impact factor will be officially published in the JCR.
*Publishing in an important language is not enough for saying one has published important papers.
Correspondence: Dr J.I. de Granda Orive.
Servicio de Neumología. Hospital Militar Central de la Defensa.
Glorieta del Ejército, s/n. 28047 Madrid. Spain.
E-mail: igo01m@saludalia.com.
Manuscript received March 25, 2003.
Accepted for publication April 1, 2003.
