short commuicatios, articles Simple arithmetic versus ituitive uderstadig: The case of the impact factor Roald Rousseau a,b,c a KHBO (Associatio K.U.Leuve), Idustrial Scieces ad Techology, Oostede, Belgium b K.U.Leuve, Dept. Mathematics, Leuve (Heverlee), Belgium c Atwerp Uiversity, IBW, Atwerp, Belgium E-mail: roald dot rousseau [at] khbo dot be Abstract: We show that as a cosequece of basic properties of elemetary arithmetic joural impact factors show a couterituitive behaviour with respect to addig o-cited articles. Sychroous as well as diachroous joural impact factors are affected. Our fidigs provide a ratioale for ot takig ucitable publicatios ito accout i impact factor calculatios, at least if these items are truly ucitable. Keywords: sychroous ad diachroous impact factors; rakig ivariace with respect to o-cited items Loet Leydesdorff Amsterdam School of ommuicatio Research (ASoR), Uiversity of Amsterdam, The Netherlads E-mail: loet [at] leydesdorff dot et sible that the impact factor of joural J is larger tha the impact factor of joural J ad that addig the same umber of ocited articles to both reverses the mutual order. Although completely atural from a mathematical poit of view, we cosider such behaviour as couterituitive. Ideed, joural J seems more visible tha joural J : how ca the addig o-cited items make joural J more visible tha joural J? Itroductio I this ote we show how simple arithmetic may ifluece our uderstadig of the impact factor. ocretely, it is pos- Joural impact factors We recall the defiitios of the sychroous ad the diachroous joural impact ISSI NEWSLETTER Vol. 7. r.. Iteratioal Society for Scietometrics ad Iformetrics 0
short commuicatios, articles factor. The -year sychroous impact factor of joural J i year Y is defied as (Rousseau, 988): it( Y, Y i) IF ( J, Y ) = = ub( Y i) = it( Y, Y i) ub( Y i) () I this formula the umber of citatios received by joural J (from all members of the pool of sources uder cosideratio) i the year Y, by articles published i the year X, is deoted as IT J (Y,X), where for simplicity we have ot icluded the idex J i equatio (). Similarly, UB(Z) deotes the umber of articles published by this same joural i the year Z. We made it clear i equatio () that the stadard sychroous joural impact factor is a ratio of averages (RoA). Hece we will deote it as RAIF. Whe = 2 oe obtais the classical Garfield (972) joural impact factor. Sice a few years also the 5-year joural impact factor is provided i Thomso Reuters Web of Sciece. The term sychroous refers to the fact that the citatio data used to calculate it are data collected i the same year. We ext recall the defiitio of the diachroous impact factor. The -year diachroous impact factor of joural J for the year Y, deoted as IM (J,Y), is defied as s+ s it( Y + i, Y ) IM ( J, Y ) = (2) ub( Y ) where s = 0 or, depedig o whether oe icludes the year of publicatio or ot. The term diachroous refers to the fact that the data that are used to calculate this impact factor derive from a umber of differet years with a startig poit somewhere i the past ad ecompassig subsequet years (Igwerse et al., 200). Rakig ivariace with respect to o-cited items We cosider the followig form of ivariace. If a performace idicator I is calculated for jourals J ad ad I ) < I ) the, if we add the same umber of publicatios with zero citatios, we require that also for the ew situatio I ) < I ). We refer to this requiremet as rakig ivariace with respect to o-cited items. This otio is totally differet from the cosistecy otios itroduced by Waltma ad va Eck (Waltma & va Eck, 2009; Waltma et al., 20) or by Marchat (2009) (uder the ame of idepedece). Recall that, for good reasos, the otio of cosistecy as defied by these authors refers to cases where the umber of publicatios (i the deomiator) is the same for both jourals. We do ot require this. Next we show that impact factors are ot rakig ivariat with respect to ocited items. osider the followig example (see Table ). Table : Data for the calculatio of the Garfield impact factor (RoA case) for the year Y. J ub(y-) 0 (+25) 30 (+25) ub(y-2) 0 30 it(y,y-) 30 60 it(y,y-2) 30 60 O the basis of Table, the Garfield impact factors of jourals J ad are IF 2,Y) = 3 ad IF 2,Y) = 2, so that IF 2,Y) > IF 2,Y). However, addig 25 o-cited publicatios, yields the ew impact factors: IF 2,Y) = 60/45 =.33 ad IF 2,Y) = 20/85 =.4, so that for the ew situatio the relatio betwee the impact factors reverses. We ote that for the classical sychroous impact factor IF 2 rakig ivariace with respect to o-cited items always holds for the special case that both IF ) < IF ) ad the sum of it(y,y-) ad it(y,y-2) is smaller for joural J tha for joural (or is equal). Ideed: deotig ISSI NEWSLETTER Vol. 7. r.. Iteratioal Society for Scietometrics ad Iformetrics
short commuicatios, articles the 2-year impact factor of joural J i (i =,2) simply by i / i we have: 2 < ad 2 2 If ow Z deotes the added umber of publicatios with o citatios we have to show that: 2 < + Z + Z 2 ( + Z) < ( + Z) 2 2 + Z < + Z 2 2 2 This is clearly true sice 2 < 2 ad Z 2 Z. We also ote that if This implies that also uder these coditios rakig ivariace with respect to o-cited items holds. Deotig by IF Z (J) the impact factor of joural J whe Z ocited items are added leads us to the followig characterizatio result. ropositio. If IF ) < IF ) the rak reversal, i.e. IF Z ) > IF Z ) occurs if ad oly if > 2 ad Z > D 2 where D = 2-2. roof. We already kow that if 2 the there is o rak reversal. If ow IF ) < IF ) this implies that 2 > 2. Its (positive) differece 2-2 is deoted as D. We have ow the followig equivaleces: IF Z ) > IF Z ) ( +Z) 2 < ( 2 +Z) 2 + D + Z 2 < 2 + Z This proves the propositio. < ad the <, 2 2 2 2 2 hece also. 2 D < ( - 2 )Z Z > D 2 This result shows the exact requiremets to have rak reversal i the case of the classical sychroous impact factor, ad hece whe it does ot occur. We cotiue our ivestigatios by cosiderig the diachroous impact factor. A simple variatio of Table shows that also the diachroous impact does ot satisfy this property either, see Table 2. Table 2: Data for the calculatio of the dyamic (=diachroous) impact factor (RoA case) for the year Y. J ub(y) 20 (+25) 60 (+25) it(y,y) 0 20 it(y,y+) 20 40 it(y,y+2) 30 60 With s = 0, we have IM 3,Y) = 60/20 = 3 ad IM 3,Y) = 20/60 = 2. Addig 25 o-cited publicatios yields the ew diachroous impact factors: IM 3,Y) = 60/45 =.33 ad IM 3,Y) = 20/85 =.4. The rak-order of the two jourals i terms of their impact factor is thus reversed by addig a equal umber of o-cited items to both. The characterizatio provided above also holds for the diachroous impact factor as it too is determied by dividig a umber of citatios by a umber of publicatios. AoR versus RoA We have show that the stadard sychroous impact factor is of the RoA-form ad that it does ot satisfy rakig ivariace with respect to o-cited items. Let us aalyze whether perhaps a Average of Ratios (AoR) form of the sychroous impact factor behaves better i this respect. First we defie the ARIF as: ARIF ( J, Y ) = it( Y, Y i) (3) ub( Y i) However, it turs out that the AoR-form behaves eve worse with respect to rakig ivariace. Ideed, cosider the case of a two-year impact factor (ARIF 2 ) ad ISSI NEWSLETTER Vol. 7. r.. Iteratioal Society for Scietometrics ad Iformetrics 2
short commuicatios, articles assume that jourals J ad have i each year equal umbers of publicatios. If RAIF 2,Y) > RAIF 2,Y), (or, IF 2,Y) > IF 2,Y)), this meas that joural J received more citatios tha joural (i the year Y). Addig the same umber of zero-cited publicatios to both, does ot chage the total umber of citatios received, ad hece J s stadard impact factor remais smaller tha (of course both impact factors decrease by icreasig the deomiators). The same argumet holds for the -year sychroous impact factor (RA-case). This, however, does ot hold for ARIF. osider the example show i Table 3. Table 3. Data for the calculatio of a two-year sychroous impact factor (AoR case) for the year Y. J ub(y-) 30 (+0) 30 (+0) ub(y-2) 20 20 it(y,y-) 0 20 it(y,y-2) 80 0 Based o the data show i Table 3, we have: ARIF 2,Y) = (0.5).(0/30+80/20) = 2.7 ad ARIF 2,Y) = (0.5).(20/30+0/20) = 2.25 so that ARIF 2,Y) < ARIF 2,Y). However, addig 0 publicatios i the year Y- yields the ew impact factors: ARIF 2,Y) = (0.5). (0/40+80/20) = 2.3 ad ARIF 2,Y) = (0.5). (20/40 + 0/20) =.75, so that for the ew situatio ARIF 2,Y) > ARIF 2,Y). ARIF is more sesitive to addig publicatios with o citatios to the deomiator tha RAIF because ARIF is a average (cf. Ahlgre et al., 2003); RAIF, however, is ot a average, but a quotiet betwee two summatios (Egghe & Rousseau, 996). It is easy to fid similar examples of violatios agaist the assumptio of rakig ivariace for ay -sychroous impact factor calculated i the AoR way. A remark cocerig the framework of impact factor calculatios: ucitable items Whe Garfield itroduced the impact factor, he decided to itroduce the otio of ucitable items. The idea was that jourals should ot be puished for publishig obituaries, correctios, editorials ad similar types of publicatios, which usually receive o or few citatios. Although this seems reasoable, there are i practice two problems with this otio. Oe is to decide which publicatios are ucitable, ad the other oe is the fact that Garfield also decided to iclude citatios to these ucitable articles whe they occur - to the total umber of received articles. It has bee show, see e.g. (Moed & va Leeuwe, 995) that this practice may lead to serious distortios i joural impact. Assume ow that if ucitable items could be defied uambiguously, ad that they are really ever cited, which way of calculatig a impact factor is the better? Takig all publicatios ito accout (icludig the ucited ucitable oes), or takig oly the citable oes (cited or ot)? The aswer is clearly that the secod method should be used, as otherwise it would be possible that joural J obtais a higher impact tha joural J due to ucitable publicatios. oclusio We have show that, as a cosequece of simple arithmetic, ot satisfyig the requiremet of rakig ivariace with respect to o-cited items is a ormal mathematical property related to takig ratios. For the calculatio of sychroous ISSI NEWSLETTER Vol. 7. r.. Iteratioal Society for Scietometrics ad Iformetrics 3
short commuicatios, articles impact factors, the stadard RoA approach is to be preferred above the AoR approach, as the RoA approach satisfies rakig ivariace with respect to o-cited items for jourals with the same umber of publicatios, while the AoR approach may fail eve i this case. We characterised whe stadard impact factors fail to have the property of rakig ivariace with respect to o-cited articles. Our fidigs provide a ratioale for ot takig ucitable publicatios ito accout i impact factor calculatios, at least if these items are truly ucitable, that is, are really ever cited. Furthermore, they provide aother argumet agaist usig averages i the case of highly skewed distributios (Ahlgre et al., 2003; Borma & Mutz, 20; Leydesdorff & Opthof, 20). Ackowledgemets The authors thak Raf Gus (UA), Wolfgag Gläzel ad two aoymous reviewers for helpful suggestios. Refereces Ahlgre,., Jarevig, B., & Rousseau, R. (2003). Requiremet for a ocitatio Similarity Measure, with Special Referece to earso s orrelatio oefficiet. Joural of the America Society for Iformatio Sciece ad Techology, 54(6), 550-560. Garfield, E. (972). itatio aalysis as a tool i joural evaluatio. Sciece 78 (Number 4060), 47-479. Igwerse,., Larse, B., Rousseau, R. & Russell, J. (200). The publicatio-citatio matrix ad its derived quatities. hiese Sciece Bulleti, 46(6), 524-528. Leydesdorff, L., & Opthof, T. (20). Remaiig problems with the New row Idicator (MNS) of the WTS. Joural of Iformetrics, 5(), 224-225. Marchat, T. (2009). A axiomatic characterizatio of the rakig based o the h-idex ad some other bibliometric rakigs of authors. Scietometrics, 80(2), 325-342. Moed, H.F. & va Leeuwe, Th. N. (995). Improvig the accuracy of istitute for scietific iformatio s joural impact factors. Joural of the America Society for Iformatio Sciece, 46(6), 46-467. Rousseau R. (988). itatio distributio of pure mathematics jourals. I: Iformetrics 87/88 (Egghe L. & Rousseau R., eds). Amsterdam: Elsevier, pp. 249-262. Waltma, L. & va Eck, J.N. (2009). A taxoomy of bibliometric performace idicators based o the property of cosistecy. I: ISSI 2009 (B. Larse & J. Leta, Eds.). Sao aulo: BIREME & Federal Uiversity of Rio de Jaeiro, pp. 002-003. Borma, L., & Mutz, R. (20). Further steps towards a ideal method of measurig citatio performace: The avoidace of citatio (ratio) averages i field-ormalizatio. Joural of Iformetrics, 5(), 228-230. Waltma, L., Va Eck, N. J., Va Leeuwe, T. N., Visser, M. S., & va Raa, A. F. J. (20). Towards a ew crow idicator: some theoretical cosideratios. Joural of Iformetrics, 5(), 37-47. Egghe, L., & Rousseau, R. (996). Averagig ad globalisig quotiets of iformetric ad scietometric data. Joural of Iformatio Sciece, 22(3), 65. ISSI NEWSLETTER Vol. 7. r.. Iteratioal Society for Scietometrics ad Iformetrics 4