KU Leuven, ECOOM & Dept MSI, Leuven (Belgium)

Transcription

1 KU Leuven, ECOOM & Dept MSI, Leuven (Belgium)

2 Structure... Basic postulates in bibliometrics.... On probability distributions and stochastic processes A model for scientific productivity A model for citation processes... Probability distributions and stochastic processes Asymptotic normality of scientometric indicators... The h-index The tail parameter. /

3 Foreword The mathematical context of scientometrics/bibliometrics In this session we will focus on the mathematical foundation of and on models for fundamental bibliometrics processes, such as publication and citation processes. Several deterministic and stochastic models will be introduced and discussed in the context of building and interpreting indicators as (statistical) functions derived from the model. Special a ention is paid to the high end of performance represented by the heavy tails of the underlying distributions. Communication networks and their graph and vector-space representation, which form the mathematical groundwork for structural analysis, require more time and space, and are therefore omi ed here. Part of the theory is presented in the following lecture. /

4 Foreword The mathematical context of scientometrics/bibliometrics In this session we will focus on the mathematical foundation of and on models for fundamental bibliometrics processes, such as publication and citation processes. Several deterministic and stochastic models will be introduced and discussed in the context of building and interpreting indicators as (statistical) functions derived from the model. Special a ention is paid to the high end of performance represented by the heavy tails of the underlying distributions. Communication networks and their graph and vector-space representation, which form the mathematical groundwork for structural analysis, require more time and space, and are therefore omi ed here. Part of the theory is presented in the following lecture. /

5 Introduction The mathematical context of scientometrics/bibliometrics Whenever a discipline reaches a stage that requires the support of statistical methods, a metrics emerges from this field. Classical examples are biometrics, psychometrics and econometrics. In this context P ( ) explained the term bibliometrics as the application of mathematical and statistical methods to books and other media of communication. /

6 Introduction The application of mathematical models, notably of stochastic models has the following important advantages.. It provides mathematical interpretations beside the scientometric ones (even beyond the particular field). Mathematical interpretation of scientometric measures can be given by parameters and statistical functions.. Although deterministic models also allow randomness, the use of probabilistic models opens new perspectives, above all, concerning inference. /

7 Introduction The deterministic approach is the easiest way to process simple counts of raw data and measurements to indicators. Mostly elementary mathematical operations (e.g., shares, averages, ratios) are used. The interpretation of more complex measures and constructs (composite indicators) becomes more problematic. In social processes, observations are subject to a variety of influences that are partially not directly measurable. The complexity of social interactions itself yields the effect that is usually interpreted as randomness. In bibliometrics, the events seem to be random as they are conditioned by a plethora of superposing actions, processes and effects. Communication, mobility, collaboration, publications and citations all are subject to these effects. G M, Thoughts and facts on bibliometric indicators, /

8 Basic postulates in bibliometrics Basic postulates help properly define bibliometrics measures. A paper receives at one time at most one citation. An author publishes only one article at one time. A paper does not receive citations prior to it its publication. The citation link between two papers is unique. The first two postulates are necessary to define mappings and allow the application of point processes. Simultaneousness of publications and citations can be solved by using an infinitesimal number ε > for publications or citations, e.g., in the same issue of a journal at time t, t + ε, t + ε, etc. The fourth postulate is straightforward and the third one is necessary to make quantification of citation impact possible. /

9 Basic postulates in bibliometrics The publication process from the bibliometric viewpoint (at time t and in the period T = [s, t] for some time s < t) Authors F t Papers Co-authors Authors F T Papers Co-authors t 1 t 3 t 2 /

10 Basic postulates in bibliometrics The citation process from the bibliometric viewpoint (at time t and in the period T = [s, t] for some time s < t) Source papers Citing papers Y t References Source papers Citing papers Y T References t 1 t 3 t 2 /

11 Basic postulates in bibliometrics In the first case, the mapping F t (for each time t being the publication date) describes an authorship. The period T = [s, t] is called publication period. F T then defines the publication process of an author. The complete origin Ft of a given publications is the set of its co-authors: If we denote the set of authors by A and that of papers by B, then we have for each element b B : Ft = {a A : F t (a) = b}. In the second case, the mapping Y t (for each time t being the publication date of citing paper) describes a citation. The period T = [s, t] is called citation window. Y T then forms the citation process of a paper. The complete origin Yt of a given publications is the set of its references: If we denote the set of source papers by A and that of citing papers by B, then we have for each element b B : Yt = {a A : Y t (a) = b}.

12 Models for the dissemination of scientific information Probability distributions A random variable (X) is a function defined on a sample space. Then its probability distribution is a function that assigns certain values (x) to the r. v. (X) describing its probability: P(X x) [, ]. Two basic types of distributions: Discrete distributions (taking values in a discrete set, e.g. non-negative integer values) Continuous distributions (taking values in a continuous set, e.g. real values) Probability distributions are used to model, for instance, the publication activity of an author, the citation impact of a paper or the number of co-authors of a paper.

13 Models for the dissemination of scientific information Probability distributions A random variable (X) is a function defined on a sample space. Then its probability distribution is a function that assigns certain values (x) to the r. v. (X) describing its probability: P(X x) [, ]. Two basic types of distributions: Discrete distributions (taking values in a discrete set, e.g. non-negative integer values) Continuous distributions (taking values in a continuous set, e.g. real values) Probability distributions are used to model, for instance, the publication activity of an author, the citation impact of a paper or the number of co-authors of a paper.

14 Probability distributions Example of discrete distribution (le ) and the density function of a continuous distributions (right) 0.30 Poisson distribution 0.50 Standard normal distribution 0.25 P(X=k)=e - k /k! ( =2) k x

15 Probability in bibliometrics Bibliometric distributions are typically skewed and have long tails. Furthermore, most distributions (publication activity, citation rates, co-authorship) are integer-valued. Below an example of a skewed, long-tailed and integer-valued distribution is given. Empirical distribution of citations a scientist received on this work f k >20 k

16 Stochastic processes In probability theory, stochastic processes are families (i.e., collections) of random variables {X t t T} defined on a given probability space, indexed by a totally ordered set T. The la er set T, which is considered to represent time might be discrete or continuous. Distinction has to be made between a stochastic process and a time series. While a stochastic process is a well-structured mapping describing evolution, a time series is a sequence of data that are measured at different time instants. Examples: Stochastic process: The cumulative number of papers an author has published in his/her career The cumulative number of citations a paper has received since its publication Time series: The annual impact factor of a journal.

17 Stochastic processes In probability theory, stochastic processes are families (i.e., collections) of random variables {X t t T} defined on a given probability space, indexed by a totally ordered set T. The la er set T, which is considered to represent time might be discrete or continuous. Distinction has to be made between a stochastic process and a time series. While a stochastic process is a well-structured mapping describing evolution, a time series is a sequence of data that are measured at different time instants. Examples: Stochastic process: The cumulative number of papers an author has published in his/her career The cumulative number of citations a paper has received since its publication Time series: The annual impact factor of a journal.

18 Stochastic processes In probability theory, stochastic processes are families (i.e., collections) of random variables {X t t T} defined on a given probability space, indexed by a totally ordered set T. The la er set T, which is considered to represent time might be discrete or continuous. Distinction has to be made between a stochastic process and a time series. While a stochastic process is a well-structured mapping describing evolution, a time series is a sequence of data that are measured at different time instants. Examples: Stochastic process: The cumulative number of papers an author has published in his/her career The cumulative number of citations a paper has received since its publication Time series: The annual impact factor of a journal.

19 Models for the dissemination of scientific information A model for scientific productivity In order to facilitate comprehensibility and interpretation, the publication process is first introduced using a deterministic model. Three groups of individuals are assumed.. Those who entering the system,. Those who are staying in the system and. Those who are leaving it. The system is described as follows. S G, Scientometrics,

20 Waring process Consider an infinite array of units indexed in succession by the non-negative integers. The content of the i-th unit is denoted by x i, the (finite) content of all units by x. y i = x i /x (i ) expresses the share of elements contained by the i-th cell. The change of content is postulated to obey the following rules. (i) Substance may enter the system from the external environment through -th unit at a rate s. (ii) Substance may be transferred unidirectionally from the i-th unit to the (i + )-th one at a rate f i (i N ). (iii) Substance may leak out from the i-th unit into the external environment at a rate g i (i N ).

21 Waring process Scheme of substance flow in the Waring process We interpret the above ratios y i as the (classical) probability with which an element is contained by the i-th unit. The stochastic process is then formed by the change of the content of the units, i.e., by the change of papers published by the authors who have entered the system. X(t) denotes the (random) number of published papers, P(X(t) = i) = y i the probability that an author in the system has published i papers in the period (, t).

22 Waring process Scheme of substance flow in the Waring process We interpret the above ratios y i as the (classical) probability with which an element is contained by the i-th unit. The stochastic process is then formed by the change of the content of the units, i.e., by the change of papers published by the authors who have entered the system. X(t) denotes the (random) number of published papers, P(X(t) = i) = y i the probability that an author in the system has published i papers in the period (, t).

23 Waring process Using the above notations, we can give a mathematical formulation for the equations of change in the system. x (t) = s(t) f (t) g (t). x i (t) = f i (t) f i (t) g i (t), (i > ) ( ) The following particular forms of the above rate terms are used: s = σ x f i = (a + b i) x i, (i ) g i = γ x i ; (i ) ( ) where σ, a, b and γ are non-negative real values.

24 Waring process Eqs ( ) and ( ) result in x (t) = Σx i = (s Σg i ) = (σ γ) x y (t) = σ (a + σ) y. y i (t) = (a + b (i )) y i (a + b i + σ) y i, (i > ) ( ) with the initial conditions y i ( ) = { if i = otherwise. We obtain x(t) = x( ) e (σ γ) t, i.e., the system is asymptotically time-invariant (stationary) only if σ = γ, otherwise, if σ > γ or σ < γ, it exponentially grows or decays, respectively.

25 Waring process The general solution of the above system of differential equations, which is independent of γ, reads y i (t) = i b ij e (a+b i+σ)t + σ a + σ j= i j= a + b (j ) a + b j + σ. As t tends to infinity, a Waring distribution (red expression) with parameters N = a/b and α = σ/b is obtained, while the blue expression vanishes. The following three special cases should be mentioned.. α = : Price distribution. N = : Yule distribution. b = : Geometric distribution

26 Waring process The general solution of the above system of differential equations, which is independent of γ, reads y i (t) = i b ij e (a+b i+σ)t + σ a + σ j= i j= a + b (j ) a + b j + σ. As t tends to infinity, a Waring distribution (red expression) with parameters N = a/b and α = σ/b is obtained, while the blue expression vanishes. The following three special cases should be mentioned.. α = : Price distribution. N = : Yule distribution. b = : Geometric distribution

27 Waring process The general solution of the above system of differential equations, which is independent of γ, reads y i (t) = i b ij e (a+b i+σ)t + σ a + σ j= i j= a + b (j ) a + b j + σ. As t tends to infinity, a Waring distribution (red expression) with parameters N = a/b and α = σ/b is obtained, while the blue expression vanishes. The following three special cases should be mentioned.. α = : Price distribution. N = : Yule distribution. b = : Geometric distribution

28 Waring process Derrek de Solla Price distinguished the following categories of authors: newcomers, continuants, transients and terminators. P G, International Forum on Information and Documentation, Within the framework of this model, s represents the group of newcomers, g i terminators, g transients and Σf i represents the group of continuants. Further developements and applications: P, JASIS, X, Utilitas Mathematica, X, Journal of the Royal Statistical Society, Series A, G S, Scientometrics, B., Drug Metabolism Reviews, S., Scientometrics, B, Scientometrics,

29 Waring process Example: Distribution of authors by publication productivity Part k Number of authors Obs Calc Obs Calc Obs Calc Obs Calc Obs Calc Obs Calc n p KS N ˆα Legend: k: number of papers, n: total number of authors, p: total number of papers, KS: K-S statistics Data source: S G ( ); P J ( )

30 Waring process Example: Distribution of authors by publication productivity Part k Number of authors Obs Calc Obs Calc Obs Calc Obs Calc Obs Calc Obs Calc n n KS N ˆα Legend: k: number of papers, n: total number of authors, p: total number of papers, KS: K-S statistics Data source: S G ( ); P J ( )

31 Waring process Population growth (T d. a) of authors according to the example by S G ( )

32 The Price distribution DeSolla Price square root law states that half of the scientific papers are contributed by the top square root of the total number of scientific authors. Versions of the Waring, Zipf and Lotka distribution have studied for satisfying this law. G S, Scientometrics, E R, Journal of Information Science, E, Scientometrics,

33 The Price distribution Fit of the Price (N =. ) and Lotka (α = ) distribution to the original Lotka data k Observed Calculated frequency frequency Lotka Price >

34 A Negative Binomial Process Consider now the same array of units as in the case of publication activity. We now assume that the system is isolated from external influences, i.e., no substance enters or leaves the system. Thus σ = γ = and only rule (ii) of the preceding paragraph remains valid and takes the following form. f i = (a + bi) x i ; a(t)/b(t) = N = const > (ii ) Hence x(t) = x( ) follows, where x( ) > is assumed. The special assumption x( ) = does not mean any restriction of generality. The system is described as follows. G S, Scientometrics, G S, Information Processing & Management,

35 A Negative Binomial Process Scheme of substance flow in the non-homogeneous birth process Let X(t) denote the number of citations received, P(X(t) = k) = y k the probability that paper has been cited k times till time t. With the above transition rules, we can write y i (t) = {(N + i ) y i (N + i) y i }b(t), (i > ) with the same initial conditions as in the case of the Waring process.

36 A Negative Binomial Process Scheme of substance flow in the non-homogeneous birth process Let X(t) denote the number of citations received, P(X(t) = k) = y k the probability that paper has been cited k times till time t. With the above transition rules, we can write y i (t) = {(N + i ) y i (N + i) y i }b(t), (i > ) with the same initial conditions as in the case of the Waring process.

37 A Negative Binomial Process Hence we obtain by successive integration the following solutions. ( ) N + k P(X(t) = k) = y k (t) = e r(t)n ( e r(t) ) k ; r(t) = k t b(u) du The probability that at time t the substance is in the k-th unit, provided it was in the i-th one at time s t, is denoted by p ik (s, t) (k i). Now the initial conditions are { if k = i p ik (s, s) = otherwise.

38 A Negative Binomial Process Hence we obtain by successive integration the following solutions. ( ) N + k P(X(t) = k) = y k (t) = e r(t)n ( e r(t) ) k ; r(t) = k t b(u) du The probability that at time t the substance is in the k-th unit, provided it was in the i-th one at time s t, is denoted by p ik (s, t) (k i). Now the initial conditions are { if k = i p ik (s, s) = otherwise.

39 A Negative Binomial Process The second system of differential equations p ik (s, t)/ t = {(N + k )p i,k (s, t) (N + k)p ik (s, t)} b(t) ; k > i results in the following solution. P(X(t) X(s) = j X (s) = i) = ( ) N + i + j = e (r(t) r(s))(n+i) ( e (r(t) r(s)) ) j, k The following two special cases worth mentioning.. Geometric distribution (N = ). Poisson distribution (N, q ; N(q ) c < )

40 A Negative Binomial Process The second system of differential equations p ik (s, t)/ t = {(N + k )p i,k (s, t) (N + k)p ik (s, t)} b(t) ; k > i results in the following solution. P(X(t) X(s) = j X (s) = i) = ( ) N + i + j = e (r(t) r(s))(n+i) ( e (r(t) r(s)) ) j, k The following two special cases worth mentioning.. Geometric distribution (N = ). Poisson distribution (N, q ; N(q ) c < )

41 A Negative Binomial Process The mean value function of the process is defined as the regression function of X(s, t) on X(s) = i. M i (s, t) = E(X(t) X(s) X(s) = i) = (N + i)(e r(t) r(s) ) ; i, t s This function will play an important role in the applications. Under the above conditions we have M(s, t) = E(X(t) X(s)) = N(e r(t) e r(s) ) The la er equation reflects the non-homogeneity of the process, i.e., for example, M(s, s + h) M(t, t + h) if s t (h > ). Non-homogeneity is an important property of citation processes.

42 A Negative Binomial Process Two examples for different ageing types of scientific literature a) fast (E(X) = ) and b) extremely slow (E(X) = ) maturing decline Year after publication (x) E(X) = 4 E(X) = 9

43 Stopping times of citation processes So far we have considered the change of the number of citations over time. In order to provide additional information about the succession of citations a paper receives during a certain time span [, t n ], we develop an appropriate model for the reception speed. Let T i denote the shortest period during which the paper has received i citations: T i = min{t n : X(t n ) i}. Random variables of this type are called stopping times. Three important properties can be derived from the definition: (i) P(T i = t n ) = P(X(t n ) i) P(X(t n ) i) (ii) P(T i = ) (iii) P(T i = t ) = P(X(t ) i)

44 Stopping times of citation processes Some conclusions from these properties: (ii) P(T i = ) > if the paper does not receive i citations, specifically, P(T = ) > if the paper is never cited. (iii) P(T i = t ) = if i =. Some further developements and applications: G, Information Processing & Management, R, Scientometrics, B, Scientometrics, E, Mathematical and Computer Modelling,

45 Probability and statistics Pólya s urn model A general model for probability distributions and stochastic processes with implications for scientometrics Assume an urn that contains a number of w white and b black balls, where w and b are positive integers. Balls are drawn at random and then returned together with s balls of the same colour, where s is an integer. If s is negative, balls of the same colours are removed (in the case of s = the drawn ball is not placed back), while s = means that no additional balls are placed back. Two basic situations are obtained resulting in two specific random variables.. The number k of white balls a er n draws.. The number k of draws till a given number n of black balls is drawn.

46 Pólya s urn model The first type of distributions is always finite. Examples: s = : binomial s < : hypergeometric Examples for the second type: s = : negative binomial (k = : geometric) s > : inverse Pólya-Eggenberger (k = : Waring). The urn model can also be used to model some stochastic processes. J K, Distributions in Statistics: Discrete distributions, J K, Urn models and their application, Advantage of the model: The self-reinforcing property reflects the cumulative advantage or succes-breeds-success phenomenon. P, JASIS,

47 Probability distributions Examples of differently shaped probability distributions obtained from the urn model (le : no tail, right: heavy tail) Geometric Neg. binomial 0.4 Waring Inv. Póly-Eggenberger f k 0.3 f k k k

48 Asymptotic normality of scientometric indicators Authorship, publication activities, references, citations and other links can be expressed by random variables. Nevertheless, the application of most mathematical-statistical methods to scientometrics are approximate solutions.. One reason is ambiguity and uncertainty (cf. Bookstein, ). Another reason: Normality is the basis and the condition of many statistical tests. Most scientometric distributions are discrete and extremely skewed, i.e., not normal, but most indicators are approximately normally distributed.

49 Asymptotic normality of scientometric indicators Authorship, publication activities, references, citations and other links can be expressed by random variables. Nevertheless, the application of most mathematical-statistical methods to scientometrics are approximate solutions.. One reason is ambiguity and uncertainty (cf. Bookstein, ). Another reason: Normality is the basis and the condition of many statistical tests. Most scientometric distributions are discrete and extremely skewed, i.e., not normal, but most indicators are approximately normally distributed.

50 Asymptotic normality of scientometric indicators Authorship, publication activities, references, citations and other links can be expressed by random variables. Nevertheless, the application of most mathematical-statistical methods to scientometrics are approximate solutions.. One reason is ambiguity and uncertainty (cf. Bookstein, ). Another reason: Normality is the basis and the condition of many statistical tests. Most scientometric distributions are discrete and extremely skewed, i.e., not normal, but most indicators are approximately normally distributed.

51 Asymptotic normality of scientometric indicators Authorship, publication activities, references, citations and other links can be expressed by random variables. Nevertheless, the application of most mathematical-statistical methods to scientometrics are approximate solutions.. One reason is ambiguity and uncertainty (cf. Bookstein, ). Another reason: Normality is the basis and the condition of many statistical tests. Most scientometric distributions are discrete and extremely skewed, i.e., not normal, but most indicators are approximately normally distributed.

52 Asymptotic normality of scientometric indicators Theorem (Central limit theorem) Let X, X,..., X n be a sequence of n independent and identically distributed random variables with finite expectation μ and variance σ >. Then the distribution of the random variable Z n = ( n i= X i nμ)/σ n converges weakly to the standard normal distribution N (, ) as n tends to infinity. Remark Under certain conditions (e.g., Lindeberg or Lyapunov condition), a weaker form of the central limit theorem, where identical distribution is not required, holds.

53 Asymptotic normality of scientometric indicators Theorem (Central limit theorem) Let X, X,..., X n be a sequence of n independent and identically distributed random variables with finite expectation μ and variance σ >. Then the distribution of the random variable Z n = ( n i= X i nμ)/σ n converges weakly to the standard normal distribution N (, ) as n tends to infinity. Remark Under certain conditions (e.g., Lindeberg or Lyapunov condition), a weaker form of the central limit theorem, where identical distribution is not required, holds.

54 Central limit theorem Remark As a consequence, the sample mean of random variables x = X i /n with any distribution belonging to the a raction domain of the normal distribution is approximately normally distributed. We have E( x) = μ and D( x) = σ/ n, where μ and σ are the expectation and standard deviation of the common distribution. According to Glivenko s theorem, the empirical distribution converges to the underlying theoretical one with probability. The relative frequency f is an unbiased and consistent estimator of the corresponding probability p. In particular, we have E(f ) = p, where p is the probability that a paper is not cited and D(f ) = p ( p )/n.

55 Central limit theorem Remark As a consequence, the sample mean of random variables x = X i /n with any distribution belonging to the a raction domain of the normal distribution is approximately normally distributed. We have E( x) = μ and D( x) = σ/ n, where μ and σ are the expectation and standard deviation of the common distribution. According to Glivenko s theorem, the empirical distribution converges to the underlying theoretical one with probability. The relative frequency f is an unbiased and consistent estimator of the corresponding probability p. In particular, we have E(f ) = p, where p is the probability that a paper is not cited and D(f ) = p ( p )/n.

56 Central limit theorem Remark As a consequence, the sample mean of random variables x = X i /n with any distribution belonging to the a raction domain of the normal distribution is approximately normally distributed. We have E( x) = μ and D( x) = σ/ n, where μ and σ are the expectation and standard deviation of the common distribution. According to Glivenko s theorem, the empirical distribution converges to the underlying theoretical one with probability. The relative frequency f is an unbiased and consistent estimator of the corresponding probability p. In particular, we have E(f ) = p, where p is the probability that a paper is not cited and D(f ) = p ( p )/n.

57 Approximate normality of means and shares Sample means and shares of uncited papers ( % of Belgian publications in with year citation window) k n f 0 k n f 0 k n f % % % % % % % % % % % % % % % % % % % % Total % G M, STI, ; Data source: Thomson Reuters Web of Knowledge

58 Approximate normality of means and shares Truncated moments for sample means and shares of uncited papers ( % of Belgian publications in with year citation window) D x y = 5.73x R² = d x D x y = 0.218x R² = d x G M, STI, ; Data source: Thomson Reuters Web of Knowledge Plot based on a characterisation theorem for the normal distribution by Glänzel (, )

59 The tail of scientometric distributions Let X be a random variable, in the present case X the citation rate of a paper. The probability mass function of the non-negative integer valued r.v. X is denoted by p k = P(X = k) for each k, the distribution function by F k = P(X < k). Furthermore we put G k := F k = P(X k). Consider now a given sample {X i } i=,...,n of size n. Assume that all elements are independent and identically distributed with F being the common distribution. Further assume that the sample elements X i are ranked in decreasing order X X..., X i... X n. Although this can be readily obtained from an ordinary ordered sample by replacing index i by (n i + ) for all i =,..., n, we will use the terms rank statistics of a statistical sample or simply ranked sample.

60 Ranked samples The easiest way to calculate theoretical values for rank statistics is using Gumbel s r-th characteristic extreme value (u r ). G, Statistics of extremes, u r := G ( r { n ) = max k : G k r }, n n is a the size of a given sample with distribution F. X r can be considered an estimator of the corresponding r-th characteristic extreme value u r.

61 The h-index Example of Gumbel s characteristic extreme values Waring distribution with N = a =2 and n = n G k u 10 u 9 u 8 u 7 u 6 u u 4 u u 2 u k

62 The h-index Jorge E. Hirsch introduced a new indicator called h-index for the assessment of the research performance of individual scientists. A scientist has index h if h of his or her N p papers have at least h citations each and the other (N p h) papers have h citations each. H, PNAS US, The papers meeting this criterion are also referred to as Hirsch core publications. An alternative index was introduced by Leo Egghe. A set of papers has a g-index g if g is the highest rank such that the top g papers have, together, at least g citations. E, Scientometrics, Alternative definition: g is the highest rank such that the top g papers have, on average, at least g citations.

63 The h-index Jorge E. Hirsch introduced a new indicator called h-index for the assessment of the research performance of individual scientists. A scientist has index h if h of his or her N p papers have at least h citations each and the other (N p h) papers have h citations each. H, PNAS US, The papers meeting this criterion are also referred to as Hirsch core publications. An alternative index was introduced by Leo Egghe. A set of papers has a g-index g if g is the highest rank such that the top g papers have, together, at least g citations. E, Scientometrics, Alternative definition: g is the highest rank such that the top g papers have, on average, at least g citations.

64 The h-index The h- and g-index (Example) Ri Cites Mean

65 The h-index In order to define the theoretical h-index we use Gumbel s r-th characteristic extreme value u r. u r := G (r/n) = max{k : G(k) r/n}, where n is a the size of a given sample with distribution F, k, r n and G = F. The theoretical h-index (h), can be defined as { max{r : u r r} = max{r : max{k : G k r/n} r}, if n > and X, h :=, otherwise, whereas the empirical h-index (ĥ) can analogously be re-defined as { max{r : max{k : X k r} r}, if n > and X, ĥ :=, otherwise.

66 The asymptotic normality of the h-index Theorem (Beirlant & Einmahl, ) Let X be the number of citations received by an author s publications. The underlying citation distribution is denoted by F. If F is continuous and x = sup{x R : F(x) < } =, then ĥ h h + n(f(ĥ) F(h)) h D N (, ), for n. B E, Asymptotics for the Hirsch Index,

67 The asymptotic normality of the h-index Definition The distribution of the r.v. X is called Paretian if G(x) = x α l(x), where l is a slowly varying function such as l(ux) lim = for all u >. x l(x) Corollary (Beirlant & Einmahl, ) If F satisfies the von Mises condition, i.e. lim consistent estimator for α, then xg (x) x G(x) + α (ĥ ĥ h) D N (, ). = α and α is a

68 Estimation of the tail parameter The estimation of the tail parameter α of Pareto-type distributions has received much a ention. Assume that {X i } n i= is a sample of iid r.v.s with Paretian distribution. Then the ranked sample has the following property. P(k log(x k /X k+ ) < x) e αx ; k k Hence Hill s estimator ( ) for α can be derived as the mean of the upper k elements of this series. H k = k k log X i log X k+ i= H k is asymptotically normally distributed (if k n) with variance /(kα ). This allows to construct confidence intervals for α.

69 QQ-plots Remark Unfortunately, the Hill estimator is not robust since it depends on the particular choice of k, and is sensitive to large values of log(x k /X k+ ). Recently methods are developed to robustify the estimator and to reduce its bias. Another method for tail analysis is the quantile-quantile plot (QQ-plot), where observations are plo ed against the quantiles of a given distribution. If the observations follow this distribution, the graph is a straight line. According to Beirlant et al. ( ), the Paretian distributions result in a linear ( log(i/(n + ), log X i ) plot, where the slope is approximately α.

70 QQ-plots Remark Unfortunately, the Hill estimator is not robust since it depends on the particular choice of k, and is sensitive to large values of log(x k /X k+ ). Recently methods are developed to robustify the estimator and to reduce its bias. Another method for tail analysis is the quantile-quantile plot (QQ-plot), where observations are plo ed against the quantiles of a given distribution. If the observations follow this distribution, the graph is a straight line. According to Beirlant et al. ( ), the Paretian distributions result in a linear ( log(i/(n + ), log X i ) plot, where the slope is approximately α.

71 QQ-plots Pareto QQ-plots for (a) Garfield and (b) another Price Medallist with the same h-index (with k = and fi ed least squares line) B., Journal of Informetrics, ; Data source: Thomson Reuters Web of Knowledge

72 Conclusions Rank-based tests are sensitive to ties, which, in turn, o en occur if observations are integer-valued. These tests should therefore be applied with the utmost care. Mean values and relative frequencies are unbiased and consistent estimators for the expectation and the corresponding probabilities. Their use in bibliometrics is therefore correct and not just a workaround. However, the underlying publication set needs to be large enough. papers is a commonly accepted lower limit. The deviation of indicators like mean citation rates and the share of [un]cited papers from corresponding values of other samples or from given reference values or expectations (or probabilities) can be tested for significance, provided the size of the underlying paper set is large enough. Seemingly large deviations between indicators may prove to be not significant. The tail parameter provides information about the high-end of activity and (citation) impact.

73 Conclusions Rank-based tests are sensitive to ties, which, in turn, o en occur if observations are integer-valued. These tests should therefore be applied with the utmost care. Mean values and relative frequencies are unbiased and consistent estimators for the expectation and the corresponding probabilities. Their use in bibliometrics is therefore correct and not just a workaround. However, the underlying publication set needs to be large enough. papers is a commonly accepted lower limit. The deviation of indicators like mean citation rates and the share of [un]cited papers from corresponding values of other samples or from given reference values or expectations (or probabilities) can be tested for significance, provided the size of the underlying paper set is large enough. Seemingly large deviations between indicators may prove to be not significant. The tail parameter provides information about the high-end of activity and (citation) impact.

74 Conclusions Rank-based tests are sensitive to ties, which, in turn, o en occur if observations are integer-valued. These tests should therefore be applied with the utmost care. Mean values and relative frequencies are unbiased and consistent estimators for the expectation and the corresponding probabilities. Their use in bibliometrics is therefore correct and not just a workaround. However, the underlying publication set needs to be large enough. papers is a commonly accepted lower limit. The deviation of indicators like mean citation rates and the share of [un]cited papers from corresponding values of other samples or from given reference values or expectations (or probabilities) can be tested for significance, provided the size of the underlying paper set is large enough. Seemingly large deviations between indicators may prove to be not significant. The tail parameter provides information about the high-end of activity and (citation) impact.

75 Conclusions Rank-based tests are sensitive to ties, which, in turn, o en occur if observations are integer-valued. These tests should therefore be applied with the utmost care. Mean values and relative frequencies are unbiased and consistent estimators for the expectation and the corresponding probabilities. Their use in bibliometrics is therefore correct and not just a workaround. However, the underlying publication set needs to be large enough. papers is a commonly accepted lower limit. The deviation of indicators like mean citation rates and the share of [un]cited papers from corresponding values of other samples or from given reference values or expectations (or probabilities) can be tested for significance, provided the size of the underlying paper set is large enough. Seemingly large deviations between indicators may prove to be not significant. The tail parameter provides information about the high-end of activity and (citation) impact.

76 Conclusions Rank-based tests are sensitive to ties, which, in turn, o en occur if observations are integer-valued. These tests should therefore be applied with the utmost care. Mean values and relative frequencies are unbiased and consistent estimators for the expectation and the corresponding probabilities. Their use in bibliometrics is therefore correct and not just a workaround. However, the underlying publication set needs to be large enough. papers is a commonly accepted lower limit. The deviation of indicators like mean citation rates and the share of [un]cited papers from corresponding values of other samples or from given reference values or expectations (or probabilities) can be tested for significance, provided the size of the underlying paper set is large enough. Seemingly large deviations between indicators may prove to be not significant. The tail parameter provides information about the high-end of activity and (citation) impact.