Existence and convergence of moments of Student s t-statistic

Transcription

1 U.U.D.M. Report 8:18 Existence and convergence of moments of Student s t-statistic Fredrik Jonsson Filosofie licentiatavhandling i matematisk statistik som framläggs för offentlig granskning den 3 maj, kl 13.15, sal 3, Ångströmlaboratoriet, Uppsala Department of Mathematics Uppsala University

2

3 Existence and convergence of moments of Student s t-statistic Fredrik Jonsson April 8, 8 Abstract Let {X i} i N be a sequence of independent, identically distributed random variables. Denote their common probability distribution F and define, for n, X n = 1 n ˆσ n = nx X i, i=1 1 n 1 nx `Xi X n, i=1 and the t-statistic random variables n ˆσ T n = n Xn, ˆσ n >,, ˆσ n =. We investigate relations between the statements E T n r <, and couples n, r) N R + and distributions F. Moreover, we investigate, assuming the existence of a random variable T such that under which circumstances lim n Tn = d T, lim n E Tn r = E T r. Some results concerning upper and lower bounds of tail probabilities of T n, mainly of asymptotic concern, will be derived and presented. We also discuss some implications of our results to statistics. 1

4 Contents 1 Introduction Point masses in F and the relation between T n and S n /V n Statement of the problem, organization of results and existing literature Motivation to the problem Moments and tail probabilities Distributional convergence and domains of attraction A result of Hotelling Notation Basic results 1.1 Existence Convergence Tail probabilities A necessary condition for existence of moments Further results Sufficiency conditions for existence of moments Some remarks concerning Theorem Necessity conditions in connection with Theorem Hotelling s condition and Theorem Examples A distribution with non-separated point masses Continuous distributions on compact support Continuous distributions on unbounded support Discrete distributions Summary and conclusions Some implications for statistical inference Proofs for Section Proofs for Section Proofs for Section Proofs for Section Proofs for Section Proofs for Section Proofs for Section Proofs for Section Proofs for Section Acknowledgements 7

5 1. Introduction Assume in the following that {X i } i N is a sequence of independent, identically d distributed random variables, X i = X. Denote their common probability distribution F and define, for n, S n := n X i, i=1 X n := n 1 S n, n ) 1/ V n := X i, U n := S n V n i=1, ˆσ n := 1 n 1 n Xi X ), n i=1 M n := max 1in X i, and the t-statistic random variables { n T n = ˆσ Xn n, ˆσ n >,, ˆσ n =. The distribution of T n is the well known t-distribution with n 1 degrees of freedom, when F is the standard normal distribution Point masses in F and the relation between T n and S n /V n Note to begin with that the event V n =, which causes trouble in the definition of U n, is identical to all observations X i, 1 i n, are equal to zero. 1) In this case we define U n = S n /V n =. Moreover, U n = n i=1 X i) n i=1 X i n i=1 1) n i=1 X i n i=1 X i = n, ) by the Cauchy-Schwarz inequality. Equality holds in ) if and only if all observations X i, 1 i n, are equal and non-zero. 3) The events 1) and 3) occur with positive probability only if F contains point masses. Note that an equivalent definition of T n, for the case ˆσ n >, is T n = n 1 n S n V n n n U n ) 1/, n. 3

6 As noted previously, e.g. in the article 5] by Efron, this may be reformulated into the following relations between the n:th t-statistic and the corresponding, so called, self-normalized sum S n /V n : for x, ˆσ n > : for x, ˆσ n > : T n > x S n V n > T n < x S n V n < ) 1/ nx n + x, 4) 1 ) 1/ nx n + x. 5) 1 This implies e.g. that T n converges in distribution toward a certain distribution as n tends to infinity precisely when the same is true of S n /V n. Moreover, for x, ˆσ n > : T n > x U n > nx n + x 1. 6) In order to maintain the relations 4) 6) also for the case ˆσ n = we introduce, for n, { Un, U n = n, or V n =, := U n, else. Recall that the starting motivation is to avoid T n undefined or equal to infinity with positive probability. In order for the relations 4) 6) still to hold this requires the modification Un of U n. Then indeed, for x : T n > x U n > nx n + x 1. 7) 1.. Statement of the problem, organization of results and existing literature Our primary aim is to investigate, for couples n, r) N R + and distributions F, under which circumstances E T n r <. This means that we intend to present necessary and sufficient conditions relative to what is found to be natural parameters of the problem. Our results on this are to be found in Sections.1 and 3, with proofs of lemmas, propositions and theorems deferred to Sections 6 and 7. We present analyzed examples to further illustrate the question, as well as the results of Sections and 3, in Section 4. Secondly, we intend to investigate, under the assumption of a random variable T such that lim n T n d = T, under which circumstances lim E T n r = E T r. n Section 1.5 reviews conditions regarding the distributional limit relation, which is often encountered in applications. The question of moment convergence is answered mainly in Section.. 4

7 Although our main concern is existence and convergence of moments, some results concerning upper and lower bounds of tail probabilities, mainly of asymptotic concern, will also be derived and presented in Section.3. We refer to Section 1.4 for the connection between these questions. The first and third problems have previously partly been investigated by Hotelling 9], under the assumption that F is absolutely continuous. We refer to Section 1.6 for a discussion of these results. There does not, to our knowledge, exist any other published material concerning theoretical results for the three problems Motivation to the problem One may ask, in view of well known results for non-randomly normalized partial sums S n, whether the moment properties of T n are connected to similar properties of X. Another supposition could be that there exist some uniform positive answers over n, r and F ) to some of the questions raised. Still another motivation to the problem, more closely related to the field of statistical inference, will be described next. The context of statistical inference There is, ever since the origin of modern statistical theory, a wide-spread use of t-statistic random variables in inference concerning the expectation EX. This is performed mainly under the assumption that X follows a normal distribution. The testing situation is often the following: The null hypothesis is EX = and observations of T n that are large in absolute value are used to reject the hypothesis. The idea behind this is that T n, for large values of n, has approximately a normal distribution when EX < and the null hypothesis, EX =, is at hand. If, on the other hand, the null hypothesis is not at hand, then it may be argued that, for µ := EX and σ := Var X, S n = S n + nµ V n V n µ n σ + µ, 8) T n = S n + nµ nˆσn n µ σ, 9) which leads to good possibilities of hypothesis rejection. Exact distributions of S n /V n and T n which depend on the parameter µ/σ) can, in case X is assumed to be normally distributed, be derived and may in turn be used to determine the rejection region with respect to a certain significance level) and power functions of the test. The effect of nonnormality on such results is naturally a question of statistical interest and has also been treated in the literature from different points of view and in different contexts. This leads into, what is sometimes called, the field of Robustness of statistical methods. See the review paper 3] and the paper 8] for some introduction into this area. The relations 8) and 9) also reveal an explanation to why T n is often preferred in place of S n /V n, namely since the former has a more influential drift term in case the null hypothesis is not at hand, leading to larger powers of the test. 5

8 The relation 9) also indicates that T n could be used to estimate the true value of the fraction µ/σ. This is of use e.g. in the important situation when one fails to reject the null hypothesis on a small sample, but believes that this is due to sample size and would like to know an appropriate size for rejection. We admit that the concerns regarding the probabilistic model and its use in the situations just described rather depend on the magnitude of tail probabilities up to certain levels than on the question whether absolute moments are finite confer Section 1.4. for the relationship). Finiteness of high order moments of T n is nevertheless, at least to some extent, desirable under testing circumstances, since the converse indicates that values unexpectedly large in absolute value have a certain tendency to occur. The question of moments also leads back to the comparison 8) 9). A presence of infinite absolute moments E T n r indicates that it could be the case that T n pays for a greater efficiency, compared to S n /V n, by having a distribution which is more spread out. It might therefore be interesting to know what properties influence the presence of infinite absolute moments; if they are related to location shifts of the underlying distribution; if the effects can be avoided by imposing regularity restrictions on F ; and if such restrictions may have no substantial effect on typical modelling. Notice that our results also cover some upper and lower bounds on tail probabilities of T n. This discussion is continued in Section Moments and tail probabilities There is a close relationship between moments of random variables and their tail probabilities: F x) := P X > x). The following well known result gives an explicit form to this relationship: Lemma 1. For any non-negative random variable X, and any r R +, EX r = r x r 1 P X > x ) dx. The result reveals in particular that the question of existence of absolute moments is related to the asymptotic behaviour of F with respect to magnitude). We will several times in the following use a general application of Lemma 1, which for convenience is stated in the following lemma: Lemma. For any finite collection of non-negative, independent and identically distributed random variables Y i, Y i = Y, i=1,... n), and any r R +, d n { n } ] E Y r i I Y i > i=1 i=1 = r Proof. Note that n ) ] P Y 1 i > x = i=1 h r+1) P Y < h )) n P Y = )) n ]dh. )] n P Y 1 > x = P Y < x 1)] n. 6

9 Note also that the event n i=1 Y i = has probability P Y = )) n, so that n { n } ] P Y 1 i I Y i > > x = P Y < x 1)] n P n. Y = )] i=1 i=1 Then apply Lemma 1 and make the change of variables h = x 1. Note that x r 1+ transforms into h r+1) Distributional convergence and domains of attraction Chistyakov and Götze ] have established the following result. Theorem 3. The sequence {S n /V n } converges in distribution precisely when i) F belongs to the domain of attraction of the normal distribution and EX =. The limit distribution is then the standard normal distribution. ii) F is α-attracted, < α <. For the cases α > 1, also assume that EX =, and for the case α = 1 that X is in the domain of attraction of the Cauchy-distribution and satisfies the so-called Feller-condition. iii) F has a slowly varying tail. The limit distribution is then a distribution concentrated on { 1, 1}. By the statement F is in the domain of attraction of the distribution G or, equivalently, F is G-attracted, it is meant that there exist real sequences {a n } and {b n } such that S n a n b n d G, as n. 1) We refer to the articles 1] and 4] for representations of the limit distributions in ii) and iii), and for further results in this area. Comments regarding relation 1) and Theorem 3 Characterizations, relative to F, regarding the relation 1) and for which {a n }, {b n } and G it holds are wellknown. Cf. e.g. the book 7]Section 9.3] or the book 1]Section 8.3]. The possible limit distributions in 1) are characterized, among other parameters, by α, < α <. Define here and throughout, with respect to F, α := sup { r : E X r < }. The corresponding quantity for G then necessarily coincides with α. The normal distribution case i) however differs in this respect, since then α G =, while only α is necessary concerning X. Characterizations for 1) with respect to F involve asymptotic conditions, in the sense that regularly varying and well-balanced right and left distribution function tails are demanded. Note that E X α may be both finite and infinite in cases i) ii). Note also that no non-trivial statement of the form 1) holds in case iii). This case may however be looked upon as α =, since tails are assumed to be slowly varying and since E X r =, for all r >. 7

10 Different behaviours in cases i) iii) are also encountered in the limiting behaviour of the random variables M n /V n. In each case there exists a limit distribution and the limits are and 1 respectively in cases i) and iii), cf. the book 1]Section 8.15]. The case α = 1 in ii) requires that F is in the domain of attraction of the Cauchy-distribution, which means that the right and left distribution function tails of F are asymptotically equal. The additional requirement, that F satisfies the so-called Feller-condition, is equivalent to the fact that the sequence {a n } in 1) may be chosen as {cn}, for some constant c. The subgaussian property Giné, Götze and Mason 6] have shown that self-normalized sums converging in distribution are subgaussian, in the sense that: sup E exp ) ] ts n /V n exp ct ), 11) n for some constant c. From this it follows, for each of the instances referred to in Theorem 3, that all moments E S n r V n converge toward the corresponding moment of the limit distribution. It also follows from 11) that all moments of a limit distribution of a sequence of self-normalized sums are finite A result of Hotelling It is previously known, Hotelling 9]), for the case when F is absolutely continuous with density function f, that the condition f n z)z dz < 1) is necessary and sufficient for equal asymptotic order of magnitude, up to a multiplicative constant, of the right tails of the n:th t-statistic and the right tails of the corresponding t-distribution. Moreover, the constant in question only depends on the quantity in 1), and the tails of the t-statistic random variable dominate the tails of the t- distribution, in the sense that the ratio tends to infinity, when the converse of 1) holds. For the corresponding result concerning the left tails, the condition becomes The condition f n z)z dz <. 13) f n z) z dz <, 14) which we shall refer to as Hotelling s condition, is, by symmetry of the t- distributions, therefore necessary and sufficient for asymptotic equality of the order of magnitude, up to a multiplicative constant, of the tails of T n and the corresponding t-distribution. The t-distributions are well known with respect to the questions raised in Section 1.3: the r:th moment exists precisely when r < n 1 and converges as n toward the corresponding standard normal moment. We collect the most relevant of these facts in a proposition. 8

11 Proposition 4. Suppose that F is absolutely continuous. It is necessary that r < n 1 for E T n r < to hold. In case 14) holds it is also sufficient. An early comparison between Hotelling s and forthcoming results The result of Hotelling, by Lemma 1, gives exact information concerning which moments exist for the t-statistic at hand under condition 14); however no exact information in case the condition is not fulfilled and no information about convergence of moments. The necessary part of Proposition 4 may be referred to as the Principle of moments only below the degrees of freedom. We shall see that it generalizes, except for instances where F is discrete cf. Theorem 11). It may, for various purposes, be natural to split, for large constants C, the integral in 14) into three parts: f n z) z dz + f n z) z dz + f n z) z dz. z C 1 C 1 z C z C To impose finiteness on the middle term is nothing but demanding local integrability of f n on the relevant interval. Finiteness of the other terms differs in the sense that there is a dependency on the behaviour of F at and ±. A related threefold partition of conditions may be found in our main result regarding sufficiency conditions for existence of moments, Theorem 15. We refer to Section 3.4 for a further comparison between these results and Hotelling s conditions. Our results are based upon the inequalities in Lemmas 5 and 6 below. This analysis enables, not only statements about existence and convergence of moments in a general setting, but also statements about tail probabilities. It is however not sufficiently precise to deliver statements about exact asymptotic magnitudes as in the case of Hotelling s results Notation The letter n will throughout be used to denote the number of X i -variables upon which the t-statistic is based, while the letter r will denote the order of an absolute moment. The same letter C will throughout be used for positive constants irrelevant for the context. If magnitude matters, or later reference occurs, C is sometimes replaced by C 1, C, etc., or e.g. by C θ, if the constant depends on a parameter θ. Regarding F The captital letter F will both refer to the probability distribution function of the random variable X, F x) = P X x), and to the corresponding probability distribution, i.e. probability measure. Note that there is a one-to-one correspondence between probability distribution functions and probability measures on R equipped with the Borel σ-algebra. This duality in notation is common in probability theory and should not cause any contradiction. Recall that the right tail probabilities were defined as F x) := 9

12 P X > x). The notation df x) = P X dx) will be used in the representation of integrals of R R measurable functions with respect to the measure F. Another classical result states that there exist certain canonical decompositions of probability measures. We transfer this result also to distribution functions and the representation of integrals, in the sense that the notation F = F ac + F cs + F d, F c = F ac + F cs will be used for the Lebesgue measure decomposition into absolutely continuous, continuous singular, discrete and continuous parts. Note that the recently introduced letters still refer to distribution functions and measures, which however may be defective, in the sense that their total mass, or limiting value at positive infinity respectively, may be less than one. The same notation is, for reasons of simplicity and unification, extended to random variables: X c has distribution F c etc. This introduces a slight danger of confusion and contradiction, e.g. since X c may typically be defective, which here means that P X c R) < 1. The notation will however only be used when merely properties of F matter, and not e.g. when there is room for interaction between random variables. The density function of F ac is denoted by f. Recall that α := sup { r : E X r < }. Moreover, to avoid repetition we use the shorthand notation { α, if E X α < ; α := any β < α, if E X α =. The motivation is the following. When α henceforth occurs in an assumption it normally is the case that the assumption becomes weaker as β increases to α, while a conclusion in which α occurs becomes stonger as β increases to α. This is combined with the notation concerning measure decompositions in the following way: α c := sup { γ : E X c γ < }, and α c := { α c, if E X c αc < ; any β < α c, if E X c αc =. Similarly for α d, α d, etc. The restriction of a measure F to a set A is denoted F A. The letter I will be reserved for the indicator function of a set A, which is written I A or I{A}. Asymptotic relations We finally adopt the following notation from the theory of real numbers and functions: x refers to the integer part of a number x the closest integer equal to or below x) and x to the closest integer equal to or above x. The reader is perhaps also familiar with the following three definitions for functions f, g : R + R + : f = Og) fx) C 1 gx), for some C 1, C and all x C f = og) f = hg, with lim hx) =, x f g f = hg, with lim hx) = 1, x f g f = Og) and g = Of). 1

13 The two latter definitions could also be modified, so that the concern, in place of x is e.g. x. The intention will be explicitly stated, or clear from the context. 11

14 . Basic results Proofs of lemmas, propositions and theorems of this section are, with some exceptions, deferred to Section 6. It follows by Lemma 1 and the fact that U n n, that E T n r = E Tn r = r E S n r r = E Un = r V n n Rewrite the right-hand side of 15), via the relation 7), as r y r 1 P Un ny ) > dy, n + y 1 which, by a change of variables, transforms into r n y r 1 P T n > y ) dy, 15) z r 1 P U n > z ) dz. 16) z r 1 P Un > z ) n 1 ) r 1 n 1)n dz. 17) n z n z) Introduce a parameter δ, < δ < 1 and split 17) into r r n δ n n δ z r 1 P Un > z ) n 1 n z z r 1 P Un > z ) n 1 n z ) r 1 n 1)n dz, n z) 18) 1 n 1)n dz. n z) 19) The quantity 18) is easily seen to be finite. The proof of Theorem 9 will moreover show that it, in case any of the conditions 1) 3) of Theorem 3 are fulfilled, converges to E T r as n tends to infinity, for each choice of δ. We are therefore lead to investigate finiteness and convergence to zero as n tends to infinity, of the quantity 19), for varying choices of r. We may, by another change of variables, equivalently do the same for n n) δ ) r h r+1) P n U n < h ) dh, ) and some choice of δ, < δ < 1. Roughly speaking, n U n is small when all variables X 1,..., X n are approximately but not exactly) equal. It therefore turns out to be possible to estimate the probability in question by conditioning on the value of one of the variables. That is, we replace ) by the conditioned version δ n n) h r+1) P n Un < h X 1 = x ) dh df x), 1) which is nothing else but δ n n) h r+1) P < n U n < h X 1 = x ) dh df x). ) 1

15 However, the case X 1 = deserves some extra attention, e.g. since there is a convention regarding the event V n =. By the equality in ) the Cauchy- Schwarz inequality), it follows from X 1 = that U n n 1. Since δ < 1, the case x = may therefore be excluded from the outer integral in ). By defining p x := P X = x), ) may then be written as n n) x δ h r+1) P n U n < h X 1 = x ) ] p x dh df x). 3) The following lemmas provide useful estimates for the questions of finiteness and convergence to zero of 3). Lemma 5. Let x = x 1,..., x n ) R n and h, 1) be given with x 1 and define n ) n u n = x i / x i. Suppose that n u n < h. Then with C 1 = 5. Moreover, i=1 i=1 x i x 1 < hc 1 x 1, for all i 1, C 1 = C 1 n, h) = + h + h, is an optimal constant for the conclusion to be valid for all x. Lemma 6. Let h, x and u n be given as in Lemma 5. Suppose that x i x 1 < C h x 1 n 1 for all i 1, with C = 1. Then n u n < h. Moreover, C = C n, h) must, in case n is even, satisfy n C n h, for the conclusion to be valid for all x. By independence and Lemma 5 we now get the upper bound n n) x δ h r+1) P X x < hc1 x )) ] p x dh df x), 4) for 3) and by Lemma 6 the lower bound n n) x δ h P r+1) X x < C h x n 1 )) p x ] dh df x). 5) 13

16 .1. Existence We collect and slightly refine some of the results of the discussion 15) 3) combined with the bounds 4) and 5) and Lemma in the following theorem. Theorem 7. The following are equivalent: i) E T n r < ; 1 ii) iii) E x X 1 r n h r+1) P X x < h x )) p x i= ] dh df x) < ; X i X 1 r I { X i X 1 >, some i n }] <. From Theorem 7 we deduce the next result. Theorem 8. For any couple n, r) N R +,.. Convergence E T n r < = E T n+1 r <. From the discussion 15) 3), the bound 4) and Theorem 8 we deduce the following theorem. Theorem 9. If, for some random variable T, lim n T n couple n, r) N R +, d = T, then, for any E T n r < = lim E T n r = E T r. n.3. Tail probabilities We next consider the implications of Lemmas 5 and 6 regarding estimation of tail probabilities of T n. It follows from relation 7) that Note that P T n > y ) = P Tn > y ) = P Un > x = P n Un < = P n Un < ny ) n + y 1 n n ) n + y 1 n n ) n + y 1 X 1 = x df x). 6) n n n + y < 1 y > n 1, 1 so that the integral in 6), as in pages 1-13, for y > n 1 may be written as P n U n < n n ) ] n + y 1 X 1 = x p x df x). 7) The estimates of Lemmas 5 and 6 therefore yield the following proposition: 14

17 Proposition 1. For y > n 1 and n n h := n + y 1, the following relations hold: P T n > y ) P T n > y ) x x C n,y := + h + h ) n 1), P X x < C ] n,y n x p n + y 1)) df x), 8) P X x < n x p n + y 1)) x Proof of Proposition 1. Assume that y > n 1 and set h := x ] df x). 9) n n n + y 1. 3) To prove 8), it suffices, by Lemma 5 and 6), to verify that h + h + h )n 1)n + h + h =. 31) n + y 1 Equality 31) is immediate from 3). To prove 9), apply Lemma 6 with C = 1 and 6). It then remains to verify that h n 1 = n n + y 1. 3) Equality 3) is also equivalent to 3)..4. A necessary condition for existence of moments The necessary condition in Proposition 4: E T n r < = r < n 1, in case F = F ac, may in fact be rediscovered from the bound 9). This may be argued for as follows: When F is continuous all p x in 9) vanish. The remaining probabilities may, for h = y 1 and some constant C 4, be minorized by ) P X x < hc 4 x, which in turn satisfy ) lim h h 1 P X x < hc 4 x = C 4 fx) x, 33) 15

18 for almost all x, by a well known theorem of Lebesgue. It therefore, in the absolutely continuous case, follows by 9), 33) and the Fatou lemma that lim inf y y P T n > y) lim inf h ) P X x < hc 4 x )) fx) dx h lim inf h ) P X x < hc 4 x )) ] fx) dx h = C fx) x ) fx) dx >. 34) It follows from 34), and the fact that 1 y 1 dy =, that y n P T n > y ) dy =. 35) We conclude from 35) and Lemma 1 that E T n =. The following theorem, also based on the estimate 9), extends this result to a larger class of distributions. Theorem 11. Suppose that F c. It is then, for E T n r <, necessary that r < n 1. 16

19 3. Further results The aim is now to further investigate what conditions to impose on F, n and r to ensure the statement E T n r <. The idea is therefore to turn Theorem 7 into more explicit conditions by finding natural, but more basic, conditions to impose on F. Theorem 15 below presents such conditions and claims their sufficiency for the desired conclusion. The main point of the examples in Section 4 is to establish that these conditions indeed are natural with respect to the desired conclusion). A further step in this direction is given in Section 3.3. Proofs of propositions and theorems of this section are, with some exceptions, deferred to Section Sufficiency conditions for existence of moments The characterization given in condition ii) of Theorem 7 leads to the following preliminary result: Proposition 1. The following condition, when satisfied for some γ > r, is sufficient for E T n r < to hold. { h γ )) ] ] } P X x < x h p x 1 x α <. 36) sup h>, x Condition 36) reduces to a bound on concentration functions see Definition 13 below) when X is bounded and F is continuous. Theorem 15 gives appropriate additional conditions for more general cases. Definition 13. Given a distribution function F on R and a parameter θ, θ, define the concentration functions h ) { Q F h; θ) := F x + h ) F x h )}, sup x h θ x+h θ Q F h) := Q F h; ), q F h; θ) := sup x x h θ x+ x h θ q F h) := q F h; ). { F x + x h ) F x x h )}, The one-variable function Q F is often referred to as the Lévy concentration function. We will assume, for simplified and stronger results, that F has separated point masses according to the following definition. Definition 14. The distribution of a random variable X is said to have separated point masses whenever there exists some ε > such that, for all x such that p x := P X = x) >, it holds that P X x < ε) = p x. Since T n all are invariant with respect to scaling of F we shall henceforth without loss of generality assume that ε = 1 in Definition 14. Arguments depending on appropriate scaling of X will not occur elsewhere. We let in this case {x k } k Z, with x k < x k+1 for all k, be an ordered collection of the points with point masses and define p k := P X = x k ). There is 17

20 no restriction to assume that such an ordering is given, by the assumption of separated point masses. One could moreover recursively) introduce points with point mass zero into this ordering to allow the convenient assumption of precisely one point x k in each interval k, k + 1). Recall from the introduction that F d and F c refer to the discrete and continuous parts of the distribution function or probability measure) F, and that F cs and F ac refer to the continuous singular and absolutely continuous parts of F c. The density function of F ac is denoted f. Theorem 15. Suppose that F has separated pointmasses. i) iii) is sufficient for E T n r <. i) F c, r and n satisfy both a) and b); ii) F c, r and n satisfy either c), d) or e); iii) F d, r and n satisfy f), The conjunction where a) q Fc h; 1) = O h λ1 ), with r < λ 1 1; b) Q Fc h; 1) = O h λ ), with r < λ 1; c) Q Fc h; θ) = Oh), for some θ, and { } nαc r min, n 1)α c + α ; 1 + α c d) F cs has compact support and, for some η, and, in case η < 1, { r min fx) x η C 1 for x C, nα c 1 η + α c, } n 1)α c + α 1 η α c ) ) ; e) F cs has compact support and f is asymptotically decreasing as x and as x. f) For some κ, with κ 1 α d κ, and p k x k κ C, 37) r n 1)α d + α 1 κ + α d. 38) In case the latter majorizing bound of r equals n 1, replace condition 38) with 39). In case the same bound is strictly larger than n 1, or in case κ = 1 and α d =, replace 38) with 4). r < n 1 39) r n 1)κ + α 4) 18

21 We refer the reader to the examples in Section 4 for sharpness of the inequalities involving both r and n in Theorem Some remarks concerning Theorem 15 Condition ii) is asymptotic with respect to F c, while condition iii) is asymptotic with respect to F d. Only asymptotic conditions regarding F d are needed since F is assumed to have separated pointmasses. Condition c) could be generalized by introducing a parameter λ as in a) and b). The bounds of r in c) and d) simplify in case α c = α, since the first term then dominates the second in each minimum. Assumption 37) with κ = α d imposes no restriction on F d. It is also necessary that κ 1 α d. This follows by well known properties of convergent sums. It is also necessary, by similar principles, that 1 η + α ac in d). To compare the two upper bounds of r given in 38) and 4), note that, for any number ε, is equivalent to n 1)α d + α = n 1 + ε)1 κ + α d ) 41) n 1)κ + α = n 1 + ε1 κ + α d ). 4) Setting ε = in 41) 4) shows that the upper bounds coincide in case any of them equals n 1. Moreover, since 1 κ + α d 1, the bound in 38) is more restrictive when the bounds are smaller than n 1 corresponding to ε < ), while the bound in 4) is more restrictive in case the bounds are larger than n 1 corresponding to ε > ). As a comment on the conjunction of a) and b) in Theorem 15 we present the following proposition. Note that the right-hand sides in i) iii) occur as assumptions in a) and b). Proposition 16. For any λ >, any distribution F with finite total mass and any θ, < θ <, i) Qh) = Oh λ ) = Qh; 1) = Oh λ ), qh; 1) = Oh λ ); ii) qh) = Oh λ ) = Qh; 1) = Oh λ ), qh; 1) = Oh λ ); iii) Qh; θ) = Oh λ ), qh; θ) = Oh λ ) Qh; 1) = Oh λ ), qh; 1) = Oh λ ). The implications i) and ii) are strict for any λ >. The next result explains the restriction λ 1, λ 1 in conditions a) and b) of Theorem 15. Also recall Theorem 11: r < n 1 is necessary in case F c. 19

22 Proposition 17. For any λ > 1, any distribution F with finite total mass and any θ, θ, i) Qh; θ) = Oh λ ) F { x θ} ; ii) qh; θ) = Oh λ ) F { x θ} = pδ, for some p. We finally note that the conditions put on the concentration functions in Theorem 15 are fulfilled with λ = 1 for all absolutely continuous distributions with a bounded density function Necessity conditions in connection with Theorem 15 By the results so far, it could be the case that conditions a) and b) fail to cover broad classes of distributions F for which E T n r <. 43) The restrictions imposed on r and n through the parameters λ 1 and λ could moreover, in general or in specific cases, be far from the actual restrictions of the situation. It turns out, as Theorem 18 reveals, that negative answers could be given in both cases. Conditions a) and b) therefore seem natural in the context considered. However, they do not provide necessary conditions for 43), since the choice of the λ-parameter may be somewhat too demanding. The corollary below settles for which values of λ conditions a) and b) become necessary for 43) to hold. Theorem 18. Assume that E T n r <, for some couple n, r). Then, for λ = r n, q Fc h) = Ohλ ). Corollary 19. Assume that E T n r <, for some couple n, r). Then, for λ = r n, i) Q Fc h; 1) = Oh λ ); ii) q Fc h; 1) = Oh λ ). Proof of Corollary 19. Corollary 19 follows from Theorem 18, since Q ; 1) q, q ; 1) q. Examples in Section 4. show that the conditions put on q ; 1) and Q ; 1) in Theorem 15 are sharp with respect to the parameters λ 1 and λ, and that the condition of Theorem 18 is sharp with respect to the parameter λ Hotelling s condition and Theorem 15 A sufficient condition of Hotelling concerning existence of moments, under the assumption that F is absolutely continuous, has been discussed in the introduction. We now investigate its relationship to relevant conditions in Theorem 15. The focus is, to begin with, on assumptions a) and b) of the latter theorem.

23 Theorem. Suppose that F is absolutely continuous with density function f. Then, for any constant C and any real number n, i) qh; 1) = Oh) = ii) Qh; 1) = Oh) = iii) z <1 1< z <C f n z) z dz < ; f n z) z dz < = qh) = O ) h n. f n z) z dz < ; Remark 1. Statements i) and ii) are not surprising in view of Theorem 15 and Hotelling s result. Remark. The assumption in iii) implies, by Hotelling s result, that E T n r <, for r < n 1. 44) The conclusion in iii) combined with Proposition 17 ii) and Theorem 15 does not lead to this optimal condition, but merely to E T n r <, for r < n 1)n 1 n 1). We refer to examples in Section 4. where the conclusion in iii) is strongest possible while the assumption, and therefore also 44), prevails. Remark 3. It follows from 44) and Theorem 18 that qh) = O h γ n ), for any γ < n 1, 45) which is weaker than the conclusion in iii). Condition d) of Theorem 15 Assume that F is absolutely continuous. Condition d) was based on assumptions of the following kind: fx) x η < C 1, for x > C, some η. 46) Such conditions may also be used as sufficiency conditions for the asymptotic part of Hotelling s condition. By the latter we mean: z >C f n z) z dz <. 47) Indeed, assumption 46) with gives the bound η = 1 α ), 48) n 1 fz) z ) z α ) ]+ z α, 1

24 which guarantees 47), since then f n z) z dz fz) z α dz <. z >C z >C Recall that assumption 46) in condition d) of Theorem 15 leads to the assumption r nα c 1 η + α c, 49) while 47), combined with non-asymptotic counterparts, leads to the optimal assumption Comparing 49) and 5) under 48) r < n 1. 5) It follows, in case η = in 48), that α n 1 and that nα c = nα c nn 1) = n 1. 1 η + α c 1 + α c n It follows, in case η > in 48), that nα c 1 η + α c = nα c α + α c = n 1 51) The conclusion is therefore that, when assumptions of the kind 46) are made in order to derive statements of the kind 47), the same assumption also, by Theorem 15, gives rise to condition 49). The latter turns out to be no more restrictive than the optimal condition 5) which follows from 47). This motivates, to some extent, that there has not been made too weak use of the assumptions made in d) in the theorem. Note that 46) leading to 49) also covers situations in which 47) does not hold and in which 5) is not sufficient.

25 4. Examples 4.1. A distribution with non-separated point masses This example shows how the results in Section may be used to analyze the case of a distribution which is not covered by Theorem 15. It also points toward the fact that the occurrence of non-separated point masses may cause T n to have heavy tails. Let X have a point mass of weight 1 at x = 1 and let further fx) = 1, for 1 x 1. We claim that E T n r < r < 1. 5) The expectation in the left-hand side of 5) may, by Theorem 7, equivalently be replaced by 1 h r+1) )) ] P X x < h p x dh df x). 53) To prove that E T n =, consider only x = 1 Since 1 replace 54) by 1 in 53), h P X 1/ < h )) ) ] dh. 54) P X 1/ < h )) P X = 1/ )) + P X = 1/ )) n P < X 1/ < h ) = ) + ) h, 1 ) h 1 dh C 1 h 1 dh =. This finishes the proof of the left-to-right implication of 5). Now assume that r < 1. To verify finiteness of 53), split into two parts: 1 1 Proof of 55) For i n, h r+1) P X 1/ < h )) ) ] dh <, 55) h r+1) P X x < h )) ]dh dx <. 56) )) P X 1/ < h ) ) = P X i 1/ < h, all i; X i 1/, some i ) P < X i 1/ < h, some i n 1)P ) < X 1/ < h 3 = n 1)h.

26 Statement 55) therefore holds, since r < 1 so that Proof of 56) Split further into two parts: x 1 1 x 1 The integral in 57) reduces to n 1 1 h r dh <. h r+1) P X x < h )) ]dh dx <, 57) h r+1) P X x < h )) ]dh dx <. 58) x 1 x 1 h r+1) P X x < h; X 1/ )) ]dh dx h n r+) dh dx <, where finiteness is due the fact that Condition 58) also holds, since n > r x x r 1 h r+1) P X x < h )) ]dh dx h r+1) dh dx x 1/) r dx = 1 r 1 y r dy <. 4.. Continuous distributions on compact support Very slow decay of the concentration function This example illustrates Theorem 18. It is also an example of a distribution F for which E T n r =, for all r >. The tail behaviour of T n is analyzed through Proposition 1. Let X be absolutely continuous with support on 1/, 1] and density function One verifies, by basic calculus, that fx) = log ) 1 1 x) 1 log 1 x). F 1 ε) = 1 log1/), < ε < 1/. log ε 4

27 It follows that Q F h) = log h 1)) 1, which means that Q F does not satisfy condition b) of Theorem 15, regardless of the choice of λ 1. Theorem 18 therefore tells us that E T n r =, for all r >. 59) It is possible to further analyze by deriving lower bounds of the tail probabilities from Proposition 1. Indeed, for y > n 1, P T n > y ) 1 1 In particular, for any h < 1/4, = h 1 = log n x P X x < fx) dx. 6) n + y 1)) P X x < h )) fx) dx 1 h 1 1 log P X x < h )) fx) dx P X > x h )) fx) dx 1 h 1 log 1 x + h) ) 1 ] fx) dx 1 h log h 1 )] ) fx) dx = 1 log ) log h 1)) n. 61) The derivation 6) 61) shows that P T n > y ) decays at least as slowly as a constant times log y) n, for y > n 1. Density function blow-up at the origin We show, by a translate of the distribution of the previous example, that location shifts of distributions may, under certain circumstances, change the tail behaviour of T n dramatically. The example in addition shows that the implication in Proposition 16 i) is strict. Let X be absolutely continuous with support on 1/, ] and density function fx) = log ) 1 x) 1 log x). It follows, as in the previous example, that Q F h) = log h 1)) 1, while, by definition, Moreover, we claim that Q F h; 1). q F h; 1) = Oh), 6) 5

28 and that Hotelling s condition f n z) z dz < 63) is satisfied for all n. It then follows from either of 6) and 63) combined with previous results) that Proof of 63) E T n r < r < n 1. 64) fz)z f n z) z ) fz) dz = dz fz) dz = 1. Proof of 6) It follows from 63) and Theorem iii), that q F h; 1) = Oh λ ), for any λ < 1. 65) This result can be sharpened to 6), since, for x 1/, ], and h < 1/, 66) P X x xh ) = = = x1+h) x1 h) fy)dy 1 log y) 1] x1+h) log x1 h) 1 log log ] ] x1 + h) log x1 h) log x1 + h) ] log x1 h) ] 1 log log ] ] 1 + h) log 1 h) log x1 + h) ] log x1 h) ] log h log x1 + h) ] log x1 h) ] 67) h log log ) log 4/3)) 1. 68) The inequality in 67) follows from the mean value theorem, while the inequality in 68) is due to the conditions 66). Intermediate decay 1, smooth behaviour This example concerns sharpness of Theorem 18 and Theorem iii). A oneparameter family of absolutely continuous distributions, similar to the distributions of the preceding two examples, is presented. Sharpness regarding Theorem 18 is verified for all λ, < λ < 1 and for n Nλ), while sharpness regarding Theorem iii) is verified for all n. 6

29 Introduce a parameter < δ < 1 and let X = X δ be absolutely continuous with support on 1/, 1] and density function fx) = c δ 1 x) δ log 1 x), for a suitably chosen positive constant c δ. One verifies that, for n R +, 1 1 Theorem iii) combined with 69) gives that f n z) dz < nδ 1, 69) Q Xδ h) = O h 1 δ). 7) Statement 7) is in fact sharp with respect to the exponent 1 δ, since for λ > 1 δ, choose ε > such that It then follows that λ ε > 1 δ. 71) h λ P 1 h X 1 ) h = c δ h λ y δ log y) dy 7) h = c δ h λ y δ+ε y ε log y) dy h C h λ y δ+ε dy = C h λ+1 δ+ε. 73) The last expression tends to infinity as h tends to zero by 71). This shows sharpness of Theorem iii). Existence of moments for the example Intermediate decay 1 Hotelling s condition for integer n) is fulfilled precisely when n δ 1, cf. 69). When this holds, Hotelling s result gives that E T n r < r < n 1. 74) Theorem 15 combined with 7) gives for general n: while Theorem 18 gives: E T n r < = r < 1 δ)n 1), 75) E T n r < = r 1 δ)n. 76) We now claim that the correct description, in case is given by: nδ > 1 77) E T n r < r 1 δ)n. 78) 7

30 Proof of 78) It suffices, by 76) and 74), to show that under 77), Therefore, fix E T n r <, for r = 1 δ)n. 79) r := 1 δ)n. Statement 79) is, by Theorem 7, equivalent to 1 Split this condition into two parts: h r+1) P X x < h )) fx) dx dh <. 1 h 1 h h r+1) P X x < h )) fx) dx dh <, 8) h r+1) P X x < h )) fx) dx dh <. 81) Proof of 8) For a < b 1, b a fy)dy C b a 1 y) δ dy = C 1 a) 1 δ 1 b) 1 δ]. 8) For x < 1 h it then follows from 8) and the mean value theorem that P X x < h ) C 1 x + h) 1 δ 1 x h) 1 δ] ) δ 1 x Ch. 83) Apply 83) to 8): C 1 1 h h 1 h r+1) P X x < h )) fx) dx dh h n r+) 1 x) δn log 1 x)dx dh. Make the change of variables, y = 1 x, change the order of integration, then proceed to obtain = = C = C h h n r+) 1 x) δn log 1 x)dx dh y δn log y) y y δn+n r+1) log y) dy h n r+) dh dy 84) y 1 log y) dy <. 85) 8

31 Proof of 81) It follows, for x 1 h, that fx) = c δ 1 x) δ log 1 x) c1 x) δ log h) It then also follows that C1 x) δ log h). 86) P X x < h ) = 1 x h fy)dy Clog h) 1 x h 1 y) δ dy = Clog h) 1 x + h ) 1 δ Clog h) h 1 δ. 87) Apply 86) and 87) to majorize the left-hand side of 81) as follows C = C h h h h r+1) P X x < h)) fx) dx dh h r+1) h 1 δ)) log h) n 1 x) δ dx dh. h +δ log h) n 1 x) δ dx dh. Make the change of variables, y = 1 x, then proceed: = = C 1 1 h h +δ log h) n 1 x) δ dx dh h h +δ log h) n y δ dy dh h 1 log h) n dh <. Intermediate decay, Cantor distributions This example is presented in order to show sharpness of conditions a) and b) in Theorem 15. The distributions F are chosen to be uniform distributions on Cantor sets on the unit interval. Let ρ, 1) be a parameter and let, for k 1, X k = Xk, ρ) be a random variable with a uniform distribution on the set C k = C k ρ). Define the sets C k for fixed ρ recursively in k. Take C =, 1], and split each subinterval of C k 1 symmetrically into three parts; one centered middle part, and one left and one right part each of relative length ρ/ to the subinterval in question). The set C k is defined by keeping each such left and right part of the subintervals of C k 1 while deleting the middle parts. This means that each set C k consists of k subintervals of length ρ/) k and that X k is absolutely continuous with 9

32 density function taking the values and ρ k respectively. We further define C := C k, k=1 X : d = lim k X k. This means that X has a uniform distribution on C. The claim is then that, for the following relations hold: λ = log 1/) log ρ/), 88) Q X h) = Oh λ ), 89) E T n r < r < λn 1). 9) Proof of 89) Define Then the following holds: h k := ρ/) k. j Q Xk h j ) j+1, j k. 91) The second inequality comes from the fact that { X k x < h j } covers at most two of the subintervals of C j. The total mass of such subintervals remains constant, that is j, for k j. The first inequality is verified by assuming that x belongs to one of the subintervals of C k. The set { X k x < h j } then covers at least one of the subintervals of C j. In this case P X k x < h j ) j, j k. 9) The following is, by 88), an equivalent form of 91): h λ j Q Xk h j ) h λ j, j k. 93) In the limit as k, inequalities 93) and 9) transform into: h λ j Q X h j ) h λ j, j. 94) h λ j P X x < h j ), j, x C. 95) Statement 89) now follows from 94) by letting j. Proof of 9) It suffices, by Theorem 15, to show that E T n r =, for r = λn 1). 96) 3

33 Statement 96) is, by Theorem 7, equivalent to 1 h r+1) P X x < h )) df x) dh =. 97) Define n := min{j : h j 1} and partition the outer integral in 97) in order to apply 95). This yields C 1 j=n h j+1 hj h r+1) P X x < h )) df x) dh hj j=n hj j=n h r+1) P X x < h )) df x) dh h j+1 h r+1) h λ) j+1 dh h j+1 h 1 dh = C hn h 1 dh =. Intermediate decay 3, absolutely continuous distributions with oscillating behaviour The preceding example might seem artificial since the distribution functions are continuous but non-differentiable. We here define another parametrized family of distributions, now absolutely continuous. Sharpness of condition b) in Theorem 15 is almost, but not completely, verified from this example. Define, for k 1 and a parameter λ, 1), t := 1, k := t k t k 1 = λk) 1 log k) λ 1 +1), k+1 := t k+1 t k = k λ 1 log k) λ 1, I k := t k 1, t k ), w k := λ 1 k+1. It follows that {t k } is a discretization of some interval 1, C λ ) where the terms k dominate the terms k+1. Indeed, so that, for any j > k, Moreover, j i=k+1 k+1 k, i t j+1 t k+1 ) j i=k+1 i. 98) w k k+1 = k 1 log k). 99) 31

34 It also follows that densities f λ, such that {, if x I k, c λ f λ x) := w k, if x I k+1, are well defined on the interval 1, C λ ), for normalizing positive constants c λ. We shall henceforth suppress dependence on the parameter λ in the notation and simply refer to X i, T n, f and F. Finally, note that, for k, j) such that 1 k < j, Results We claim that and that j w i i+1 log k) 1 log j) 1, 1) i=k j i log k) λ 1 log j) λ 1. 11) i=k Q F h) = O h λ), 1) E T n r < r λn 1). 13) Note that statement 1) through Theorem 15 implies that E T n r < = r < λn 1). This means that the example demonstrates sharpness of condition b) in Theorem 15 almost, but not completely. Proof of 1) It must be verified, for some constant C not depeding on x or h, that P X x < h ) Ch λ. 14) In order to prove 1), let k be given and assume that Proof of 14), part I x I k+1. First consider h such that Then h k+1. 15) P X x < h ) hw k. But then also hw k = h λ 1 k+1 = hλ h 1 λ λ 1 k+1 hλ 1 λ k+1 λ 1 k+1 = hλ, proving 14) under condition 15). 3

35 Proof of 14), part II Next consider h such that It follows from the right inequality in 16) that k+1 < h t k+3 x. 16) P X x < h ) w k k 3 + w k 1 k 1 + w k k+1 + w k+1 k+3 C w k k+1 ) = C λ k+1. 17) Statement 14) now follows from 17) and the left inequality in 16). Proof of 14), part III Proceed with h such that We then claim that, t k+3 < x + h < C λ. 18) P X x < h ) C P X x, x + h] ). 19) To verify 19), assume without loss of generality that x h 1. Then recall 1) 11). This yields P X x, x + h] ) log k) 1 log j h ) 1, 11) P X x h, x] ) log m h ) 1 log k) 1, 111) with an appropriate choice of integers m h < k < j h fulfilling the relations log k) λ 1 log j h ) λ 1 h, 11) log m h ) λ 1 log k) λ 1 h. 113) Inequality 19) transforms, through the approximations 11) 111), into log m h ) 1 log k) 1 C log k) 1 log j h ) 1). 114) To verify 114), introduce z 1 := log j h ) 1, z := log k) 1, z 3 := log m h ) 1, p := λ 1. By the mean value theorem we then have for some ξ 1, ξ such that z 3 z z z 1 = zp 3 zp z p zp 1 ξp 1 1 ξ p 1 z 1 ξ 1 z, z ξ z 3,, 33

36 so that, in view of 11) 113), z p 3 zp z p zp 1 C, and conclusion 114) thereby follows. Next, define an integer j k + by the property that By 19) and 1), it then follows that On the other hand, t j 1 < x + h t j ) P X x < h ) CP t k X t j+1 ) = C h t j 1 t k+1 j 1 i=k+1 i C j w i i+1 i=k log k) 1 log j) 1]. 116) C log k + 1)) λ 1 log j 1)) λ 1] C log k) λ 1 log j) λ 1]. 117) To prove that 14) holds under 18) it remains, by 116) and 117), to verify that log k) 1 log j) 1] log k) λ 1 log j) λ 1] λ, 118) or, in other words, that log k) 1 log j) 1] λ 1 + log j) 1] λ 1 log k) 1] λ 1. However, the last inequality is nothing but the convexity property of the function x x λ 1 recall that λ 1 > 1). Proof of 14), part IV Finally, consider h such that Define an integer j < k by the property that C λ x + h, 1 < x h. 119) t j 1 < x h t j+1. 1) 34

37 It then follows that P X x < h ) P t j 1 X ) = w i i+1 i=j Clog j) 1. 11) On the other hand, by the right-hand inequality in 1) and the first inequality in 119), But, since h x t j+1 ) + C λ x). C λ t j+1 i=j+1 i Clog j) λ 1. 1) conclude from 11) and 1) that 14) holds also under 119). Proof of 13), part I Note that E T n r in the left-hand side of statement 13) may, by Theorem 7, be replaced by n E X 1 r X i X 1 ]. r 13) i= Since F is concentrated on a bounded interval separated from, replace 13) by n E X i X 1 ]. r 14) i= Then decompose the expectation into k= n E i= and split each summand into two parts: n E i= n E i= X i X 1 r I { X 1 I k+1 } ], 15) { X i X 1 r I X i I k+1, all 1 i n} ], 16) } { X i X 1 r I {X 1 I k+1 I X i I k+1, some i n} ]. 17) Proof of 13), part II To prove that the terms 16) are summable when r := λn 1), 18) 35