3. Probability inequalities

Transcription

1 3. Probability inequalities 3.. Introduction. Probability inequalities are an important instrument which is commonly used practically in all divisions of the theory of probability inequality is the Chebysjev inequality which estimates the tail probability of a sum of independent random variables. However while being very general this inequality does not provide the optimal order of decreasing an infinity. Among classical probabilities are those by S.N. Bernstein (see [, III, 3, Theorem 8]) and Bennet [2] Hoeffding [3]. The Bernstein inequality is applicable is applicable in case when the probabilities P(X j > x) decrease exponentially as x for all summand forming the sum n j= X j. As to the Bennet-Hoeffding inequality, the latter is intended for bounded random variables. The inequality by Yu.V. Prokhorov [4] is adjacent to the Bennet Hoeffding inequality. That was rather a limited list of probability inequalities to the moment when I started my research in this area. The problem was consisted in finding inequalities which would give adequate result for the distribution having moments only of a finite order. The first inequality of a such type was deduced in my paper [5]. This inequality is incomplete in the sense that it is valid only for deviations of the order larger than n ln n. Never the less this inequality allowed me to obtain the optimal nonuniform bound on CLT for i.i.d. random variables. More sophisticated inequalities were obtained after a few years in the joint paper with my post-graduate student D.Kh. Fuc [6]. The latters are universal since they are formulated in terms of truncated moments and therefore do not demand any moment restrictions. Because of this they are very convenient for a variety of applications. Several of these applications are discussed in [6] and my survey paper [7]. One of possible applications is deducing moment inequalities. For example, the well-known Rosenthal [8] inequality is proved in several lines with the aid of one of the inequalities obtained in [6], (see, in this connection, [9], s well as [7] and [0].) Another application consists in finding sufficient conditions for the strong law of large numbers s well as estimating the rate of convergence for laws of large numbers (see [6] and [7].) In more detail applications of the inequalities obtained in [6] are discussed in the next sections Upper bounds. I mentioned already in the section Large deviations, that in my work [5] a probability inequality was deduced, which allowed to obtain a nonuniform analogue of the Berry Esseen bound with rate of decrease x 3. This is the faithful form of the inequality: F n (x) < n ( F (y) ) { [ ] 2 }( ) x/y m ln y ln(ncm K m ) ncm K m + exp 2n +, () y y m x > 0, y > 0. Here F n (x) is the distribution function of the sum S n = n i= X i of i.i.d. random variables, F (x) is the distribution function of one summand, c m = E X i m <, m > 2, K m = + (m + ) m+2 e m.

2 ( ) 2 m ln y If 2n y < c and F (y) = 0, then by (), ( ) x/y F n (x) < e c+ ncm K m. (2) y m Choosing y small enough with respect to x, one can attain needed rate of decrease in the righthand side of the inequality (2). This fact which used repeatedly in the work [5] predetermined high degree of applicability of the inequality () and its generalizations. The disadvantage of the inequality () is that it gives appropriate bounds only for x large enough. This disadvantage was removed in the investigation joint with D.H. Fuc [6], my post-graduate student from Vietnam, who studied a post-graduate course at Novosibirsk State University in The modified inequalities act already on the whole axis, and, moreover, they have place for differently distributed random variables. Since these inequalities do not require any moment restriction, they are very universal and so have numerous applications. The results, obtained in [6], are presented in my survey paper [7]. Let formulate one of the results, obtained in [6]. To this end we introduce the following notations: A + t (Y ) = k= E {X tk; } 0 < X k y k, B 2 (Y ) = k= E {X 2k; } X k y k, where Y = { } y, y 2,..., y n. In these terms { P(S n x) P(X i > y i ) + exp α 2 x 2 / ( 2e t B 2 (Y ) )} ( βx/y + A + t (Y )/(βxy )) t (3) provided that t 2, EX i = 0, i =,..., n, β = t/(t + 2) and α = β. Now we turn our attention to some applications, derived in [6]. Applying Theorem 2 from [6], I showed in the work [], that for n and x > cn, if EX = 0 and for x P(S n x) n( F (x)), F (x) x α h(x), where α >. Analogous result was obtained by S.G. Tkachuk [2] provided that F (x) was attracted to a stable law with the exponent α > 2. The result of Tkachuk was reproved by R. Doney [3]. In the work of Doney [4] the local limit theorem is proved under conditions of Tkachuk with the help of Corollary.5 from [6]. In [5] A. Spataru applied mentioned earlier Theorem 2 from [7] for deducing the asymptotics lim ε 0 log ε n n P( S n εn) = α α, where α > is the exponent of the stable law, F (x) is attracted to which. In the tracks of [5 7, ] A.A. Borovkov [6] undertook an extensive investigation, the result of which was the inequality of the form P(S n > x, max k n X k < y) c[nv (y)] x/y. (4) 2

3 Here S n = max S k, V (y) is a regularly varying function, which majorizes F (y), c is a <k n constant, which is not calculated explicitly. Comparing the inequalities (2) and (4), we can see, that the latter is a generalization of the inequality (2) to S n. Now we return to the inequality (3). Note that the coefficient c t := 2 α2 e t in the exponent is equal to 2(t + 2) 2 e t. Obviously, c t decreases fastly, when t grows. Even for t = 2 the value of c t is sufficiently small, precisely, c t Naturally, the idea to deduce a more perfect inequality arises, in order that the coefficient x under 2 will be close to. This is realized in my work [0] by means of introducing an B 2 (Y ) 2 additional parameter. The improved inequality looks as follows: P(S n B n x) < { } βx P(X j y j ) + exp y α2 x 2 + 2e c B 2 (Y ) { βx } ( [( t ) t ] A + ) βx/y + exp + t (Y ). (5) y c βxy t j= Here t, 0 < α <, β = α, 0 < c t, y max y j. It is assumed that j n EX j = 0, j =, n. As compared to inequality (3), inequality (5) contains the additional parameter c. Put x = y = B n u in the inequality (5), where u is some fixed number, Bn 2 = n σ2 i, σj 2 = EXj 2, and j =, n. Let the Liapounoff ratio L n = E X j 3 /Bn 3 tends to zero as n. Then the first and third summands in (5) tend to 0 for t = 3 and every fixed c and β. Choosing α to be closed to, and c closed to 0, we can attain that α 2 /e c < ε, where ε is arbitrarily small. In turn this means that On the other hand, by virtue of the central limit theorem, lim sup P(S n > B n u) < e u2 /2. (6) n lim P(S n > B n u) = Φ(u), n where Φ(u) is the standard normal law. It is well known that for u > Φ(u) < 2π e u2 /2. On the other hand, as it is shown in [7, p. 63], ( u + )( ) Φ(u) > e u2 /2. u 2π 3

4 u Thus, the bound (6) is optimal up to a factor not exceeding. 2π (u 2 +) Note that the starting point for all inequalities stated above is the bound of the form P(S n > x) < e hx where g j (h) = Ee hx j. If X j are normally distributed, n j= P(S n > x) < exp{b 2 nh 2 /2 hx}. g j (h), (7) Putting x = B n u and minimizing the right-hand side in h, we arrive at the bound P(S n > B n u) < e u2 2. Mention now the inequality of another kind, which does not contain the exponent of the form e cx2 in contrast to (3) and (5): P(S n x) ( ) B P(X i y) + 2e x/y 2 x/y n. (8) xy This inequality was obtained in my joint with I.F. Pinelis paper [9]. In particular, for y = x/2, ( P(S n x) P X i > x ) + 4e 2 B4 n 2 x. (9) 4 We can see that the last inequality can give nontrivial bound for the probability P(S n x) only for x > 4 /4 e /2 B n 2.33B n, while application of the Chebyshev inequality leads to the nontrivial estimate for x > B n. Moreover, the Chebyshev inequality is more precise than (9) for B n < x < 2eB n. Thus, the inequality (9) is more precise with respect to the Chebyshev one only if the ratio x/b n is sufficiently large. Up to now the case in point was the bounds in terms of power moments. Consider the more general case. To this end we assume that the moments b gi := E{e g(x i), X i 0} are finite, where the function g(x) increases and lim x g(x)/ ln x =. Denote B gn = n i= b gi. Let g(x) be differentiable, where (a) g (x) 0 for x ; (b) there exists δ > 0 such, that g (x) > δ/x. The condition (a) means that e g(u) decreases slower than e u. Respectively, it follows from the condition (b) that e g(u) decreases faster than u 2. Put S(x) = e g(x) g (x)x 2. Let γ, {γ i } 3, β be positive constants such that 3 γ i =, β = γ /2 γ 2 /δ, γ <, and let a be the solution of the equation (x + )/x = e x. Denote by S (x) the function inverse to S(x). In these terms the following inequality holds P(S n x) exp{γ 3 /γ γ βa 2 x 2 /2(a + )B 2 n} + exp{γ 3 /γ βax/s (γ 2 ax/e a B gn )} +(γ/γ 3 )B B/y gn exp{γ 3 /γ g(γx)β/γ} + 4 P(X i γx). (0) i=

5 This inequality was gotten in my joint with Sakojan work [8]. It is formulated in [7] as well. Inequality (0) is similar in form to inequality (3). However, in contrast to the latter it contains four summands, in the right-hand side rather than three ones. Moreover, the second term in the right-hand side has no analogues in the inequality (3). In [8] an example is constructed, which shows, that it is not possible to omit this summand Lower bounds. In the survey [7], already mentioned, the following inequality obtained for x 2B n, P(S n x) P(X j 2x). () 2 In particular, it follows from (), that it is not possible to get rid of the first summand in the right-hand side of the inequality (8). In the inequality () the boundedness of summands is not required unlike the well-known Kolmogorov bound. In my work [7], under assumption that the random variables X j have third moments, the bound of the following form was obtained: j= P(S n > x) ( Φ(x))e c 2L n x 3 ( c 3 L n x), (2) which acts for < x < c /L n. Here L n is the Lyapounoff ratio, c i, i =, 3, are some constants which are written out in explicit form. It is obvious that the bound (2) is more general as compared to the Kolmogorov one. Moreover, as it is shown in (2), it is considerably more precise as well. In a sense of precision the inequality (2) is closed to the results of Feller and Lenart (see [9, 20]) which have place only for the bounded summands. Estimates () and (2) complement each other, but they do not cover all possible cases. Firstly, for L n c the interval in which the bound (2) acts, is degenerated. Secondly, it can happen that P(X j > 2B n ) = 0 for all j, and then the bound () turns out to be trivial. In fact, let, for instance, X j be i.i.d. and P(X = ) = P(X = ) = p, P(X = 0) = 2p. In this case, obviously, L n = / 2pn. Therefore, if np < 2/c 2, the bound (2) is inapplicable at all, and the bound () gives the trivial result P(S n x) 0 for np > /32. Lower bounds for P(S n > x) are deduced in my work [2], which are more precise than the bounds () and (2), namely, in the situation, when the Lyapounoff ratio is large, and the summands X j either are bounded or their distributions have quickly decreasing tails. To formulate these bounds, we need some additional notations. Let ξ(p) be the number of successes in nonhomogeneous Bernoulli trials with probabilities of successes: p, p 2,..., p n, p = (p, p 2,..., p n ). Introduce the notation P (p; x) = P(ξ(p) > x). If to assume that X j are symmetrically distributed, we have P(S n > x) ( 2 P p(a); x ) n ( p j (A)), (3) A ( p where A is any positive number, p(a) = p p,..., n p n ), p j = p j (A) = P(X j A). j= 5

6 Using the inequality P( X s > 2x) 2P( X > x), where X s is the symmetrization of X, we get the bound Incidentally, the bound P(S n > x) 2 P ( p(a); 2x A ) n ( p j (A)). P(η n m 0 ) > 3 is deduced in the work [2], where η n is the number of successes in n Bernoulli trials with a constant probability of success p <, where m 2 0 is entire part of np. In English translation is changed by. In this form the bound is unimprovable. 3 2 Return to the inequality (). As we can see, it presuppose the finiteness of the variance. Meanwhile in [7] the inequality P(S n x) j= [ ] P(X j 2x) P(Sn j x) P(max X i > x) i j was deduced as an intermediate result, where S j n = S n X j. Here there are no any moment restrictions. Evidently, and P(S j n x) > B2 n x 2 P(max X i > x) < B2 n i j x. 2 Substituting these bounds in (4), we obtain (). Thus, B n appears in the resultant formulation only on the final step. The presence of B n in () makes the latter less general but more simple and visual, giving a classical form for it. One must take into account also that none bounds of such type existed at that time. Granting that we can write (4) in the form P(S n > x) > P(max i j X i > x) < i= P(X i > x), i= [ P(X i > 2x) inf P(Sn j > x) j i= ] P(X i > x). The inequality of such type was obtained for random variables with values in Banach space in my work [22]. In conformity to usual random variables it looks as follows: P( S n > x) > i= [ P( X i > ax) P( Sn i > (a )x) 6 j= (4) ] P( X j > ax). (5)

7 The modulus in this inequality appears since it defines the norm in the space R. A one-sided inequality of the type (5) for identically distributed random variables is obtained in the work of A.A. Borovkov [6], namely, [ P(S n > x) > np(x > αx) P(S n > ( α)x) n 2 ] P(X > αx), where X coincides with X in distribution. This inequality is proved in the same method as () Some applications. (A) Moment inequalities. One of the numerous applications of the probabilistic inequalities considered above is derivating the bounds for the moments E S n t and half-moments E{Sn; t S n > 0}, E{ S n t ; S n < 0}. This approach was first applied in my joint with Pinelis work [9]. The matter is in estimating the integral 0 x t P(S n > x) dx. Various bounds for P(S n > x) were obtained in [6] by then. In [9] the following two-sided bound is used, P( S n x) { x P( X i y) + 2 exp y x ( xy )} y ln +. (6) Bn 2 Let y = x/c, c > t/2. Multiplying both parts of the inequality by tx t and integrating in x from 0 up to, we get that E S n t c t A t + 2te c 0 ( x x t 2 cdx, + ) cbn 2 where A t = n E X k t. In the right-hand side the integral is equal to 2 c t/2 B(t/2, c t/2)b t n, where B(u, v) is the Euler beta-function. Thus, E S n t c t A t + tc t/2 e c B(t/2, c t/2)b t n. (7) For the first time the inequality of the type (7) was obtained by Rosenthal [8] with the help of complicated considerations. The proof, stated above, is much simpler. Notice that in [8] the Rosenthal inequality is formulated in a form which is rather different from (7), namely: for t > 2 2 t max{a/t t, B n } < (E S t ) /t < K t max{a /t t, B n }, where K t < 2 t. In my work [0] the following bound is deduced with the help of the inequality (5) in the case t = 2: [ (( t ) t ) E{Sn; t S 0} e 2 2 t+ β + A + t + c π ( e c ) ] t/2 tµt B t 2 α 2 n, (8) where µ t is the absolute moment of the order t of the standard normal random variable, 0 < α <, β = α, 0 < c t. 7

8 We can see that the right-hand side of the inequality (8) depends on the free parameters α and c. Varying them, we can get the optimal bounds depending on the relation between A + t and Bn. t Notice that alternative methods (see [23 25]) do not give possibility to deduce inequalities of the type (8) for half-moments. (B) The weak law of large numbers. The probability inequalities which discussed above are very convenient for estimating the rate of convergence in the law of large numbers. If to assume the independent random variables X j are identically distributed and EX j = 0, EXj 2 = σ 2, then by virtue of the inequality (6) ( P( S n > nε) < np X > nε ) 2 ( + 8e 2 Bn ) 4, (9) nε where X = X j in distribution. If nε > 2σ n, i.e. n > 4σ 2 /ε 2, then according to the lower bound (6) P( S n > nε) > np( X > 2nε). (20) Of course, the last inequality is trivial if the random variables are bounded, say, X < M, and 2nε < M. If the σ 2 =, one can use other inequalities obtained in [6, 7], [0]. Certainly, one can write bounds, similar to (9), (20), and in the case, when the random variables X j are not identically distributed. (C) The strong law of large numbers. Let X, X 2,..., X n be independent random variables. Notice, first of all, that, without loss of generality, one can restrict himself to symmetrically distributed X j (see, for instance, [26]), while studying applicability of the strong law of large numbers. Let I r = {n : 2 r + n 2 r+ }, χ r = X 2 r n. n I r Yu.V. Prokhorov [27] proved that the strong law of large numbers holds if and only if for all positive numbers ε P(χ r ε) <. (2) r=0 Thus, the problem of finding necessary and sufficient conditions for the strong law of large numbers amounts to obtaining upper and lower bounds for the probability of large deviations of sums n I r X n. Using probability inequalities, we can easily formulate different versions of sufficient or necessary conditions which are not necessary and sufficient ones simultaneously. To this end let us put K(t, δ, r) = 2 n u 2δ u t df k (u), H(δ, r) = 2 2r σ 2 k, where summation is over all k I r. It is proved in my joint with Dao Ha Fuc paper [6], that if there exists a sequence of positive numbers δ r such that for all positive ε the following 8

9 conditions are fulfilled simultaneously: and r= r= k I r P(X k > 2 r δ r ) <, (εδ t r /K(t, δ r, r) + ) ε/δ r <, t 2, exp{ ε/h(δ r, r)} <, r= then the strong law of large numbers holds. Numerous corollaries from this result are given in [5] and [6]. For instance, if for t 2 and β > the following conditions are fulfilled: and P(X k > kε) <, ε > 0 k= (K t,r ) β <, r= where K t,r = 2 rt E X k t, then the strong law of large numbers holds. Necessary and sufficient conditions for the strong law of large numbers in terms of individual summands are found in my work [29]. These conditions are verified easier than the condition of Prokhorov(2). To formulate them, we need the notations f n (h, ε) = e hu df n (u), u ne ψ r (h, ε) = n I r f n (h, ε) h f n(h, ε). Define h r (ε) as the solution of the equation ψ r (h, ε) = nε if sup ψ r (h, ε) εn; otherwise we h put h r (ε) =. It is proved in [29] that the strong law of large numbers holds if and only if (i) n= P(X n > nε) <, (ii) r= e εh r(ε)n r <, ε > 0. At first sight the condition (ii) seems not so effective. Nevertheless, there is easy to deduce from it the following criteria belonging to Prokhorov [28]: the condition where H r = 2 2r k I r σ 2 k exp{ ε/h r } <, ε > 0, r=, is sufficient for the strong law of large numbers if X n = o(n/ ln ln n). 9

10 References [] Petrov V.V.. Limit theorems of probability theory// Clarendon, 995, 300 p. [2] Bennet G. Probability inequalities for the sum of independent random variables// J. Amer. Statist. Assoc., 962, 57, No 297, [3] Hoeffding W. Probability inequalities for sums of bounded random variables// J. Amer. Statist. Assoc., 963, 58, No 30, [4] Prokhorov Yu.V. One extremal problem of the theory of probability// Theory Probab. Appl., 959, 4, No 2, Original Russian Teor. Verojatn. i Primen., 959, 4, No 2, [5] Nagaev S.V. Some limit theorems for large deviations// Theory Probab. Appl., 965, 0, No 2, PDF Original Russian Text@ Teor. Verojatn. i Primen., 965, 0, No 2, [6] Fuk Dao Kha, Nagaev S.V. Probability inequalities for sums of independent random variables// Theory Probab. Appl., 97, 6, No 4, (letter to editor: 976, 2, No 4, 875). PDF Original Russian Text@ Teor. Verojatn. i Primen., 97, 6, No 4, (letter to editor: 976, 2, No 4, 896.) [7] Nagaev S.V. Large deviations of sums of independent random variables// Ann. Prob., 979, 7, No 5, PDF [8] Rosenthal H.P. On the subspaces of L p (p > 2) spanned by sequences of independent random variables// Israel J. Math., 970, 8, [9] Nagaev S.V., Pinelis I.F. Some inequalities for the distributions of sums of independent random variables// Probab. Appl., 977, , 22, No 2, PDF Original Russian Text@ Teor. Verojatn. i Primen., 977, 22, No 2, [0] Nagaev S.V. Some refinements of probabilistic and moment inequalities// Theory Probab. Appl., 997, 42, No 4, PDF Original Russian Text@ Teor. Verojatn. i Primen., 997, 42, No 4, [] Nagaev S.V. On asymptotic behaviour of probabilities of one-sided large deviations// Probab. Appl., 98, 26, No 2, PDF Original Russian Text@Teor. Verojatn. i Primen., 98, 26, No 2, [2] Tkachuk S.G. Limit theorems for sums of independent random variables, belonging to the attraction domain of a stable law// Ph.D., Tashkent, 977. (In Russian.) [3] Doney R.A. One-sided large deviation theorems in the case of infinite mean// Research report, Manchester Centre for Statistical Science, 995. [4] Doney R.A. One-sided local large deviation and renewal theorems in the case of infinite mean// Probab. Theory Relat. Fields, 997, 07,

11 [5] Spataru A. Precise Asymptotics in Spitzer s Law of Large Numbers// J. of Theor.Probab., 999, 2, No 3, [6] Borovkov A.A. Estimates for the distribution of sums and maxima of sums of random variables mihout the Cramer condition // Sibirsk.Mat.Zh., 2000, 4, No 5, (In Russian). [7] Nagaev S.V. Lower bounds on large deviation probabilities for sums of independent random variables// Theory Probab. Appl., 200, 46, No, PDF Original Russian Text@ Teor. Verojatn. i Primen., 200, 46, No, [8] Nagaev S.V., S.K. Sakojan. On a bound for a probability of large deviations// In: Limit Theorems and Mathematical Statistics (In Russian), FAN, Tashkent, 976, PDF [9] Feller W. Generalization of a probability limit theorem of Cramer// Trans. Amer. math.soc., 943, 54, N 3, [20] Lenart C. On certain theorems of Berry and a limit theorem of Feller// Mat.Časopis, 968, 8, No, [2] Nagaev S.V. Lower bounds on probabilities of large deviations for sums of independent random variables// Theory Probab. Appl., 200, 46, No 4, PDF Teor.Verojatn. i Primen., 200, 46, No 4, [22] Nagaev S.V. Probabilities of large deviations in Banach spaces// Math. Notes, 983, 34, No 2, PDF Original Russian Text@ Mat. Zametki, 983, 34, No 2, [23] Pinelis I.F. Optimum bounds for the distributions of martingales in Banach spaces// Ann. Probab., 994, 7, [24] Johnson W.B., Schechtman G., and Zinn J. Best possible constants in moment inequalities for linear combinations of independent and exchangeable variables// Ann. Probab., 99, 3, [25] Ibragimov R., Sharakhmetov Sh. The sharp constant in the Rosenthal inequality for random variables with zero mean// Theory Probab. Appl., 200, 46, No, Original Russian Teor.Verojatn. i Primen., 200, 46, No, [26] Prokhorov Yu.V. Strong stability of sums and infinitely divisible distributions.// Theor. Probab. Appl., 958, 3, [27] Prokhorov Yu.V. On the strong law of large numbers// Izv. Akad. Nauk SSSR, Ser. Mat. (in Russian), 950, 4, [28] Prokhorov Yu.V. Some remarks on the strong law of large numbers// Theor. Probab. Appl., 959, 4,

12 [29] Nagaev S.V. On sufficient and necessary conditions for the strong law of large numbers// Theory Probab. Appl., 972, 7, No 4, PDF Original Russian Teor. Verojatn. i Primen., 972, 7, No 4,