Sharp large deviation results for sums of independent random variables

Transcription

1 Sharp large deviation results for sums of independent random variables Xiequan Fan, Ion Grama, Quansheng Liu To cite this version: Xiequan Fan, Ion Grama, Quansheng Liu. Sharp large deviation results for sums of independent random variables. Science China Mathematics, 15, 58 9), pp <1.17/s >. <hal > HAL Id: hal Submitted on 8 Aug 15 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2 Sharp large deviation results for sums of independent random variables Xiequan Fan a,b,, Ion Grama c, Quansheng Liu c,d a Regularity Team, INRIA Saclay, Palaiseau, 911, France b MAS Laboratory, Ecole Centrale Paris, 995 Châtenay-Malabry, France c LMBA, UMR 65, Univ. Bretagne-Sud, Vannes, 56, France d College of Mathematics and Computing Science, Changsha University of Science and Technology, Changsha, 414, China Abstract We show sharp bounds for probabilities of large deviations for sums of independent random variables satisfying Bernstein s condition. One such bound is very close to the tail of the standard Gaussian law in certain case; other bounds improve the inequalities of Bennett and Hoeffding by adding missing factors in the spirit of Talagrand 1995). We also complete Talagrand s inequality by giving a lower bound of the same form, leading to an equality. As a consequence, we obtain large deviation expansions similar to those of Cramér 1938), Bahadur-Rao 196) and Sakhanenko 1991). We also show that our bound can be used to improve a recent inequality of Pinelis 14). Keywords: Bernstein s inequality, sharp large deviations, Cramér large deviations, expansion of Bahadur-Rao, sums of independent random variables, Bennett s inequality, Hoeffding s inequality MSC: primary 6G5; 6F1; secondary 6E15, 6F5 1. Introduction Let ξ 1,..., ξ n be a finite sequence of independent centered random variables r.v. s). Denote by S n = ξ i and = E[ξi ]. 1) Starting from the seminal work of Cramér [13] and Bernstein [1], the estimation of the tail probabilities P S n > x), for large x >, has attracted much attention. Various precise inequalities and asymptotic results have been established by Hoeffding [5], Nagaev [3], Saulis and Statulevicius [41], Chaganty and Sethuraman [1] and Petrov [35] under different backgrounds. Assume that ξ i ),...,n satisfies Bernstein s condition E[ξ k i ] 1 k!εk E[ξ i ], for k 3 and i = 1,..., n, ) for some constant ε >. By employing the exponential Markov inequality and an upper bound for the moment generating function E[e λξ i ], Bernstein [1] see also Bennett [3]) has obtained the following inequalities: for all x, PS n > x) inf λ E[eλSn x) ] 3) Corresponding author. fanxiequan@hotmail.com X. Fan), quansheng.liu@univ-ubs.fr Q. Liu). ion.grama@univ-ubs.fr I. Grama), Preprint submitted to Elsevier August 8, 15

3 where x = B x, ε ) := exp { x exp 1 + xε/) } { x 4) }, 5) x xε/ ; 6) see also van de Geer and Lederer [47] with a new method based on Bernstein-Orlicz norm and Rio [4]. Some extensions of the inequalities of Bernstein and Bennett can be found in van de Geer [46] and de la Peña [14] for martingales; see also Rio [38, 39] and Bousquet [11] for the empirical processes with r.v. s bounded from above. Since lim ε/ PS n > x) = 1 Φx) and lim ε/ B ) x, ε = e x /, where Φx) = 1 x π e t dt is the standard normal distribution function, the central limit theorem CLT) suggests that Bennett s inequality 4) can be substantially refined by adding the factor { x Mx) = ) 1 Φx) exp where πmx) is known as Mill s ratio. It is known that Mx) is of order 1/x as x. To recover a factor of order 1/x as x a lot of effort has been made. Certain factors of order 1/x have been recovered by using the following inequality: for some α > 1, PS n x) inf t<x E }, [ Sn t) + ) α x t) + ) α where x + = max{x, }; see Eaton [17], Bentkus [4], Pinelis [36] and Bentkus et al. [7]. Some bounds on tail probabilities of type ) PS n x) C 1 Φx), 7) where C > 1 is an absolute constant, are obtained for sums of weighted Rademacher r.v. s; see Bentkus [4]. In particular, Bentkus and Dzindzalieta [6] proved that C = 1 41 Φ )) is sharp in 7). When the summands ξ i are bounded from above, results of such type have been obtained by Talagrand [45], Bentkus [5] and Pinelis [37]. Using the conjugate measure technique, Talagrand cf. Theorems 1.1 and 3.3 of [45]) proved that if the r.v. s satisfy ξ i 1 and ξ i b for a constant b > and all i = 1,..., n, then there exists an universal constant K such that, for all x Kb, PS n > x) inf λ E[eλS n x) ] Mx) + K b ) 8) H n x, ) Mx) + K b ), 9) ], where H n x, ) = { ) x+ ) } n n x n+ n. x + n x

4 Since Mx) = O 1 x), x, equality 9) improves on Hoeffding s bound Hn x, ) cf..8) of [5]) by adding a missing factor F 1 x, b ) = Mx) + K b of order 1 x for the range x Kb. Other improvements on Hoeffding s bound can be found in Bentkus [5] and Pinelis [36]. Bentkus s inequality [5] is much better than 9) in the sense that it recovers a factor of order 1 x for all x instead of the range x Kb, and do not assume that ξ i s have moments of order larger than ; see also Pinelis [37] for a similar improvement on Bennett-Hoeffding s bound. The scope of this paper is to give several improvements on Bernstein s inequalities 3), 5) and Bennett s inequality 4) for sums of non-bounded r.v. s instead of sums of bounded from above) r.v. s, which are considered in Talagrand [45], Bentkus [5] and Pinelis [36]. Moreover, some tight lower bounds are also given, which were not considered by Talagrand [45], Bentkus [5] and Pinelis [36]. In particular, we improve Talagrand s inequality to an equality, which will imply simple large deviation expansions. We also show that our bound can be used to improve a recent upper bound on tail probabilities due to Pinelis [36]. Our approach is based on the conjugate distribution technique due to Cramér, which becomes a standard for obtaining sharp large deviation expansions. We refine the technique inspired by Talagrand [45] and Grama and Haeusler [3] see also [19, ]), and derive sharp bounds for the cumulant function to obtain precise upper bounds on tail probabilities under Bernstein s condition. As to the potential applications of our results in statistics, we refer to Fu, Li and Zhao [1] for large sample estimation and Joutard [8, 9] for nonparametric estimation. In these papers, many interesting Bahadur-Rao type large deviation expansions have been established. Our result leads to simple large deviation expansions which are similar but simpler) to those of Cramér 1938), Bahadur-Rao 196) and Sakhanenko 1991). For other important applications, we refer to Shao [44] and Jing, Shao and Wang [6], where the authors have established the Cramér type self-normalized large deviations for normalized x = on 1/6 ); see also Jing, Liang and Zhou [7]. From the proofs of theorems in [44, 6, 7], we find that the self-normalized large deviations are closely related to the large deviations for sums of bounded from above r.v. s cf. [18]). Our results may help extend the Cramér type self-normalized large deviations to a larger range. The paper is organized as follows. In Section, we present our main results. In Section 3, some comparisons are given. In Section 4, we state some auxiliary results to be used in the proofs of theorems. Sections 5-7 are devoted to the proofs of main results.. Main results All over the paper ξ 1,..., ξ n is a finite sequence of independent real-valued r.v. s with E[ξ i ] = and satisfying Bernstein s condition ), S n and are defined by 1). We use the notations a b = min{a, b}, a b = max{a, b} and a + = a. Throughout this paper, C stands for an absolute constant with possibly different values in different places. Our first result is the following large deviation inequality valid for all x. Theorem.1. For any δ, 1] and x, PS n > x) ) [ 1 Φ x) 1 + C δ 1 + x) ε ], 1) where x x = δ)xε/ and C δ is a constant only depending on δ. In particular, if x = o/ε), ε/, then )[ ] PS n > x) 1 Φ x) 1 + o1). 11) 3

5 The interesting feature of the bound 1) is that it decays exponentially to and also recovers closely the shape of the standard normal tail 1 Φx) when r = ε becomes small, which is not the case of Bennett s bound Bx, ε ) and Berry-Essen s bound PS n > x) 1 Φ x) + C ε. Our result can be compared with Cramér s large deviation result in the i.i.d. case cf. 34)). With respect to Cramér s result, the advantage of 1) is that it is valid for all x. Notice that Theorem.1 improves Bennett s bound only for moderate x. A further significant improvement of Bennett s inequality 4) for all x is given by the following theorem: We replace Bennett s bound B ) x, ε by the following smaller one: { )} B n x, ε ) = B x, ε ) x exp nψ n, 1) 1 + xε/ where ψt) = t log1 + t) is a nonnegative convex function in t. Theorem.. For all x, where PS n > x) B n x, ε ) F x, ε ) 13) B n x, ε ), 14) F x, ε ) = Mx) R xε/) ε ) 1 15) and Rt) = { 1 t+6t ) 3 1 3t) 3/ 1 t) 7, if t < 1 3,, if t 1 3, 16) is an increasing function. Moreover, for all x α ε with α < 1 3, it holds Rxε/) Rα). If α =.1, then 7.99Rα) To highlight the improvement of Theorem. over Bennett s bound, we note that B n x, ε ) Bx, ε ) and, in the i.i.d. case or, more generally when ε = c n, for some constant c > ), nx, ε ) B n = B nx, ε ) exp { c x n}, 17) where c x >, x >, does not depend on n. Thus Bennett s bound is strengthened by adding a factor exp { c x n}, n, which is similar to Hoeffding s improvement on Bennett s bound for sums of bounded r.v. s [5]. The second improvement in the right-hand side of 13) comes from the missing factor F x, ε ), which is of order Mx)[1 + o1)] for moderate values of x satisfying x = o ε ), ε. This improvement is similar to Talagrand s refinement on Hoeffding s upper bound H n x, ) by the factor F 1 x, b/); see 9). The numerical values of the missing factor F x, ε ) are displayed in Figure 1. Our numerical results confirm that the bound B n x, ε )F x, ε ) in 13) is better than Bennett s bound Bx, ε ) for all x. For the convenience of the reader, we display the ratios of B nx, r)f x, r) to Bx, r) in Figure for various r = 1 n. The following corollary improves inequality 1) of Theorem.1 in the range x α ε with α < 1 3. It corresponds to taking δ = in the definition 11) of x. 4

6 The missing factor F x, r) F x, r) r =.1 r =.5 r =.5 r = x Figure 1: The missing factor F x, r) is displayed as a function of x for various values of r = ε. Ratio of bound 13) to Bx, r) Ratio r =.1 r =.1 r =.1 r =.5 r = x Figure : Ratio of B n x, r) F x, r) to Bx, r) as a function of x for various values of r = ε = 1 n. 5

7 Corollary.1. For all x α ε with α < 1 3, ) [ PS n > x) 1 Φ x) Rα) 1 + x) ε ], 18) where x is defined in 6) and Rt) by 16). In particular, for all x = o ε ), ε, )[ ] PS n > x) 1 Φ x) 1 + o1) = B x, ε ) [ ] M x) 1 + o1). 19) The advantage of Corollary.1 is that in the normal distribution function Φx) we have the expression x instead of the smaller term x figuring in Theorem.1, which represents a significant improvement. Notice that inequality 19) improves Bennett s bound B ) x, ε by the missing factor M x)[1 + o1)] for all x = o ) ε. For the lower bound of tail probabilities PS n > x), we have the following result, which is a complement of Corollary.1. Theorem.3. For all x α ε with α 1 where ˇx = all x = o ε ), λ 1 λε) 3 with λ = PS n > x) 9.6, x/, and c α = 67.38R xε/ ) [ 1 Φ ˇx) 1 c α 1 + ˇx) ε ], α α ε, )[ ] PS n > x) 1 Φ ˇx) 1 o1). ) is a bounded function. Moreover, for Combining Corollary.1 and Theorem.3, we obtain, for all x.1 ε, PS n > x) = 1 Φ x1 + θ 1 c 1 x ε ) )[ ] ) 1 + θ c 1 + x) ε, ) where c 1, c > are absolute constants and θ 1, θ 1. This result can be found in Sakhanenko [43] but in a more narrow zone. Some earlier lower bounds on tail probabilities, based on Cramér large deviations, can be found in Arkhangelskii [1], Nagaev [33] and Rozovky [3]. In particular, Nagaev established the following lower bound PS n > x) ) 1 Φx) e c1x3 ε 1 c 1 + x) ε ) for some explicit constants c 1, c and all x 1 5 ε. For more general results, we refer to Theorem 3.1 of Saulis and Statulevicius [41]. In the following theorem, we obtain a one term sharp large deviation expansion similar to Cramér [13], Bahadur-Rao [], Saulis and Statulevicius [41] and Sakhanenko [43]. Theorem.4. For all x < 1 1 ε, PS n > x) = inf λ E[eλSn x) ] F 3 x, ε 1) ), ) where F 3 x, ε ) = Mx) θr 4xε/) ε, 3) 6

8 θ 1 and Rt) is defined by 16). Moreover, inf λ E[e λsn x) ] Bx, ε ). In particular, in the i.i.d. case, we have the following non-uniform Berry-Esseen type bound: for all x = o n), PS n > x) Mx) inf λ E[eλS n x) ] C Bx, ε ). 4) n Theorem.4 holds also for ξ i s bounded from above. In this case the term 7.99 θr 4xε/) can be significantly refined; see [18]. In particular, if ξ i ε, then 7.99 θr 4xε/) can be improved to 3.8. However, under the stated condition of Theorem.4, the term 7.99 θr 4xε/) cannot be improved significantly. When Bernstein s condition fails, we refer to Theorem 3.1 of Saulis and Statulevicius [41], where explicit and asymptotic expansions have been established via the Cramér series cf. Petrov [34] for details). When the Bernstein condition holds, their result reduces to the result of Cramér [13]. However, they gave an explicit information on the term corresponding to our term 7.99 θr 4xε/). Equality ) shows that inf λ E[e λsn x) ] is the best possible exponentially decreasing rate on tail probabilities. It reveals the missing factor F 3 in Bernstein s bound 3) and thus in many other classical bounds such as Hoeffding, Bennett and Bernstein). Since θ 1, equality ) completes Talagrand s upper bound 8) by giving a sharp lower bound. If ξ i are bounded from above ξ i 1, it holds that inf λ E[e λsn x) ] H n x, ) cf. [5]). Therefore ) implies Talagrand s inequality 9). A precise large deviation expansion, as sharp as ), can be found in Sakhanenko [43] see also Györfi, Harremöes and Tusnády [4]). In his paper, Sakhanenko proved an equality similar to ) in a more narrow range x 1 where ε, ) PSn ε > x) 1 Φt x ) C t x = e t ) ln inf λ E[eλSn x) ] x /, 5) is a value depending on the distribution of S n and satisfying t x x = Ox ε ), ε, for moderate x s. It is worth noting that from Sakhanenko s result, we find that the inequalities 4) and 7) hold also if Mx) is replaced by Mt x ). Using the two sided bound 1 1 Mt), t, 6) π1 + t) π1 + t) and Mt) 1 π1 + t), t see p. 17 in Itō and MacKean [] or Talagrand [45]), equality ) implies that the relative errors between PS n > x) and Mx) inf λ E[e λsn x) ] converges to uniformly in the range x = o ) ε as ε, i.e. ) PS n > x) = Mx) inf λ E[eλS n x) ] 1 + o1). 7) Expansion 7) extends the following Cramér large deviation expansion: for x = o ) 3 ε as ε, )[ ] PS n > x) = 1 Φ x) 1 + o1). 8) To have an idea of the precision of expansion 7), we plot the ratio Ratiox, n) = PS n x n) Mx) inf λ E[e λs n x n) ] in Figure 3 for the case of sums of Rademacher r.v. s Pξ i = 1) = Pξ i = 1) = 1. From these plots we see that the error in 7) becomes smaller as n increases. 7

9 n = 49 n = 1 Ratio Ratiox, n) Ratio Ratiox, n) x x n = 1 Ratio Ratiox, n) x Figure 3: The ratio Ratiox, n) = r.v. s. PS n x n) is displayed as a function of x for various n for sums of Rademacher Mx) inf λ E[e λsn x n) ] 3. Some comparisons 3.1. Comparison with a recent inequality of Pinelis In this subsection, we show that Theorem.4 can be used to improve a recent upper bound on tail probabilities due to Pinelis [37]. For simplicity of notations, we assume that ξ i 1 and only consider the i.i.d. case. For other cases, the argument is similar. Let us recall the notations of Pinelis. Denote by Γ a the normal r.v. with mean and variance a >, and Π θ the Poisson r.v. with parameter θ >. Let also Denote by δ = Π θ Π θ θ. n E[ξ+ i )3 ]. 9) Then it is obvious that δ, 1). Pinelis cf. Corollary. of [37]) proved that: for all y, PS n > y) e3 9 PLC Γ 1 δ) + Π δ > y), 3) where, for any r.v. ζ, the function P LC ζ > y) denotes the least log-concave majorant of the tail function Pζ > y). So that P LC ζ > y) Pζ > y). By the remark of Pinelis, inequality 3) refines the Bennet-Hoeffding inequality by adding a factor of order 1 x in certain range. By Theorem.4 and some simple calculations, we find that, for all y = on), P LC Γ 1 δ) + Π δ > y) 8

10 where By the inequality PΓ 1 δ) + Π δ > y) inf λ E[eλΓ 1 δ) + Π y) δ ] = inf λ E[e λy+fλ,δ,) ] My/) C n ) fλ, δ, ) = λ 1 δ) + e λ 1 λ)δ. e x 1 x x + x + ) k My/) C ), 31) n cf. proof of Corollary 3 in Rio [4]) and the fact that log1 + t) is concave in t, it follows that, for any λ >, log E[e λs n ] = log E[e λξ i ] log 1 + λ λ k ) E[ξ i ] + k! E[ξ+ i )k ] log 1 + λ E[ξ i ] + n log λ n = n log k=3 k=3 λ k ) k! E[ξ+ i )3 ] k=3 λ k ) ) E[ξ i ] + k! E[ξ+ i )3 ] k=3 ) n log λ λ k n + k! δ) k= n ) fλ, δ, ). By the last line, Theorem.4 implies that, for all y = on), PS n > y) inf λ E[e λy+n log1+ 1 n fλ,δ,)) ] 1 + o1) ) My/) + C n ) inf λ E[e λy+n log1+ 1 n fλ,δ,)) ] My/) C ) n. 3) Note that n log n fλ, δ, )) fλ, δ, ). By the inequalities 3), 31) and 3), we find that 3) not only refines Pinelis constant e ) to 1 + o1) for large n, but also gives an exponential bound sharper than that of Pinelis. 3.. Comparison with the expansions of Cramér and Bahadur-Rao Notice that the expression inf λ E[e λsn x) ] can be rewritten in the form exp{ nλ n x n )}, where Λ nx) = sup λ {λx 1 n log E[eλS n ]} is the Fenchel-Legendre transform of the normalized cumulant function of S n. In the i.i.d. case, the function Λ x) = Λ nx) is known as the good rate function in large deviation principle LDP) theory see Deuschel and Stroock [16] or Dembo and Zeitouni [15]). Now we clarify the relation among our large deviation expansion ), Cramér large deviations [13] and the Bahadur-Rao theorem [] in the i.i.d. case. Without loss of generality, we take 1 = 1, where 1 is the variance of ξ 1. First, our bound ) implies that: for all x = o n), P S n > x) = e n Λ x/ n) Mx) [ 1 + O )] 1 + x, n. 33) n 9

11 Cramér [13] see also Theorem 3.1 of Saulis and Statulevicius [41] for more general results) proved that, for all x = o n), { )} )] PS n > x) x 3 x 1 + x = exp λ n [1 + O, n, 34) 1 Φx) n n where λ ) is the Cramér series. So the good rate function and the Cramér series have the relation n Λ x n ) = ) x x3 x n λ n. Second, consider the large deviation probabilities P S n n > y ). Since Sn n, a.s., as n, we only place emphasis on the case where y is small positive constant. Bahadur-Rao proved that, for given positive constant y, ) Sn P n > y = e n Λ y) [ 1 + O c ] y 1y t y πn n ), n, 35) where c y, 1y and t y depend on y and the distribution of ξ 1 ; see also Bercu [8, 9], Rozovky [31] and Györfi, Harremöes and Tusnády [4] for more general results. Our bound ) implies that, for y small enough, ) Sn P n > y = e n Λ y) My [ n) 1 + Oy + 1 ] ). 36) n In particular, when < y = yn) and y n as n, we have ) Sn P n > y = e n Λ y) [ ] y 1 + o1), n. 37) πn Expansion 36) or 37) is less precise than 35). However, the advantage of the expansions 36) and 37) over the Bahadur-Rao expansion 35) is that the expansions 36) or 37) are uniform in y where y may be dependent of n), in addition to the simpler expressions without the factors t y and y ). 4. Auxiliary results We consider the positive r.v. Z n λ) = n e λξ i E[e λξ i ], λ < ε 1, the Esscher transformation) so that E[Z n λ)] = 1. We introduce the conjugate probability measure P λ defined by dp λ = Z n λ)dp. 38) Denote by E λ the expectation with respect to P λ. Setting and we obtain the following decomposition: where b i λ) = E λ [ξ i ] = E[ξ ie λξi ], i = 1,..., n, E[e λξi ] T k λ) = η i λ) = ξ i b i λ), i = 1,..., n, S k = T k λ) + Y k λ), k = 1,..., n, 39) k b i λ) and Y k λ) = k η i λ). In the following, we give some lower and upper bounds of T n λ), which will be used in the proofs of theorems. 1

12 Lemma 4.1. For all λ < ε 1, 1.4λε)λ 1 1.5λε)1 λε) 1 λε + 6λ ε λ T n λ) 1.5λε 1 λε) λ. Proof. Since E[ξ i ] =, by Jensen s inequality, we have E[e λξi ] 1. Noting that E[ξ i e λξi ] = E[ξ i e λξi 1)], λ, by Taylor s expansion of e x, we get T n λ) E[ξ i e λξ i ] = λ + + k= λ k k! E[ξk+1 i ]. 4) Using Bernstein s condition ), we obtain, for all λ < ε 1, + k= λ k k! E[ξk+1 i ] 1 + λ ε = k= k + 1)λε) k 3 λε 1 λε) λ ε. 41) Combining 4) and 41), we get the desired upper bound of T n λ). By Jensen s inequality and Bernstein s condition ), from which we get E[ξ i ]) E[ξ 4 i ] 1ε E[ξ i ], E[ξ i ] 1ε. Using again Bernstein s condition ), we have, for all λ < ε 1, + E[e λξi ] 1 + k= λ k k! E[ξk i ] 1 + λ E[ξ i ] 1 λε) 1 + 6λ ε 1 λε = 1 λε + 6λ ε. 4) 1 λε Notice that gt) = e t 1 + t + 1 t ) satisfies that gt) > if t > and gt) < if t <, which leads to tgt) for all t R. That is, te t t1 + t + 1 t ) for all t R. Therefore, for all λ < ε 1, ξ i e λξi ξ i 1 + λξ i + λ ξi ). Taking expectation, we get E[ξ i e λξ i ] λe[ξi ] + λ E[ξ3 i ] λe[ξi ] λ 1 3!εE[ξ i ] = 1 1.5λε)λE[ξi ], 11

13 from which, it follows that E[ξ i e λξi ] 1 1.5λε)λ. 43) Combining 4) and 43), we obtain the following lower bound of T n λ): for all λ < ε 1, T n λ) E[ξ i e λξ i ] E[e λξ i ] 1 1.5λε)1 λε) 1 λε + 6λ ε λ 1.4λε)λ. 44) This completes the proof of Lemma 4.1. We now consider the following cumulant function Ψ n λ) = log E[e λξ i ], λ < ε 1. 45) We have the following elementary bound for Ψ n λ). Lemma 4.. For all λ < ε 1, and Ψ n λ) n log λ ) 1 + λ n1 λε) 1 λε) λt n λ) + Ψ n λ) λ 1 λε) 6. Proof. By Bernstein s condition ), it is easy to see that, for all λ < ε 1, Then, we have + E[e λξ i ] = 1 + k= Ψ n λ) λ k k! E[ξk i ] 1 + λ E[ξ i ] λε) k = 1 + λ E[ξi ] 1 λε). k= log 1 + λ E[ξi ] ). 46) 1 λε) Using the fact log1 + t) is concave in t and log1 + t) t, we get the first assertion of the lemma. Since Ψ n ) = and Ψ nλ) = T n λ), by Lemma 4.1, for all λ < ε 1, Ψ n λ) = λ T n t)dt Therefore, using again Lemma 4.1, we see that λ t1.4tε) dt = λ 1 1.6λε). λt n λ) + Ψ n λ) 1.5λε 1 λε) λ + λ 1 1.6λε) λ 1 λε) 6, 1

14 which completes the proof of the second assertion of the lemma. Denote λ) = E λ [Yn λ)]. By the relation between E and E λ, we have E[ξ λ) = i e λξ i ] E[e λξ E[ξ ie λξi ]) ) i ] E[e λξ, λ < ε 1. i ]) Lemma 4.3. For all λ < ε 1, Proof. Denote fλ) = E[ξ i eλξi ]E[e λξi ] E[ξ i e λξi ]). Then, Thus, 1 λε) 1 3λε) 1 λε + 6λ ε ) λ) 1 λε) 3. 47) f ) = E[ξ 3 i ] and f λ) = E[ξ 4 i e λξ i ]E[e λξ i ] E[ξ i e λξ i ]). fλ) f) + f )λ = E[ξ i ] + λe[ξ 3 i ]. 48) Using 48), 4) and Bernstein s condition ), we have, for all λ < ε 1, Therefore E λ [η i ] = E[ξ i eλξ i ]E[e λξ i ] E[ξ i e λξ i ]) E[e λξ i ]) E[ξ i ] + λe[ξ3 i ] E[e λξ i ]) ) 1 λε 1 λε + 6λ ε E[ξi ] + λe[ξi 3 ]) 1 λε) 1 3λε) 1 λε + 6λ ε ) E[ξ i ]. λ) 1 λε) 1 3λε) 1 λε + 6λ ε ). Using Taylor s expansion of e x and Bernstein s condition ) again, we obtain λ) E[ξi e λξi ] 1 λε) 3. This completes the proof of Lemma 4.3. For the r.v. Y n λ) with λ < ε 1, we have the following result on the rate of convergence to the standard normal law. Lemma 4.4. For all λ < ε 1, sup P λ y R ) Yn λ) λ) y Φy) ε 3 λ)1 λε). 4 Proof. Since Y n λ) = n η iλ) is the sum of independent and centered respect to P λ ) r.v. s η i λ), using standard results on the rate of convergence in the central limit theorem cf. e.g. Petrov [34], p. 115) we get, for λ < ε 1, ) sup P Yn λ) λ λ) y Φy) C E λ [ η i 3 ], λ) y R 13

15 where C 1 > is an absolute constant. For λ < ε 1, using Bernstein s condition, we have E λ [ η i 3 ] 4 E λ [ ξ i 3 + E λ [ ξ i ]) 3 ] As E λ [ ξ i 3 ] E[ ξ i 3 exp{ λξ i }] 4 ε [ E j= λ j j! ξ i 3+j] j + 3)j + )j + 1)λε) j. j= j + 3)j + )j + 1)x j = d3 dx 3 j= we obtain, for λ < ε 1, x j 6 =, x < 1, 1 x) 4 j= E λ [ η i 3 ε ] 4 1 λε) 4. Therefore, we have, for λ < ε 1, ) sup P Yn λ) λ λ) y Φy) ε 4C 1 3 λ)1 λε) 4 y R where the last step holds as C 1.56 cf. Shevtsova [4]). Using Lemma 4.4, we easily obtain the following lemma. Lemma 4.5. For all λ.1ε 1, sup P λ y R Y n λ) y ) 1 λε Proof. Using Lemma 4.3, we have, for all λ < 1 3 ε 1, It is easy to see that 1 λε ε λ)1 λε), 4 Φy) 1.7λε ε. λ)1 λε) 1 λε + 6λ ε 1 λε) 1 3λε. 49) P λ Y n λ) y ) Φy) 1 λε ) P Yn λ) λ λ) y y Φ λ)1 λε) λ)1 λε)) ) + Φ y Φy) λ)1 λε) =: I 1 + I. 14

16 By Lemma 4.4 and 49), we get, for all λ < 1 3 ε 1, I ε 3 λ)1 λε) Rλε) ε. Using Taylor s expansion and 49), we obtain, for all λ < 1 3 ε 1, 1 I ye y 1 λε) π λ)1 λε) 1 1 ye y 1 λε) 1 λε + 6λ ε π 1 λε) 1 3λε 1 1 ) 1 λε 1 1 λε + 6λ ε eπ1 λε) 1 λε) 1 3λε 1. By simple calculations, we obtain, for all λ.1ε 1, P λ Y n λ) y ) Φy) 1 λε 1.7λε ε. This completes the proof of Lemma Proofs of Theorems.1-. In this section, we give upper bounds for PS n > x). For all x and λ < ε 1, by 38) and 39), we have: PS n > x) = E λ [Z n λ) 1 1 {Sn>x}] Setting U n λ) = λy n λ) + T n λ) x), we get Then, we deduce, for all x and λ < ε 1, = E λ [e λsn+ψnλ) 1 {Sn >x}] = E λ [e λtnλ)+ψnλ) λynλ) 1 {Yn λ)+t n λ) x>}]. 5) PS n > x) = e λx+ψnλ) E λ [e Unλ) 1 {Un λ)>}]. PS n > x) = e λx+ψ nλ) In the sequel, denote by N, 1) a standard normal r.v Proof of Theorem.1 From 51), using Lemma 4., we obtain, for all x and λ < ε 1, PS n > x) e λx+ λ 1 λε) e t P λ < U n λ) t)dt. 51) e t P λ < U n λ) t)dt. 5) For any x and β [,.5), let λ = λx) [, ε 1 ) be the unique solution of the equation λ βλ ε 1 λε) = x. 15

17 This definition and Lemma 4.1 implies that x/ λ = 1 + xε/ β)xε/ and T n λ) x. 53) Using 5) with λ = λ, we get where PS n > x) e β)λε) x e t P λ < U n λ) t)dt, 54) x = By 53) and Lemma 4.5, we have, for λ.1ε 1, = λ 1 λε. e t P λ < U n λ) t)dt e y x P λ < Un λ) y x ) xdy e y x P < N, 1) y) xdy + 1.7λε ε ) = M x) +.14λε ε. 55) Since e t P λ < U n λ) t)dt 1 and M 1 t) π 1 + t) for t cf. 6)), combining 54) and 55), we deduce, for all x, with and PS n > x) e 1 1 β)λε x 1 x [ 1 {λε>.1} +e 1 1 β)λε x 1 Φ x) + e 1 x.14λε ε )] 1 {λε.1} 1 Φ x)) I 11 + I 1 ), 56) I 11 = exp { 1 } [ 1 ] β)λε x π 1 + x) 1 {λε>.1} 57) [ I 1 = e 1 1 β)λε x 1 + π 1 + x).14λε ε )] 1 {λε.1}. Now we shall give estimates for I 11 and I 1. If λε >.1, then I 1 = and } [ I 11 exp {.11 β) x ] π 1 + x). 58) By a simple calculation, I 11 1 provided that x 8 1 β λ = x1 λε) < β 1.1) = 1 β. Then, using 1λε > 1, we obtain I π 1 + x) π 1 + x) λ ε β 1 + x) ε note that β [,.5)). For x < 1 β, we get

18 If λε.1, we have I 11 =. Since 1 + π 1 + x).14λε ε ) ) π 1 + x) λε π 1 + x) ε ) = J 1 J, it follows that I 1 exp { 1 1 β)λε x} J 1 J. Using the inequality 1 + x e x, we deduce { )} I 1 exp λε 1 β) x.14 π 1 + x) J. If x β, we see that 1 1 β) x.14 π 1 + x), so I 1 J. For x < β, we get λ = x1 λε) < β. Then I1 1 + π 1 + x).14λε ε ) < 1 + π 1 + x) ) ε 1 β ) β x) ε. Hence, whenever λε < 1, we have I 11 + I ) β ) 1 + x) ε 1 β. 59) Therefore, substituting λ from 53) in the expression of x = λ and replacing 1 β by δ, we obtain inequality 1 λε 1) in Theorem.1 from 56) and 59). 5.. Proof of Theorem. For any x, let λ = λx) [, ε 1 ) be the unique solution of the equation By Lemma 4.1, it follows that x/ λ = 1 + xε/ xε/ Using Lemma 4.4 and T n λ) x, we have, for all λ < ε 1, = e t P λ < U n λ) t)dt λ.5λ ε 1 λε) = x. 6) e yλλ) P λ < Un λ) yλλ) ) λλ)dy and T n λ) x. 61) e yλλ) ε P < N, 1) y) λλ)dy λ)1 λε) 4 e yλλ) dφy) = F := M λλ) ) ε 3 λ)1 λε) 4 ε 3 λ)1 λε) 4. 6) 17

19 Using λ = λ and e t P λ < U n λ) t)dt 1, from 51) and 6), we obtain PS n > x) [F 1] exp { λx + Ψ n λ) }. By Lemma 4., inequality 63) implies that { λ )} PS n > x) [F 1] exp λx + n log 1 +. n1 λε) Substituting λ from 61) in the previous exponential function, we get PS n > x) [F 1] B n x, ε ). 63) Next, we give an estimation of F. Since Mt) is decreasing in t and M t) 1 π t, t >, it follows that M λλ) ) Mx) Using Lemma 4.3, by a simple calculation, we deduce By Lemma 4.3, it is easy to see that 1 1 ) + x λλ). π λ λ) M λλ) ) Mx) 1 λ 1.5λε 1 λε) 1 3λε) + π λ λ) 1 λε) 1 λε + 6λ ε 1.5λε)1 λε + 6λ ε ) 1 λε) 1 3λε) + ε π λε1 λε)4 1 3λε) + /1 λε + 6λ ε ) 1.11R λε ) ε. 64) 6.88 Hence, it follows from 6), 64) and 65) that Implementing 66) into 63) and using λε x ε, we obtain inequality 13). F ε 3 λ)1 λε) 6.88R λε ) ε 4. 65) Mx) R λε ) ε. 66) 6. Proof of Theorem.3 In this section, we give a lower bound for PS n > x). From Lemma 4. and 5), it follows that, for all λ < ε 1, PS n > x) exp { λ } 1 λε) 6 E λ [e λynλ) 1 {Yn λ)+t n λ) x>}]. Let λ = λx) [, ε 1 /4.8] be the unique solution of the equation λ1.4λε) = x. 67) 18

20 This definition and Lemma 4.1 implies that, for all x /9.6ε), x/ λ = xε/ and x T n λ). 68) Therefore, PS n > x) exp { λ } 1 λε) 6 E λ [e λynλ) 1 {Ynλ)>}]. Setting V n λ) = λy n λ), we get where ˇx = where G = 6.88 that PS n > x) exp } { ˇx e t P λ < V n λ) t)dt, 69) λ. By Lemma 4.4 and an argument similar to that used to prove 6), it is easy to see that 1 λε) 3 Returning to 69), we obtain e t P λ < V n λ) t)dt M λλ) ) G, ε. Since Mt) is decreasing in t and λ) 3 λ)1 λε) 4 e t P λ < V n λ) t)dt M ˇx) G. PS n > x) 1 Φ ˇx) G exp Using Lemma 4.3, for all x /9.6ε), we have λε 1/4.8 and G 6.88R λε ) ε. } { ˇx. 1 λε) 3 Therefore, for all x /9.6ε), PS n > x) 1 Φ ˇx) 6.88R λε ) } ε { exp. Using the inequality M 1 t) π 1 + t) for t, we get, for all x /9.6ε), ) [ PS n > x) 1 Φ ˇx) R λε ) 1 + ˇx) ε ]. In particular, for all x α/ε with α 1/9.6, a simple calculation shows that and λε α α R λε ) ) α 67.38R α This completes the proof of Theorem.3. ) R cf. Lemma 4.3), it follows 19

21 7. Proof of Theorem.4 Notice that Ψ nλ) = T n λ) [, ) is nonnegative in λ. Let λ = λx) be the unique solution of the equation x = Ψ nλ). This definition implies that T n λ) = x, U n λ) = λy n λ) and e λx+ψ nλ) = inf λ e λx+ψ nλ) = inf λ E[eλS n x) ]. 7) From 51), using Lemma 4.4 with λ = λ and an argument similar to 6), we obtain PS n > x) = M λλ) ) θ 1 ) ε inf 3 λ)1 λε) 4 λ E[eλS n x) ], 71) where θ 1 1. Since Mt) is decreasing in t and M t) 1 π t M λλ) ) Mx) By Lemma 4.1, we have the following two-sided bound of x: 1 1.5λε)1 λε) 1 λε + 6λ λ T nλ) ε in t >, it follows that 1 x λλ) π λ λ) x. 7) = x 1.5λε λ. 73) 1 λε) Using the two-sided bound in Lemma 4.3 and 73), by a simple calculation, we deduce and λ λ) x 1 λε) 1 3λε) 1 λε + 6λ ε ) λ 74) x λλ) λ 1.5λε 1 λε) 1 λε) 1 3λε) + 1 λε + 6λ. 75) ε From 7), 74), 75) and Lemma 4.3, we easily obtain M λλ) ) Mx) 1.11R λε ) ε. 76) By Lemma 4.3, it is easy to see that Combining 76) and 77), we get, for all λ < 1 3 ε 1, M λλ) ) + where θ 1. By 73), it follows that, for all λ < 1 3 ε 1, 6.88 ε 3 λ)1 λε) 6.88R λε ) ε 4. 77) 6.88θ 1 ε 3 λ)1 λε) = Mx) θ R λε ) ε 4, 78) λε 1 λε + 6λ ε 1 1.5λε)1 λε) x ε 4x ε. 79) Implementing 78) into 71) and using 79), we obtain equality ) of Theorem.4. Notice that R < restricts 4x ε < 1 1 ε 3, which implies that x < 1.

22 [1] Arkhangelskii A N. Lower bounds for probabilities of large deviations for sums of independent random variables. Theory Probab Appl, 1989, 34: [] Bahadur R, Rao R R. On deviations of the sample mean. Ann Math Statist, 196, 31: [3] Bennett G. Probability inequalities for the sum of independent random variables. J Amer Statist Assoc, 196, 57: [4] Bentkus V. An inequality for tail probabilities of martingales with differences bounded from one side. J Theoret Probab, 3, 16: [5] Bentkus V. On Hoeffding s inequality. Ann Probab, 4, 3: [6] Bentkus V, Dzindzalieta D A. Tight Gaussian bound for weighted sums of Rademacher random variables. ArXiv: , 13 [7] Bentkus V, Kalosha N, van Zuijlen M. On domination of tail probabilities of super)martingales: explicit bounds. Lithuanian Math J, 6, 46: 1 43 [8] Bercu B. Inégalités exponentielles pour les martingales. Journées ALEA, 8, 1: 1 33 [9] Bercu B, Rouault A. Sharp large deviations for the Ornstein-Uhlenbeck process. Theory Probab Appl, 6, 46: 1 19 [1] Bernstein S N. The Theory of Probabilities. Moscow, Leningrad, 1946 [11] Bousquet O. A Bennett concentration inequality and its application to suprema of empirical processes. C R Acad Sci Paris Ser I,, 334: [1] Chaganty N R, Sethuraman J. Strong large deviation and local limit theorems. Ann Probab, 1993, 1: [13] Cramér H. Sur un nouveau théorème-limite de la théorie des probabilités. Actualite s Sci Indust, 1938, 736: 5 3 [14] de la Peña V H. A general class of exponential inequalities for martingales and ratios. Ann Probab, 1999, 7: [15] Dembo A, Zeitouni O. Large deviations techniques and applications. Springer, New York, 1998 [16] Deuschel J D, Stroock D W. Large deviations. Academic Press, Boston, 1989 [17] Eaton M L. A probability inequality for linear combination of bounded randon variables. Ann Statist, 1974, : [18] Fan X, Grama I, Liu Q. About the constant in Talagrand s inequality for sums of bounded random variables. ArXiv:16.51, 1, 1 [19] Fan X, Grama I, Liu Q. Cramér large deviation expansions for martingales under Bernstein s condition. Stochastic Process Appl, 13, 13: [] Fan X, Grama I, Liu Q. Sharp large deviations under Bernstein s condition. C R Acad Sci Paris Ser I, 13, 351: [1] Fu J C, Li G, Zhao L C. On large deviation expansion of distribution of maximum likelihood estimator and its application in large sample estimation. Ann Inst Statist Math, 1993, 45:

23 [] Itō K, MacKean H P. Difussion Processes and Their Sample Paths. Springer, 1996 [3] Grama I, Haeusler E. Large deviations for martingales via Cramer s method. Stochastic Process Appl,, 85: [4] Györfi L, Harremöes P, Tusnády G. Some refinements of large deviation tail probabilities. ArXiv:15.15v1 [math.st], 1 [5] Hoeffding W. Probability inequalities for sums of bounded random variables. J Amer Statist Assoc, 1963, 58: 13 3 [6] Jing B Y, Shao Q M, Wang Q. Self-normalized Cramér-type large deviations for independent random variables. Ann Probab, 3, 31: [7] Jing B Y, Liang H Y, Zhou W. Self-normalized moderate deviations for independent random variables. Sci China Math, 1, 5511): [8] Joutard C. Sharp large deviations in nonparametric estimation. J Nonparametr Stat, 6, 18: [9] Joutard C. Strong large deviations for arbitrary sequences of random variables. Ann Inst Stat Math, 13, 651): [3] Rozovky L V. A lower bound of large-deviation probabilities for the sample mean under the Cramér condition. J Math Sci, 3, 1186) [31] Rozovky L V. Large deviation probabilities for some classes of distributions statisfying the Cramér condition. J Math Sci, 5, 181) [3] Nagaev S V. Large deviations of sums of independent random variabels. Ann Probab, 1979, 7: [33] Nagaev S V. Lower bounds for the probabilities of large deviations of sums of independent random variables. Theory Probab Appl,, 46: 79 1; [34] Petrov V V. Sums of Independent Random Variables. Springer-Verlag, Berlin, 1975 [35] Petrov V V. Limit Theorems of Probability Theory. Oxford University Press, Oxford, 1995 [36] Pinelis I. An asymptotically Gaussian bound on the Rademacher tails. Electron J Probab, 1, 1735): 1 [37] Pinelis I. On the Bennett-Hoeffding inequality. Ann Inst H Poincaré Probab Statist, 14, 51): 15 7 [38] Rio E. A Bennett type inequality for maxima of empirical processes. Ann Inst H Poincaré Probab Statist,, 6: [39] Rio E. Sur la fonction de taux dans les inégalités de Talagrand pour les processus empiriques. C R Acad Sci Paris Ser I, 1, 35: [4] Rio E. Inégalités exponentielles et inégalités de concentration. Institut Mathématique de Bordeaux, 1, 1: 1. [41] Saulis L, Statulevicius V A. Limit theorems for large deviations. Springer, 1991 [4] Shevtsova I G. An improvement of convergence rate estimates in the Lyapunov theorem. Doklady Math, 1, 8:

24 [43] Sakhanenko A I. Berry-Esseen type bounds for large deviation probabilities. Siberian Math J, 1991, 3: [44] Shao Q M. A Cramér type large deviation result for Student s t-statistic. J Theoret Probab, 1999, 1): [45] Talagrand M. The missing factor in Hoeffding s inequalities. Ann Inst H Poincaré Probab Statist, 1995, 31: [46] van de Geer S. Exponential inequalities for martingales, with application to maximum likelihood estimation for counting process. Ann Statist, 1995, 3: [47] van de Geer S, Lederer J. The Bernstein-Orlicz norm and deviation inequalities. Probab Theory Relat Fields, 13, 157: 5 5 3