SUB-BERNOULLI FUNCTIONS, MOMENT INEQUALITIES AND STRONG LAWS FOR NONNEGATIVE AND SYMMETRIZED U-STATISTICS 1. By Cun-Hui Zhang Rutgers University

Transcription

1 The Aals of Probablty 1999, Vol. 27, No. 1, SUB-BERNOULLI FUNCTIONS, MOMENT INEQUALITIES AND STRONG LAWS FOR NONNEGATIVE AND SYMMETRIZED U-STATISTICS 1 By Cu-Hu Zhag Rutgers Uversty Ths paper cocers momet ad tal probablty equaltes ad the strog law of large umbers for U-statstcs wth oegatve or symmetrzed kerels ad ther multsample ad decoupled versos. Sub- Beroull fuctos are used to obta the momet ad tal probablty equaltes, whch are the used to obta ecessary ad suffcet codtos for the almost sure covergece to zero of ormalzed U-statstcs wth oegatve or completely symmetrzed kerels, wthout further regularty codtos o the kerel or the dstrbuto of the populato, for ormalzg costats satsfyg a smple codto. Momets of U-statstcs are bouded from above ad below by that of maxma of certa kerels, up to scalg costats. The multsample ad decoupled versos of these results are also cosdered. 1. Itroducto. Ths paper cocers momet ad tal probablty equaltes ad the strog law of large umbers (SLLN) for U-statstcs wth oegatve or symmetrzed kerels ad ther multsample ad decoupled versos Overvew. Let X X 1 be a sequeces of d radom varables wth a commo dstrbuto F. For real Borel fuctos hx 1 x k, the U- statstcs wth kerel h are defed by S k / ( k) wth 11 S k = 1 k k hx 1 X k where k = 1 k 1 1 < 2 < < k. Our vestgato s motvated by two problems. The frst oe s the order of ES k : gve a odecreasg oegatve fucto satsfyg certa regularty codtos [e.g., x =x m ], fd fuctoals µ F h such that 12 C k µ F h E ( S k ) C k µ F h 1 where C k ad C k are uversal costats. The secod problem s the SLLN: gve a sequece of ormalzg costats b satsfyg certa regularty codtos (e.g., b = 1/p for some 0 <p<2), fd ecessary ad Receved Jue 1996; revsed August Supported part by the Natoal Scece Foudato ad Natoal Securty Agecy. AMS 1991 subect classfcatos. Prmary 60F15; secodary 60G50. Key words ad phrases. Sub-Beroull fucto, strog law of large umbers, momet equalty, expoetal equalty, tal probablty, U-statstcs. 432

2 STRONG LAWS FOR U-STATISTICS 433 suffcet codtos (asc) o F ad h for 13 S k /bk = b k hx 1 X k 0 1 < 2 < < k We fd that the cocept of sub-beroull fucto, defed (1.10) for k = 2 a specal case ad formally defed Secto 2.1, s very useful our vestgato of the precedg two problems ad some addtoal problems. Bascally, a oegatve Borel fucto φx 1 x k s a sub-beroull fucto of X 1 X k wth parameter θ 1 θ k f ts codtoal expectatos, gve subsets of the X s, are o greater tha the correspodg codtoal expectatos of a product of depedet Beroull varables wth the same parameters. We coect sub-beroull fuctos to oegatve kerels h 0 through some ormalzg kerels ψ = ψ x 1 x k, postve Borel fuctos gve Secto 3.1 for geeral k ad (1.9) ad (1.11) for k = 2, such that φ x 1 x k =h/ψ are sub-beroull fuctos wth parameters k/k/. It wll be show Theorem 3.4 Secto 3.2 that, for all h 0, odecreasg oegatve fuctos g ad tegers m 1 ad k, 14 Eg ( ξ k ) ( S k ξ k ) m Eg ( ξ k ( ) ) k + m Nk E k where ξ k = max 1 k k ψ X 1 X k ad N k s a Posso varable wth EN k = k. It wll also be show Theorem 3.4 that for k 15 P { ξ k >t C k P { S k >t/2 for some uversal C k. These equaltes provde crucal elemets our solutos to (1.2) ad (1.3). Momet equaltes for sub-beroull fuctos also mply a exteso of the Berste equalty from k = 1 to geeral k, Corollary 2.4 Secto 2.3, for decoupled symmetrzed bouded kerels. The symmetrzed, multsample ad/or decoupled versos of the strog law ad momet equaltes are also gve. Some basc equaltes for sub-beroull fuctos are provded for geeral depedet (ot ecessarly detcally dstrbuted) varables. The paper s orgazed as follows. I the rest of ths secto, we dscuss detal the case k = 2 after gvg our otato. I Secto 2, we descrbe sub-beroull fuctos ad some basc equaltes. I Secto 3, we cosder equaltes of type (1.4) ad (1.5). I Secto 4, we provde the SLLN. Secto 5 cotas examples about momet codtos for SLLN Notato. Let X l X l 1 be depedet sequeces of d radom varables from possbly dfferet dstrbutos. The multsample verso of (1.1) s defed by 16 S = 1 1 =1 k k =1 h ( X 1 1 X k ) k a.s.

3 434 C.-H. ZHANG where = 1 k gves the sample szes. The ormalzed sums S / k l=1 l are the k-sample U-statstcs wth the kerel h.for 1 = = k =, 17 S k = 1 k k h ( X 1 1 X k ) where k = 1 k 1 l 1 l k. Whe X X 1 X k are detcally dstrbuted, S k becomes the decoupled verso S k (1.1). Let fx y be a real Borel fucto wth f 2 x y =hx y, ad ε ε ε l 1l 1 be d Rademacher varables depedet of X X l 1l 1, Pε = 1 =Pε = 1 =1/2. Defe 18 = ε fx = ε f X T k k They are the symmetrzed versos of (1.1) ad (1.7). Here ad the sequel, X =X 1 X k, X =X 1 1 X k k, ε = k l=1 ε l ad ε = k l=1 εl l for = 1 k. I addto to the varables troduced above, we shall use the followg otato throughout. Let Y Y l 1l 1 be depedet varables, θ θ l 1l 1 be costats 0 1, ad δ δ l 1l 1 be depedet Beroull oes wth Pδ l For = 1 k, set y = y 1 y k, Y = Y 1 Y k ad Ỹ = T k k k = 1 = θ l ad Pδ = 1 = θ. Y 1 1 Y k k. Set θ =θ 1 θ k ad θ =θ 1 1 θ k k. Set δ = k l=1 δ l ad δ = k l=1 δl l, the same maer as ε ad ε (1.8). Let N θ be a bomal varable wth parameters θ ad N λ be a Posso varable wth EN λ = λ. Also, for all real a, A a = 1fA =. The dstrbutos of X l, l 1, are ot assumed to be detcal, to cover the mult sample case. The varables Y Y l are ot assumed to have the same dstrbuto (ether betwee dfferet or betwee dfferet l), to be used to descrbe basc equaltes for sub-beroull fuctos. The codtoal expectatos ad momets of sub-beroull fuctos are domated by those of products of δ δ l, wth odetcal θ θ l geeral (betwee dfferet l as well as dfferet ). The Rademacher varables ε ε ε l 1l 1 are assumed to be depedet of X X l Y Y l δ δ l 1l 1 throughout. Also, x 1 x m = maxx 1 x m, x 1 x m = mx 1 x m Case k = 2. Let us dscuss k = 2 more detal. Let hx y = hy x 0. For θ>0 ad 1, defe { ( ) hx y c 1 y θ =sup c>0 E θ sup = 0 hx y c { ( ) hx c 0 θ =sup c>0 E 1 X 2 θ 2 hx 1 X 2 c 1 X 1 θ c 1 X 2 θ c

4 STRONG LAWS FOR U-STATISTICS 435 ad 19 ψx y θ =hx y c 1 x θ c 1 y θ c 0 θ It ca be easly see (also cf. Lemma 3.1) that for φ = hx 1 X 2 /ψx 1 X 2 θ φ 1 Eφ X Eδ 1 δ 2 δ = 1 2 Eφ Eδ 1 δ 2 f δ are d Beroull varables wth mea θ. I other words, the codtoal expectatos of φ (1.10) are domated by ther δ 1 δ 2 versos. I ths sese, we call φ a sub-beroull fucto of X 1 X 2. The fucto c 1 y 1/ as (1.9) s the same as m y = supm EhX y m m, a quatty whose essece has bee used to approxmate the ceter of the dstrbuto of a sum of d oegatve radom varables [each dstrbuto ths case s hx y]. I fact, Lemma 2.3 of Klass ad Zhag (1994) shows that PS y cy 1//3 02 ad PS y 3cy 1/ 03 wth S y = =1 hx y. I ths paper, 111 ξ 2 = max ψx X 2/ 1 < { = max max hx X max < c 1X 2/c 0 2/ are used to approxmate the ceter ad momets of (1.1) for k = 2. The maxmum of hx X represets the extreme term; the maxmum of c 1 X 2/ represets the extreme term of =1 hx X the sum over ; whle c 0 2/ represets the overall ceter of the double sum. For more dscussos, see Klass ad Nowck (1997). It wll be show Theorems 3.2 ad 3.4 that (1.4) ad ts two-sample verso 112 E ( S 2 / ξ 2 ) mg ( ξ 2 ) ( Eg ξ 2 ){ E1 + N1 m 2 ad ther extesos to geeral k, hold for all odecreasg oegatve fuctos g ad m 1, where ξ 2 = max 2 ψx 1 X 2 1/ wth ψ beg the X 1 X 2 -verso of ψ (cf. Secto 3.1). These theorems also assert that (1.5) ad ts two-sample verso 113 P { ξ 2 >t 24P { S 2 t/2 ad ther extesos to geeral k, hold for all postve t. Let be a fucto satsfyg 114 x x x 0cx Mc α x c c 1x 0 for some α>0. Ths cludes x =x α. It follows from (1.12) ad (1.13) that 115 C Mα E ξ 2 E S 2 C M α E ξ 2

5 436 C.-H. ZHANG wth C M α = 1/24M2Vc α ad C M α = Mcα +E1 + N 1 m 2, where m 1 <α m. The upper boud above follows from (1.12) as S 2 M max S 2 / ξ 2 α 1 ξ 2 ad c α c m for c 1. Iequalty (1.15) for geeral k ad ts oe-sample verso [based o (1.4) ad (1.5)] are gve Corollares 3.3 ad 3.5. Va dfferet methods, Klass ad Nowck (1997) obtaed (1.15) the depedet but o-d case, usg fuctos h x y 0 place of a fxed hx y. Ther results volved the costructo of dfferet costats. Let b = 1/p. Suffcet momet codtos for the SLLN (1.3) were gve by Hoeffdg (1961), Serflg (1980), Se (1974), Techer (1992) ad Gé ad Z (1992). By the Kolmogorov ad Marckewcz Zygmud strog laws, (1.3) holds for k = 1 f ad oly f EhX =0 for p 1 ad EhX p <. However, the case k 2 s qute dfferet. Gé ad Z (1992) gave a example to exhbt that the codto EX p < s ot ecessary for (1.3) whe hx y =xy. I Example 5.2 below, (1.3) holds for b = k/p ad some symmetrc h but EhX 1 X k p1+ε = for all ε>0, where 0 <p<2 ad p 1 = p/k pk 1/2 <p. For the specal case hx y =xy ad EX = 0 wheever EX <, Cuzck, Gé ad Z (1995) obtaed asc for the SLLN (1.3) uder certa regularty codtos o the dstrbuto of X (e.g., X symmetrc, PX >xregularly varyg), ad Zhag (1996) obtaed asc wthout regularty codtos o X. Some extesos of these results for k>2 are also avalable these papers. The SLLN ths paper gve asc for (1.3) for geeral oegatve kerels ad ts symmetrzed ad/or multsample versos. Theorem 1.1. Let S 2 ad S 2 be as (1.1) ad (1.7) ad c 1 θ ad c 0 θ be as (1.9). Suppose hx y =hy x 0, X X 1 X 2 are detcally dstrbuted, ad sup 1 2 b 2 m= m/b 2 m <. The the SLLN (1.3) holds ff T 2 /b 0 a.s., ff ξ 2 /b 2 0 a.s., ff S 2 /b 2 0 a.s., ff T 2 /b 0 a.s., ff ξ 2 /b 2 0 a.s., ff the followg three codtos hold: 116 c 0 1//b P { c 1 X 1/ >b 2 < =1 P { hx 1 X 2 >b 2 c 1X 1 1/ c 1 X 2 1/ < =1 Remark. 119 Codto (1.18) ca be replaced by P { hx 1 X 2 >b 2 c 1X 1 1/ c 1 X 2 1/ < =1 The proof of Theorem 1.1 ad ts extesos for geeral k ad multsample versos are gve Secto 4.

6 STRONG LAWS FOR U-STATISTICS Sub-Beroull fuctos. I the followg three subsectos, we shall (1) defe sub-beroull fuctos ad descrbe the motvato, (2) provde upper bouds for codtoal expectatos of products ad momets of sums of sub-beroull fuctos ad (3) provde some expoetal equaltes Sub-Beroull fuctos. A radom varable φ ( ) Y 1 Y k s called a sub-beroull fucto of a radom vector ( ) Y 1 Y k wth parameter θ 1 θ k f 0 φ 1 ad for all A 1k 21 E [ φy 1 Y k Y l l A c] θ l From ths defto, sub-beroull fuctos are oegatve fuctos of Y 1 Y k whose codtoal expectatos gve subsets of Y 1 Y k are uformly bouded from above by the products of the θ s the complemetary subsets of θ 1 θ k. Cosder the case of k = 1. By defto φ = φ Y are sub-beroull fuctos of Y ff 0 φ 1 ad Eφ θ. Such varables φ are domated momets by Beroull varables δ wth Eδ = θ the sese that Eφ m Eδ m = θ for all m 1. Although ths does ot mply stochastc domace (.e., Pφ >t Pδ >tmay ot hold for all t), t s strog eough to assure 22 ( ) m E φ = =1 1 m m m E m E δ =1 m φ =1 l A ( = E ) m δ =1 for all tegers m 1. Thus, as far as momets of sums are cocered, Beroull varables are the optmal oes amog all sub-beroull varables. We shall show below that products of depedet Beroull varables are optmal for geeral k amog all sub-beroull fuctos Expectatos of products ad momets of sums. Proposto 2.1. Beroull fuctos of Ỹ =Y 1 1 ) ) Suppose φ (Ỹ ad φs (Ỹ, 1 s m, are sub- Y k wth parameters θ =θ 1 θ k k for = 1 k k. The, for all s = 1s ks k, 1 s m, ad A 2, 23 [ m E φ s Ỹ s F A c ] m k l A s θ l l s [ m = E k l=1 ] δ l l s δ = 1 where F A = σy l l A, A s =l k l s l t s<t m l l s A, A c = 2 \ A, ad δ = δ l l s l l s A. Moreover, for all s k, 1

7 438 C.-H. ZHANG 1 s m, 24 ( ) ( m m E φ s Ỹ E s k s l=1 δ l l ) Cosequetly, for = k l=1 1 l wth = 1 k, 25 ( ) m E φ Ỹ k l=1 ( l E =1 ) m δ l Remark. Sce Y l are depedet (betwee dfferet l as well as dfferet ), the deces are allowed to have tes. Proof. Set φ = φ Ỹ. By (2.1), E [ φ s Ỹt s<t m F A c ] = E [ φs Y l l s l A s ] l A s θ l l s Repeated applcatos of ths equalty for s = 1m gve the equalty (2.3). The detty (2.3) follows from P { δ s = 1 δ t = 1s<t m δ = 1 = P { δ s = 1 δ l l s = 1l A s = θ l l s l A s Fally, smlarly to (2.2), (2.4) s proved by frst wrtg the product of sums as sum of products ad the applyg (2.3) wth A = 2 (trval F A c) to each (product) term the sum to allow substtuto of φ s by δ. For the sgle sequece Y, we have the followg aalogous result. Proposto 2.2. Suppose φ Y ad φ s Y, 1 s m, are sub- Beroull fuctos of Y wth parameters θ for = 1 k k. The, for all s = 1s k s k, 1 s m ad A 1, 26 [ m ] E φ s Y s F A c m θ l s l A s [ m = E k l=1 ] δ l s δ = 1 where A s =l k l s t s<t mad 1 k l s A, F A = σy A, A c = 1 \ A, ad δ = δ l s l s A. Moreover, for all s k, 1 s m, 27 ( ) ( m m E φ s Y E s s l=1 k δ l )

8 STRONG LAWS FOR U-STATISTICS 439 Cosequetly, wth T = =1 δ, ( m 28 E φ Y ) E k ( T Remark. The symmetrzed versos of (2.5) ad (2.8) ca be easly produced usg the Khtche equalty. ad Remark. For all odecreasg oegatve g, k ) m EN θ gn θ m θegn θ + 1 EN λ gn λ =λegn λ + 1 These ad Corollary 2.1 of Gleser (1975) mply ( ) m ( ) m ( ) m ET m ENm EN θ m T N θ Nλ λ E E E where θ = λ / = =1 θ / ad T s as (2.8). Thus, the T (2.8) ad the sums o the rght-had sde of (2.5) ca be replaced by Posso varables. The proof of Proposto 2.2 s omtted as t s early detcal to the proof of Proposto Expoetal equaltes. There are several ways of obtag expoetal equaltes for the tal probabltes of U-statstcs from momet equaltes. Here we shall oly preset oe for symmetrzed ad decoupled U-statstcs. Proposto 2.3. Let 0 < θ 1. Suppose φ Ỹ = f 2 Ỹ are sub- Beroull fuctos of Ỹ wth a commo parameter θθ for all = 1 k k. The, 29 ( E exp t ε f Ỹ 1/k) k/2 2E exp ( t 2 N θ /2 ) k where N θ s bomal θ. Cosequetly, { 1/k ε 210 P f Ỹ k/2 k ( t 2 exp t 2 /2 θ + t/2 Corollary 2.4. Suppose f Ỹ c ad Ef 2 Ỹ σ 2 for all = 1 k k. The { 1/k ( ε 211 P f Ỹ t 2 ) /2 k/2 t 2 exp σ 2/k + tc/2σ k 1/k k )

9 440 C.-H. ZHANG For k = 1, (2.11) becomes the Berste equalty. For d X ad completely degeerate kerels f wth f c ad Ef 2 σ 2, Arcoes ad Gé [(1993), page 1501] obtaed the equalty { P k fx k/2 1/k ( t c k exp c k ) t2 σ 2/k +tc/ 2/k+1 ad ts symmetrzed ad/or decoupled versos wth mplctly specfed uversal costats c k ad c k. Ther equaltes gve smaller upper bouds for σ k+1/k /ct =o1 tha (2.11) although the breakdow pot t = σ k+1/k /c s the same. For related expoetal equaltes for the Rademacher chaos, we refer to Ledoux ad Talagrad (1991). Proof of Proposto 2.3. Set T = ε f k Ỹ ad S = k f 2 Ỹ. Let Z be a N0 1 varable depedet of N θ. By the Khtche equalty ad (2.5) of Proposto 2.1, E T 2m ( EZ 2m) k ( ) m ( E S EZ 2m EN m k θ) By the Jese equalty E T 2m/k E ( Z ) 2m. N θ Sce e x e x +e x, the left-had sde of (2.9) s bouded by { λ T E 2 1/k 2m λ 2m 2 2m! 2m! E( ) 2m Z N θ m=0 m=0 = 2E exp ( ) ( λz N θ = 2E exp t 2 N θ /2 ) wth λ = t/. Thus, (2.9) holds. The proof of (2.10) from (2.9) s early detcal to the proof of the Berste equalty Chow ad Techer [(1988), page 111]. By (2.9) ad the Markov equalty, 212 { P k ε f Ỹ k/2 1/k t 2 exp λt (1 + θ { expλ 2 /2 1 ) for all λ>0. Take λ = t/θ + t/2. Sce λ 2 /2 < 2 ad e x 1 x/1 x/2 for 0 x<2, θ { expλ 2 /2 1 θλ2 /2 1 λ 2 /4 θλ2 /2 1 λ/2 = tλ 2 Thus, the rght-had sde of (2.12) s bouded by 2 exp tλ1 + tλ/2 2 exp tλ/2 ad the proof s complete. Proof of Corollary 2.4. Let θ = σ 2 /c 2 1 ad φ = φ Ỹ =θ k 1 c 2 f 2 Ỹ. The φ are sub-beroull fuctos of Ỹ wth parameter θθ, as the φ verso of (2.1) holds for >0 due to φ θ k 1 θ k ad for

10 STRONG LAWS FOR U-STATISTICS 441 = 0 due to Eφ θ k. Set λ = θc 1/k / θ. It follows from (2.10) that the left-had sde of (2.11) s bouded by { θ k/2 P θc k ε f Ỹ k/2 1/k t λ ( 2 exp t 2 /2 λ 2 θ + λt/2 Hece, (2.11) holds as λ 2 θ =θc 2 1/k = σ 2/k ad λ = σ 1/k / θ = c/σ k 1/k. ) 3. Momets of maxma ad sums. I ths secto we provde momet ad tal probablty equaltes for maxma ad sums of products [e.g., (1.4), (1.5), (1.12) ad (1.13)]. We shall provde the ormalzg kerels Secto 3.1, the equaltes the d ad multsample cases Secto 3.1 ad equaltes for depedet ot detcally dstrbuted varables Secto 3.3. Secto 3.4 cotas the proofs of Theorems 3.2 ad 3.4 Secto 3.2. We shall use the followg otato to shorte expressos: k =1k, a =a 1 a, ad for all a k ad A k wth sze A =, a l l A = a l1 a l wth l 1 <l beg the ordered labels A Costructo of ormalzg kerels. Let hy k be a oegatve Borel fucto ad Y k =Y 1 Y k be a radom vector wth ot dstrbuto F Yk. Gve θ k =θ 1 θ k, we shall fd a ormalzg kerel ψy k such that φy k =hy k /ψy k s a sub-beroull fucto of Y k wth parameter θ k. I addto, the ormalzg kerels should be small eough to be used the proof of equaltes both drectos such as (1.4) ad (1.5). We shall classfy the 2 k equaltes of (2.1) accordg to A, the sze of A, ad cosder those wth A =k, = k0. The ormalzg kerel s defed by ψy k =ψy k θ k hf Yk = max h y k =h 0 y k 0 k { h y k =h +1 y k max c A y l l A A=k A k = k 10, h k y k =hy k =c k y k, ad for A k ad A = k 33 ( c A yl l A c) ( = c A yl l A c ) θ k hf Yk { [ ] hy k = f c 0 E h +1 Y k c Y l = y l l A c θ l l A

11 442 C.-H. ZHANG Note that c 0 does ot deped o y k. Defe φ A y k =φ A y k θ k h F Yk by 34 φ A y k = hy k h +1 y k c A y l l A Lemma 3.1. Let θ k, h ad F Yk be as (3.1) (3.4). For all fxed y l l A, the fucto φ A y k (3.4) s a sub-beroull fucto of Y l l A wth the parameter θ l l A, ad Eφ A Y k Y l l A = l A θ l for c A Y l l A > 0. I partcular, φy k = h/ψ = φ 0k y k s a sub- Beroull fucto of Y k wth the parameter θ k. Proof. It mmedately follows from (2.1), (3.2) ad (3.3) that φ A y k s codtoally sub-beroull. Its mea s gve by (3.3) as the codtoal expectato o the rght-had sde s cotuous c for c > Momet equaltes the d ad multsample cases. We shall provde momet equaltes volvg maxma ad sums, exteded (1.4) ad (1.5) wth d X ad ther multsample versos wth depedet d sequeces X l, l 1. Let hx 1 x k be a fxed kerel. For k ad k l=1 1 l defe 35 S = hx S = h X Let θ k =θ 1 θ k ad θ be fxed parameters. Let F X be the ot dstrbuto of X =X 1 X k ad F Xk be the ot dstrbuto of X k = X 1 X k. Defe 36 ξ θ = max ψ ( ) X θθhf Xk ξ k = ξ k k/ 37 ξ θ k = max ψ ( ) X θ k hf X ad 38 where ψ s gve by (3.1). ξ = ξ k l=1 1 1/ l 1 1/ k ξ k = ξ k 1/1/ Theorem 3.2. Let g be a odecreasg oegatve fucto. The, for all k l=1 1 l ad tegers m 1, 39 E ( ) mg ( ) ( ) S / ξ k θ k ξ θ k Eg ξ θ k E ( ) m 1 + N l θ l l=1

12 STRONG LAWS FOR U-STATISTICS 443 Furthermore, for all = 1 k ad real umbers t>0 ad 0 <ε<1, 310 P { ξ >t P { S >εt ( 3 k 2 k ) 1 ε + 2 2k As (1.15), we have the followg corollary. Corollary 3.3. Let be a fucto satsfyg (1.14) for some α m. The, M2Vc α 1 E ξ 3 k 2 k /4 + 2 k E S M ( c α + { E1 + N 1 m k) E ξ Theorem 3.4. Let g be a odecreasg oegatve fucto. The, for all k ad tegers m 1, 311 E ( ( ) mg ( ( ) k + m N S /ξ θ) ξ θ) Eg ξ θ E θ k Furthermore, there exsts a fucto C k ε such that for all k ad real umbers t>0 ad 0 <ε<1, 312 P { ξ k For kk + 1, (1.13) holds wth C k ε = =0 >t C k ε P { S k >εt ( k k + 1 ){1 + E( N )2 k+1 k 1 1 ε 2 Remark. If s a multpler of k!, (3.12) holds for C k ε = k =0 )2 1 ( k ){1 + E( N k k 1 ε 2 Corollary 3.5. Let be a fucto satsfyg (1.14) for some α m. The, Eξ k ES k M2Vc α C ME( ξ k ) { ( ) k + m c α + E Nk k 1/2 k Theorems 3.2 ad 3.4 are proved Secto Geeral depedet varables. I ths secto, we cosder the expectatos of the product of a maxmum ad several sums for depedet varables Y Y l 1l 1, whch may have dfferet dstrbutos. Proposto 3.6. Suppose ψ Ỹ are measurable fuctos of Ỹ ad φ s Ỹ are sub-beroull oes wth parameters θ for k. Let k s =

13 444 C.-H. ZHANG k l=1 Il s for some Is l 1. The, for all odecreasg oegatve fuctos g, k ad m 1 ( ) m 313 Eg ξ φ s Ỹ Eg ( ) ξ k m { E 1 + Tl s k s l=1 where ξ = max ψ Ỹ ad T l s = δ l Is l. Proposto 3.7. Suppose ψ Y are measurable fuctos of Y ad φ s Y are sub-beroull oes wth parameters θ for k. Let k s = I k s k for some I s 1. The, for all odecreasg oegatve fuctos g, k ad m 1, ( ) m 314 Egξ φ s Y Eg ( ) m ( ) k + Ts ξ E k k s where ξ = max ψ Y ad T s = I s δ. Remark. For geeral k l l, we may apply Proposto 3.7 to φ Y = = φ Y /k! ad ψ Y =maxψ Y =, where = 1 k s regarded as a set. Proof of Proposto 3.6. Set ψ = ψ Ỹ ad φ s = φ s Ỹ. Before provdg the full proof of (3.13), we shall frst take a look at the case where k = 1 ad m = 2. Let ψ φ, 1, be depedet radom vectors wth 0 φ 1 ad Eφ = θ. Let be the dex at whch max gψ s reached. We have ( )( 1 )( E max gψ 2 ) φ 1 φ =1 Egψ φ =1 1 1 =1 1 =1 1 =1 1 =1 Egψ φ φ 2 I 2 2 Egψ φ φ 1 I 1 Egψ φ 1 φ 2 I 1 2 Sce g s odecreasg ad oegatve, { Egψ φ 1 φ 2 I 1 2 E max gψ Eφ 1 φ Egψ Eδ 1 δ 2

14 STRONG LAWS FOR U-STATISTICS 445 Smlarly, Egψ φ φ I Egψ Eδ ad Egψ φ 2 0 φ 1. Isertg these equaltes to (3.15), we obta 317 ( 1 )( 2 ) Egψ φ 1 φ 2 1 =1 1 =1 Egψ ( ) 1 + ET 1 + ET 2 + ET 1 T 2 Egψ E ( )( ) 1 + T T2 Egψ as where T = =1 δ. Ths s (3.13) for k = 1 ad m = 2. To proof (3.13) wth geeral k ad m, we shall compare the dces s = 1s k s, 1 s m, wth = 1 k products of the form gψ m φ s s, where s the dex at whch the maxmum ξ s reached. Let =l s 1 l k 1 s m. Gve A ad 1 m, defe A = l s l s A as a A-dmesoal vector of postve tegers. Set A = l s A Is l, whch s the space of combed labels A. Defe π A = π A 1 m =I { l s = l l s A whch dcate dfferet patters of match betwee 1 m ad. Ths facltates the calculato of sums volvg dfferet patters of match betwee ad. Sce A π A = 1, we have, as (3.15), 318 Egξ ( m s s φ s s ) = { m Egψ A = m Egψ A A A φ s s φ s s π A c where 1 m = ad s = 1s k s wth l s = l s for l s A ad l s = l for l s A, gve ad A. Gve A ad A, let o = o 1 o k be the dex at whch the maxmum of ψ s reached over the set A A = 1 k l l k l k A, ad s = 1s k s wth l s = l s for l s A ad l s = o l for l s A. Note that A A s the space of combed dces whch do ot volve the specfed A.Wehave m m Egψ φ s s Egψ o φ s s Let F A A be the σ-feld geerated by Y l l l l s l s A. Sce o s F A A measurable, for all = 1 k A A [ E gψ o m ] [ m φ s s F A A = gψ oe φ s s Y l l s ] l s A

15 446 C.-H. ZHANG o the evet { o =, where s = 1s k s wth l s = l s for l s A ad l s = l for l s A. Sce φ s are sub-beroull fuctos of Ỹ, by (2.3), [ m E φ ] s s Y l l s A E Thus, for the gve A ad A we have as (3.16), Egψ l s m φ s s Egψ oe l s A l s A δ l l s δ l l s Egψ E Isertg ths to (3.18), we fd as (3.17), ( ) m Egξ φ s s Egψ E s s A A A Egψ 319 E A Hece (3.13) holds for geeral k ad m. = Egξ E k l=1 l s A l s A l s A T l s m ( ) 1 + Tl s δ l l s Proof of Proposto 3.7. Let be the locato of the maxmum ξ. Smlarly to the proof of Proposto 3.6, we fd va Proposto 2.2 that Eg ( ) m ( ) ξ φ s Eg ( ) m ( ) ξ E δ k s A ss = Eg ( ( ) ) m Ts ξ E A where s = #l l s A. Sce there are ( k ) subsets of 1k of sze k, m A ( Ts s ) m = Ths completes the proof. k =0 ( )( ) k Ts = k k s ( m k + Ts 3.4. Proofs of Theorems 3.2 ad 3.4. We shall use Propostos 3.6 ad 3.7 to prove (3.9) ad (3.11). The Catell equalty below s appled to sums of sub-beroull varables Lemma 3.1 the proof of (3.10) ad (3.12). Lemma 3.8 (Catell equalty). Let W be a radom varable wth EW = µ ad VarW =σ 2. The PW ε µ ε 2 /σ 2 +µ ε 2 for all ε<µ. k δ l l s )

16 STRONG LAWS FOR U-STATISTICS 447 Proof of Theorem 3.2. Let φ = h X /ψ X θ k hf X. By Lemma 3.1, φ are sub-beroull varables wth parameter θ k. By (3.5) ad (3.7), S / ξ θ k φ, so that (3.9) follows drectly from Proposto 3.6. Let us prove (3.10). Let = 1 k ad A k be fxed wth A = k. Defe 320 ξ A = max { ( l c A X l l A ) l 1 l l A c where c A y =c A y 1/ 1 1/ k hf X as (3.3). Let l l A c be the dex at whch the maxmum (3.20) s reached. Defe 321 A = ( ) φ A X A wth φ A y k =φ A y k 1/ 1 1/ k hf X as (3.4), where A = { 1 k l 1 l l A l = l l Ac By Lemma 3.1, E A ξ A =1 for ξ A > 0, ad by Lemma 3.1 ad (2.5), E [ 2 ] A ξ A EN 2 l 1/ l ( EN1) 2 k = 2 k l A Thus, by Lemma 3.8 wth W = A P { A >ε ξ A 1 ε 2 2 k 1+1 ε 2 o the set ξ A > 0. Sce S / ξ A A by (3.4), ths mples P { ξ A >t 2k 1+1 ε 2 P { S 1 ε 2 >εt Sce ξ s the maxmum of ξ A over all A k ad there are ( k ) of these wth A =k, P { ξ >t P { S >εt k ( ) k 2 k 1+1 ε 2 1 ε 2 Ths completes the proof. =0 Proof of Theorem 3.4. Let φ = hx /ψx θθhf Xk. By Lemma 3.1, φ are sub-beroull varables. By (3.5) ad (3.6), S /ξ θ φ, so that (3.11) follows drectly from Proposto 3.7. We shall oly prove (3.12) for kk + 1 wth the explct C kε. The proof of (3.10) ca be used to prove (3.12) f ξ k ca be decoupled. Let us dvde 1 to k + 1 blocks B l as evely as possble. Let A k wth A =k ad blocks, say B 1 B, be fxed. Let ξ A be the maxmum

17 448 C.-H. ZHANG of c A X over l=1 B l, reached at 1, ad A be the sum as (3.21), wth fxed frst compoets of = 1 k, l = l l, over +1 k k k+1 l=+1 B l k. The, by Lemma 3.8 ad (2.8) P { A >ε ξ µ ε2 A v +µ ε2 where µ =k/k ( ) k ad v E ( N λ )2 µ k 2, wth beg the sze of k+1 l=+1 B l ad λ = k/. Cosder the smallest possble µ wth = sk+1+ ad = sk+1, B l =s+1for l, for some s. I ths case, k/ /k 1 by algebra for s k, whch holds as kk + 1. Thus, µ k/k /k k 1. I the case of largest possble λ, wth =s+1k+1 ad = sk+1+k+1 ad the the smallest possble s = k, kk + 1, wehaveλ = k/ k + 1. Therefore, 1+v E( N k+1 k )2 = 1+v, say. As the proof of Theorem 3.4, by Lemma 3.8, P { { ξa >t v 1 + P { S k 1 ε 2 >εt ) ways to select these Now, ξ k s the maxmum of ξ A over totally ( k+1 blocks ad the over = 0k, so that P { ξ k >t P { k ( ){ S k >εt k The proof s complete. =0 v 1 ε 2 4. SLLN. Let fx 1 x k be a real Borel fucto. Let f 2 x 1 x k = hx 1 x k. I ths secto we gve asc for the SLLN (1.3), ts multsample verso 41 S k /b k 0 a.s. ad ther symmetrzed versos S k T k /bk/2 0 a.s. where s gve by (1.7), ad T k assume throughout ths secto that b k 44 k sup 1 T k /b k/2 0 a.s. m= ad T k m k 1 b k m < are gve by (1.8). We shall We shall also assume that the fucto f s permutato varat, fx 1 x k =fx 1 x k for all 1 k k k.

18 STRONG LAWS FOR U-STATISTICS 449 Theorem 4.1. Let S k be gve by (1.7) ad ξ k by (3.8). Let ε > 0, 1 3. Let be a sequece of postve tegers such that 1 <γ 1 +1 / γ 2 <, 1. The (4.1) ad (4.3) are equvalet to each other ad to each ad all of the followg statemets: 45 ξ k /b k 0 as =1 P { S k >ε 1 b k < P { T k >ε 2 b k/2 < =1 =1 P { ξ k >ε 3 b k < Remark. These are the multsample versos of the SLLN, sce X 1 X k are allowed to have dfferet dstrbutos. Theorem 4.2. Let S k be gve by (1.1) ad ξ k by (3.6). Let ε > 0 ad be as Theorem 4.1. Suppose X X 1 X k are d radom varables. The, (1.3), (4.2) ad (4.6) (ad other statemets Theorem 4.1) are equvalet to each other ad to each ad all of the followg statemets: 49 ξ k /bk 0 as =1 =1 =1 P { S k >ε 1 b k < P { T k >ε 2 b k/2 < P { ξ k >ε 3 b k < We state Lemma 3.5 ad Proposto 4.2 of Zhag (1996) here as t s appled some crucal parts the proofs. Lemma 4.3. Let η be oegatve radom varables ad A be evets. The η I A I A0 η + I A c A +1 η = 0 = 0 = 0 =+1

19 450 C.-H. ZHANG For dsot sets of postve tegers A 1 A l, defe the sum of crossblock terms S k A 1 A l = hx I { A 1 A l k where A 1 A l s the set of vectors 1 k such that 1 k l=1 A ad 1 k A for all 1 l. For example, S k A /( A) k s the U-statstc based o the set of varables X A, where A s the sze of the set A. Proposto 4.4. Let A, 0 l, be dsot sets of postve tegers ad a be real umbers dexed by vectors = 1 k. The a I { A 1 A l A 0 A 1 A l = l 1 l =0 0<m 1 < <m l a I { 1 k A 0 A m1 A m for all sets of ftely may vectors. I partcular, S k A 1 A k = k 1 k =0 0<m 1 < <m k S k A 0 A m1 A m Note that ( k l=1 A l ) 1 k S A 1 A k are multsample U-statstcs. Proof of Theorem 4.1. We shall prove , ad WeuseM to deote a arbtrary postve costat. We may choose ay value of ε (large or small), sce (4.1) has othg to do wth the scalg. () By Lemma 3.8 wth W = T k 2 / S k gve S k Khtche equalty, {( T k ) 2 P 1/2 S k k S 1/ /4 ad the See the proof of (3.10) ad Gé ad Z [(1994), page 122] for detals. () See (3.10) Theorem 3.2. () Ths part s very close to the proof of Theorems 2.2 (suffcecy) ad 3.1 Zhag (1996). Let ε 3 = 1 ad ε>0. By the Doob

20 STRONG LAWS FOR U-STATISTICS 451 equalty for the reverse martgale T k { P max < +1 ( T k ) 2 b k max < +1 ( 2k +1 4 ε 2 b k / k,, codtoally o ξ k >ε 2 ξ k +1 1/ 1/ b k ( 2k ) E ε 2 b k ) ( T k E k max < +1 ( T k k ) 2 I { ξ k b k 4γ2k 2 E ( T k ) 2I { ε 2 b k ξ k b k = 4γ2k 2 k ε 2 b k Eh X 1 I { ξ k where X 1 =X 1 1 Xk 1. Thus, { ( P max T k ) 2/b k < >ε 2 +1 =1 b k ) 2 I { ξ k +1 b k P { ξ +1 1/ 1/ >b k k + M Eh X b k 1 I { ξ k b k By (4.8), the frst sum o the rght s fte. By (4.4) ad Lemma 4.3, k b k Eh X 1 I { ξ k b k, M + M k +1 Eh X b k 1 I { ξ k >b k ξ k +1 b k It follows from (4.8) ad (3.9) of Theorem 3.2 (wth ξ θ k It >b k ) that the rght-had sde above s bouded by = ξ k ad gt = M + M S k E I { ξ b k k +1 M + M M + M E >b k ξ k +1 b k +1 S k I { ξ ξ k k >b k P { ξ k >b k < (v) Ths s a cosequece of Proposto 4.4 ad the Borel Catell lemma. For detals, see Step 3 of the proof of Theorem 4.1 of Zhag (1996), page 1608.

21 452 C.-H. ZHANG (v) The proof s smpler tha (). (v) See (v). (v) The Borel Catell lemma. Proof of Theorem 4.2. By Proposto 4.4, 13 46, as (v) of the proof of Theorem 4.1. Sce hx 1 x k 0, by de la Peña ad Motgomery-Smth (1995). The proofs of , , ad , are detcal to those of , ad , respectvely. Proof of Theorem 1.1. () (1.18). Let 2 m < 2 +1 be the dex at whch Pξ k >b k reaches ts maxmum over 2 <2 +1. Takg = m 2 (4.12) wth ε 3 = 1 ad the = m 2+1,wefd P { ξ k >bk < =1 For k = 2 ad b 2 c 02/, hx 1 X 2 >b 2 c 1X 1 2/ c 1 X 2 2/ ad (1.9) mply φ X 1 X 2 =1, wth φ x y =hx y/ψx y 2/, so that the left-had sde of (1.18) s bouded by =2 Eφ X 1 X 2 I { ξ 2 = =2 2 1 E 2 >b2 φ X I { ξ 2 >b2 =2 2E ( ) 2+N P{ ξ 2 >b2 < The equalty above s a cosequece of Proposto 3.7, wth gx =Ix > b 2. Also, (1.16) follows from ξ2 /b 2 0 a.s. ad (1.17) follows from (4.13), as (4.13) mples 1 Pmax 1 c 1 X 2/ >b 2 <. () (1.16) By the Borel Catell lemma (1.16) We obta by takg m to be the dex of the mmum the block 2 <2 +1 the proof of (). 5. Examples. For 0 <p<2, Gé ad Z (1992) proved that EfX 1 X k p < s suffcet for (4.2) wth b = 1/p. Here we gve a example to show that the pth momet codto s some sese far away from ecessary. By the equvalece of (4.1) ad (4.3) ad the Kolmogorov ad Marckewcz Zygmud strog laws, we have the followg example. Example 5.1. Let Y l Y l 1 be depedet sequeces of d radom varables. Suppose EY l p l <,0<pl 2, 1 l k, p 1 + +p k < 2k. The k/p k l=1 =1 ε l Y l 0 a.s., where k/p = 1/p /p k. Example 5.2. Take 0 <p<2 ad set p 1 = p/k pk 1/2 <p. Defe fx 1 x k = k x 2/p 1 k 1 x 2 x k, the permutato symmetrzed verso of

22 STRONG LAWS FOR U-STATISTICS 453 the kerel x 2/p 1 1 x 2 x k. Let X be a oegatve varable wth EX 2 < but EX 2+ε = for all ε>0. The k/p k ε 1 ε 2 ε k fx 1 X k 0 a.s., whle EfX 1 X k p1+ε = for all ε>0. Proof. It s clear that EfX 1 X k p1+ε = for all ε>0. By the equvalece of (4.2) ad (4.3), t suffces to show k/p k ε 1 ε 2 ε k fx 1 1 X k k 0 a.s., whch s a cosequece of Example 5.1 wth Y 1 = X 1 2/p 1, Y l = X l for 2 l k ad p 2 = =p k = 2. Ackowledgemet. I thak the referee for may valuable suggestos. REFERENCES Arcoes, M. ad Gé, E. (1993). Lmt theorems for U-processes. A. Probab Chow, Y. S. ad Techer, H. (1988). Probablty Theory. Sprger, New York. Cuzck, J., Gé, E. ad Z, J. (1995). Laws of large umbers for quadratc forms, maxma of products ad trucated sums of..d. radom varables. A. Probab de la Peña, V. ad Motgomery-Smth, S. J. (1995). Decouplg equaltes for the tal probabltes of multvarate U-statstcs. A. Probab Gé, E. ad Z, J. (1992). Marckewcz type laws of large umbers ad covergece of momets for U-statstcs. I Probablty Baach Spaces 8 (R. M. Dudley, M. G. Hah ad J. Kuelbs, eds.) Brkhäuser, Bosto. Gé, E. ad Z, J. (1994). A remark o covergece dstrbuto of U-statstcs. A. Probab Gleser. L. J. (1975). O the dstrbuto of the umber of successes depedet trals. A. Probab Hoeffdg, W. (1961). The strog law of large umbers for U-statstcs. Isttute of Statstcs Mmeo Ser. 302, Uv. North Carola, Chapel Hll. Klass, M. J. ad Nowck (1997). Order of magtude bouds for expectatos of 2 -fuctos of o-egatve radom blear forms ad geeralzed U-statstcs. A. Probab Klass, M. J. ad Zhag, C.-H. (1994). O the almost sure mmal growth rate of partal sum maxma. A. Probab Ledoux, M. ad Talagrad, M. (1991). Probablty Baach spaces. Sprger, New York. Serflg, R. J. (1980). Approxmato Theorems of Mathematcal Statstcs. Wley, New York. Se, P. K. (1974). O L p -covergece of U-statstcs. A. Ist. Statst. Math Techer, H. (1992). Covergece of self-ormalzed geeralzed U-statstcs. J. Theoret. Probab Zhag, C.-H. (1996). Strog laws of large umbers for sums of products. A. Probab Rutgers Uversty Departmet of Statstcs Hll Ceter for Mathematcal Sceces Busch Campus Pscataway, New Jersey E-mal: czhag@stat.rutgers.edu