Almost sure limit theorems for U-statistics

Almost sure limit theorems for U-statistics Hajo Holzmann, Susanne Koch and Alesey Min 3 Institut für Mathematische Stochasti Georg-August-Universität Göttingen Maschmühlenweg 8 0 37073 Göttingen Germany Abstract We relax the moment conditions from a result in almost sure limit theory for U-statistics due to Beres and Csai (00). We extend this result to the case of convergence to stable laws and also prove a functional version. MSC: 60F05, 60F7 Keywords: Almost sure limit theorem; Functional limit theorem; Stable distributions; U-statistics Introduction U-statistics generalize the concept of the sample mean of independent identically distributed (i.i.d.) random variables. The statistical interest in U-statistics stems from the fact that they form a class of unbiased estimators of a certain parameter with minimal variances. We begin by introducing some notation and recalling the concept of U-statistics. Let X, X,... be i.i.d. random variables with common distribution function F (x). Let m and let h : R m R be a measurable function symmetric in its arguments. The U-statistic with ernel h is defined by ( ) n U n (h) = h(x i,..., X im ), n m. m i <i <...<i m n Hajo Holzmann acnowledges financial support of the graduate school Gruppen und Geometrie Susanne Koch, Institut für Mathematische Stochasti, Maschmühlenweg 8-0, 37073 Göttingen, GERMANY 3 Alesey Min acnowledges financial support of the graduate school Strömungsinstabilitäten und Turbulenz

The ernel h is called degenerate with respect to F (x) if for all j m h(x,..., x m )df (x j ) 0, where < x,..., x j, x j+,..., x m <. Let R and for i = 0,..., m let θ = Eh(X,..., X m ) h i (x,..., x i ) = Eh(x,..., x i, X i+,..., X m ), i h i (x,..., x i ) = ( ) i h (x j,..., x j ), =0 (j,...,j ) {,...,i} ζ i = Var(h i (X,..., X i )). Note that h 0 = θ and h m (x,..., x m ) = h(x,..., x m ). Furthermore, the h i are degenerate for i =,..., m (see Dener (985)). If ζ > 0 the U-statistic U n (h) is called non-degenerate and degenerate otherwise. The smallest integer c for which ζ c > 0 is called the critical parameter of the U-statistic U n (h). Without loss of generality we will assume θ = 0 throughout the rest of this article. The theory of U-statistics started to develop intensively after Hoeffding s fundamental article (948). He showed asymptotic normality of non-degenerate U-statistics under the assumption Eh (X,..., X m ) <, () using the following representation of U-statistics U n (h) = mu n (h ) + m = ( ) m U n (h ), () where mu n (h ) is a sum of i.i.d. random variables and U n (h ),..., U n (h m ) are U- statistics with degenerate ernels. In the degenerate case n c/ U n (h) wealy converges to a multiple Wiener integral whenever h is square integrable (see e.g. Dener (985)). Here c is the critical parameter of U n (h). In the late 80s Brosamler (988) and Schatte (988) independently proved a new type of limit theorem. This type of statement extends the classical central limit theorem in the i.i.d. case to a pathwise version and is therefore called an almost sure central limit theorem (ASCLT). In the 90s many studies were done to prove almost sure limit theorems (ASLT) in different situations, for example in the case of independent but not necessarily identically distributed random variables (see Beres and Dehling (993)). Excellent surveys on this topic may be found in Atlagh and

Weber (000) as well as in Beres (998). Recently Beres and Csai (00) obtained a general result in almost sure limit theory. They used it to prove almost sure versions of several classical limit theorems. In particular they stated the following theorem for U-statistics. Theorem.. Let c be the critical parameter of the U-statistic U n (h). Under the assumption () lim n log n = l { c/ U (h)<x} = G(x) a.s. for any x C G, where G is the limit distribution of n c/ U n (h) and C G denotes the set of continuity points of G. In the present note we relax the moment condition in Theorem. and extend the statement in two directions. First we will obtain an ASLT with stable limiting distribution for a non-degenerate U-statistic. Furthermore we extend Theorem. to a functional version. Preliminaries Let (Y n ) n be a sequence of random elements taing values in a Polish space (S, d) and let G be a probability measure on the Borel σ-field in S. We say that (Y n ) n satisfies the ASLT with limiting distribution G if with probability, (log n) δ Y / G, n. = Here δ Y is the Dirac measure at Y and denotes wea convergence of measures. Throughout this note the following lemma will be of fundamental interest. Lemma.. Let (Y n ) n be a sequence of S-valued random elements which satisfies the almost sure limit theorem (ASLT) with some limiting distribution G. Assume that Z n is another sequence of S-valued random elements on the same probability space such that almost surely, d(y n, Z n ) 0. Then Z n also satisfies the ASLT with limiting distribution G. Proof. By a well nown principle in almost sure limit theory (see e.g. Lacey and Philipp (990)), (Y n ) n satisfies the ASLT with limiting distribution G if and only if (log n) Ψ( Y (ω) ) Ψ(x) dg(x) a.s. S = 3

for any bounded Lipschitz function Ψ. Using the Lipschitz property of Ψ and the assumption that d(y n, Z n ) 0 we conclude that log n = [ Ψ ( Y (ω) ) Ψ ( Z (ω) )] C log n = d( Y (ω), Z (ω) ) 0 where C is a Lipschitz constant for Ψ. This proves the Lemma. a.s., In the sequel we will mae use of the following lemma which is a consequence of a more general result due to Giné and Zinn (99). Lemma.. Let h(x,..., x l ) be measurable and degenerate. Let q ( l, l). If E h(x,..., X l ) l/q <, (3) then with probability n q h(x i,..., X il ) 0. i <...<i l n 3 Relaxing the moment assumption In this section we will relax the moment assumption of Theorem.. For the wea convergence of n c/ U n (h) (where c denotes the critical parameter of U n (h)) Korolyu and Borovsich (994) weaened the assumption () to E h (X,..., X ) /( c) <, = c,..., m. (4) We are going to prove the validity of Theorem. under these assumptions: Theorem 3.. Let c be the critical parameter of the U-statistic U n (h). If (4) is satisfied then the statement of Theorem. is true. Proof. First of all note that, if c is the critical parameter of U n (h), then the functions h i (x,..., x i ) = 0 a.s. for all i =,..., c, and so from the Hoeffding decomposition n c/ U n (h) n c/ ( m c ) U n (h c ) = n c/ m =c+ ( ) m U n (h ). (5) By Theorem., n c/( m c ) Un (h c ) satisfies the ASLT. To complete the proof it suffices to show that the sum on the right-hand side of (5) converges to zero a.s. This follows from Lemma. by letting l = and q = c/ for = c +,..., m. An application of Lemma. finishes the proof. 4

4 Convergence to stable distributions As we shall see in this section, under some mild moment conditions wea convergence of a sequence of non-degenerate U-statistics to a stable limit distribution implies the validity of the corresponding ASLT. Let G α denote a stable law with characteristic exponent α. Theorem 4.. Let for some α (, ] n α ml(n) U n(h) A n G α, (6) where L(n) is a slowly varying function for which lim inf n L(n) > 0. If then α E h (X,..., X ) α( )+ <, =,..., m, (7) lim n log n = l{ } = G /α ml() U (h) A <x α (x) a.s. Remars () Assumption (6) is very common in almost sure limit theory, when one deduces an ASLT from the validity of the corresponding wea limit theorem (see e.g. Beres and Dehling (993)). () One has wea convergence in (6) if the distribution function of h (X ) belongs to the domain of attraction of G α and if the moment condition E h(x,..., X m ) α α+ < (8) holds (see Heinrich and Wolf (993)). (3) It is not difficult to show that if the ernel h satisfies (8), then (7) holds. (4) Theorem 4. will be true for any slowly varying function, if E h (X,..., X ) p < for some p > α α( )+, =,..., m. Proof of Theorem 4.. Let us start by showing that Indeed n α ( Un (h) mu n (h ) ) 0 a.s. (9) ml(n) n α ( Un (h) mu n (h ) ) = ml(n) m = ( ) m n α ml(n) U n(h ), and letting q = + and l = we can apply Lemma.. This proves (9). α Maing use of (6) and (9) we conclude that also n α L(n) U n(h ) A n G α. (0) 5

It is nown that wea convergence of normalized sums of real valued i.i.d. random variables to some stable law G α implies the corresponding ASLT (see Ibragimov and Lifshits (999)). Hence (0) implies lim n log n = l { α = G α (x) a.s. L() U (h ) A <x} An application of Lemma. completes the proof. 5 Functional version of the ASLT 5. The non-degenerate case In the non-degenerate case the functional version of Theorem. can be deduced directly from Theorem of Lacey and Philipp (990), which deals with sums of i.i.d. random variables. Throughout this section we assume that () holds. Let D[0, ] denote the space of cadlag functions on [0, ] and let W denote the Wiener measure on D[0, ] (see Billingsley (999)). We introduce the following D[0, ]-valued random functions { (m nζ ) Y n (t) = nt U nt (h) : t [m/n, ], 0 : t [0, m[, () n where denotes the integer part of a real number. Theorem 5.. Suppose that Eh (X,..., X m ) < and ζ > 0, then the sequence of random functions defined in () satisfies the ASLT with limiting measure W. Proof. First we want to see that, under suitable normalization, the difference between U n (h) and its first term in the Hoeffding decomposition () tends to 0 a.s. Since m ( ) m U n (h) mu n (h ) = U n (h ), and all ernels h c on the right-hand side are degenerate with degree c, we can apply Lemma. with q = c / and l/q = 3/ to conclude that n / (U n (h) mu n (h )) 0 a.s. Then also Let n / = max ( U (h) mu (h ) ) 0 a.s. () m n Z n (t) = nζ nt U nt (h ), t [0, ]. Note that nu n (h ) = n i= h (X i ) is a sum of i.i.d. random variables. From the ASCLT for i.i.d. random variables (see Theorem in Lacey and Philipp (990)), the Z n (t) satisfy the ASLT with limiting measure W. From () we deduce that Y n Z n 0, where. denotes the supremum norm on D[0, ]. Hence we can apply Lemma. to Y n (t) and Z n (t) to prove the Theorem. 6

5. The degenerate case In order to obtain a functional ASLT in the degenerate case, we need the following version of a general result by Beres and Csai (00) for function valued random elements. Theorem 5.. Let X n, n, be independent random variables taing values in some measurable space (E, B) and let f l : E l D[0, ], l N, be measurable mappings such that f l (X,..., X l ) G for some distribution G on D[0, ]. Assume that for each < l, where l m for some fixed m, there exists a mapping f,l : E l D[0, ] such that E ( f l (X,..., X l ) f,l (X +,..., X l ) ) C ( l ) α for some C, α > 0. Then the sequence ( f l (X,..., X l ) ) l limiting distribution G. satisfies the ASLT with Proof. The proof proceeds along the lines of the proof in Beres and Csai (00). From now on we restrict ourselves to ernels of degree. In this case the distribution invariance principle for degenerate U-statistics was obtained by Neuhaus (977) (for the general case see Dener, Grillenberger and Keller (985)). It states that for a degenerate ernel h : R R the sequence of D[0, ] valued random elements converges in distribution to n i<j nt ( λ w (t) t ), h(x i, X j ) (3) where the w are independent Brownian motions and the λ are the eigenvalues of the integral operator associated with h (see Neuhaus (977)). The distribution of this limiting process will be denoted by G. Theorem 5.3. Assume that the ernel h : R R is degenerate and satisfies (). Then the sequence defined in (3) satisfies the ASLT with limiting distribution G. Proof. We want to apply Theorem 5.. Denote h = Eh (X, X ) and let f l (x,..., x l )(t) = l f,l (x +,..., x l )(t) = l 7 i<j lt + i<j lt h(x i, x j ), h(x i, x j ),

where l, t [0, ] and the empty sum is 0. By Theorem 5., it will be enough to show that E ( f l (x,..., x l ) f,l (x +,..., x l ) ) C/l. (4) Evidently [ f l (x,..., x l ) f,l (x +,..., x l ) 4l E + max n<+ max + n l ( i<j n ( n h(x i, x j ) ) h(x i, x j ) ) ] where n denotes summation over all indices i, j n with i < j. Observe that due to the degeneracy of h, n h(x i, X j ), + n l, is a martingale with respect to the canonical filtration F n = σ(x,..., X n ). Using Doob s inequality and once more the degeneracy of h we estimate [ ( E n h(x i, X j ) ) ] 4E ( h(x i, X j ) ) l Similarly max + n l [ E max n<+ ( i<j n From (5) and (6) it follows that = 4 l Eh (X i, X j ) 4l h. (5) h(x i, x j ) ) ] 4 ( + E ( f l (x,..., x l ) f,l (x +,..., x l ) ) 4l 4 h The Theorem is proved. Acnowledgements C/l. ) h. (6) ( l + ( )) + We are grateful to H. Dehling for drawing our attention to a helpful publication by E. Giné and J. Zinn. 8

References Atlagh, M. and Weber, M. (000), Le théorème central limite presque sûr, Expositiones Mathematicae 8, 97 6. Beres, I. (998), Results and problems related to the pointwise central limit theorem, in B. Szyszowicz, ed. Proc. Internat. Conf. on Asymptotic Methods in Probability and Statistics (Elsevier, Amsterdam), pp. 59 96. Beres, I. and Csai, E. (00), A universal result in almost sure limit theory, Stochastic Processes and their Application 94, 05 34. Beres, I. and Dehling, H. (993), Some limit theorems in log density, Annals of Probability, 640 670. Billingsley, P. (999), Convergence of Probability Measures (Wiley, New Yor, nd ed.). Brosamler, G.A. (988), An almost everywhere central limit theorem, Math. Proc. Camb. Phil. Soc. 04, 56 574. Dener, M. (985), Asymptotic Distribution Theory in Nonparametric Statistics (Vieweg & Sohn, Braunschweig/Wiesbaden). Dener, M., Grillenberg C. and Keller G. (985), A note on invariance principles for v. Mises statistics. Metria 3, 97 4. Giné, E. and Zinn, J. (99), Marciniewicz type laws of large numbers and convergence of moments for U-statistics, in: R.M. Dudley, M.G. Hahn and J. Kuelbs, eds. Probability in Banach Spaces, 8: Proceedings of the 8th Internat. Conf. (Birhäuser) pp. 73-9. Heinrich, L. and Wolf, W. (993), On the convergence of U-statistics with stable limit distribution, Journal of Multivariate Analysis 44, 6 78. Hoeffding, W. (948), A class of statistics with asymptotically normal distribution, Ann. Math. Statist. 39, 35 346. Ibragimov I.A. and Lifshits, M.A. (999), On almost sure limit theorems, Theory Probab. Appl. 44, 54 7. Lacey, M. and Philipp, W. (990), A note on the almost sure central limit theorem, Statistics and Probability Letters 9, 0 05. Korolju, V.S. and Borovsich, Yu.V. (994), Theory of U-Statistics (Kluwer Academic Publishers, Dordrecht-Boston-London). Neuhaus, G. (977), Functional limit theorems for U-statistics in the degenerate case, J. Multivariate Anal. 7, 44 439. Schatte, P. (988), On strong versions of the central limit theorem, Math. Nachr. 37, 49 56. 9