A More General Central Limit Theorem for m-dependent. Random Variables with unbounded m. Los Angeles, CA May Abstract

Size: px

Start display at page:

Download "A More General Central Limit Theorem for m-dependent. Random Variables with unbounded m. Los Angeles, CA May Abstract"

Lorraine Holmes
5 years ago
Views:

1 A More General Central Limit Theorem for m-dependent Random Variables with unbounded m Joseph P. Romano Department of Statistics Stanford University Stanford, CA Michael Wolf Department of Statistics UCLA Los Angeles, CA May 1997 Abstract In this article, a general central limit theorem for a triangular array of m-dependent random variables is presented. Here m may tend to innity with the row index at a certain rate. Our theorem is a generalization of previous results. Some examples are given that show that the generalization is useful. In particular, we consider the limiting behavior of the sample mean of a running average process, the limiting behavior of a spectral estimator, and the moving blocks bootstrap distribution. The examples make it clear the consideration of asymptotic behavior with the amount of dependence m increasing with n is useful even when the underlying processes are weakly dependent (or even independent), because certain natural statistics that arise in the analysis of time series have this structure. KEY WORDS: Central limit theorem, m-dependent random variables. Joseph P. Romano, Phone: (415) , Fax: (415) , romano@stat.stanford.edu.

2 1 Introduction Central limit theorems for m-dependent random variables (with m xed) have been proved by Hoeding and Robins (1948), Diananda (1955), Orey (1958) and Bergstrom (1970). Berk (1973) proved a theorem for the case of a triangular array with unbounded m, that is, m may be a function of the row index and tend to innity at a certain rate. However, his theorem is somewhat limited by the fact that it only allows for a moderate amount of positive dependence. The variance of the sum of the random variables in the n-th row eventually needs to be of the order of the corresponding sample size. Our theorem is intended to extend Berks' result and allow for stronger dependency structures as well. The point of this paper is to present the theorem and illustrate with a number of examples. We would like to note, however, that it will be a necessary tool for developing inferential procedures for moderately dependent and long-range dependent random variables; these are research topics under current investigation. The paper is organized as follows. Section contains the main theorem. Some examples are presented in Section 3. These include: an example to clearly show how the result generalizes that of Berk (1973); the sample mean of a moving average process; estimates of the variance of the sample mean, and the corresponding spectral estimates; the moving blocks bootstrap distribution. The proof of the theorem appears in the Appendix. The Theorem Theorem.1 Let f n;i g be a triangular array of mean zero random variables. For each n = 1; ; let d = d n, let m = m n, and suppose n;1 ; : : :; n;d is an m-dependent sequence of random variables. Dene a+k?1 Bn;k;a V ar( n;i ); i=a B n B n;d;1 V ar( d n;i ): Assume the following conditions hold. For some > 0 and some?1 < 1: E j n;i j + n for all i; (1) B n;k;a =(k1+ ) K n for all k m; a; () B n =(dm ) L n ; (3) K n =L n = O(1); (4) + n =Ln = O(1); (5) 1

3 m 1+(1?)(1+=) =d! 0: (6) Then B?1 n ( n;1 + + n;d ) =) N(0; 1): Remark.1 Note that our theorem extends the previous result by Berk (1973) in two ways. Berk essentially proved this theorem for the special case = 0. Condition (ii) of his theorem corresponds to assumption () of our theorem with replaced by 0. The greater generality of our theorem is needed to accommodate stronger dependency structures. For example, it can handle the situation of V ar( n;1 ; : : :; n;d ) d 1+, for positive < 1. Note that for > 0 the condition on m in (6) becomes weaker. It is therefore desirable to have this extension, rather than making = 0 work by a proper standardization of the n;i sequence. Secondly, unlike Berk's theorem, our conditions permit the bounding constants in (1), (), and (3) to depend on the row index n. On the one hand, this allows for greater ease of using the theorem. For example, to satisfy the moment bound (1), one othwerwise might have to worry about a proper standardization of the n;i in the rst place. On the other hand, and more importantly, it might result in weaker conditions on m again. If the bound L n in (3) tends to innity with n, then the power ( + )= in (5) can play a crucial rule. Berk (1973) also gave an example demonstrating the sharpness of his result. This means that assumption (6) cannot be relaxed. Indeed, for the sequence of the example in his paper, assumptions (1) to (3) are satised (with = 0), but the limit of m 1+= =d is 1 rather than 0 and there is no asymptotic normality. We will illustrate our theorem with some examples in the next section. They show that the greater generality of our theorem is useful. Example 3.1 utilizes the extension for > 0. Example 3.3 benets from the fact that the moment bounds n, K n, and L n may depend on the row index n. 3 Examples Example 3.1 (A possibly unfair coin and a denitely unfair owner) Suppose that a professor for the history of gambling has learned of the recent discovery of a valuable coin that was used in the casinos of the pharaohs in ancient Egypt. The professor is interested in whether the casino might have changed the odds to its favor by using a biased coin. Denote the unknown probability of heads by p. The problem is that the owner of the coin will let nobody touch his dearest possession. He agrees to toss the coin a number d of times and report the result to the professor for a large amount of money. Since he suspects that each toss will result in some small damage to the coin, he decides on the following strategy of cheating

4 in hope of getting away with fewer actual than reported tosses. He will repeat the following experiment h times. For 1 i h: Toss the coin j times. Denote the number of heads by H i. Write down lh i, as number of heads in lj pseudo tosses. In the end he will report = l P h H i as the number of heads, pretending to have tossed the coin d = hlj times. Of course, in reality he only will have tossed the coin hj times. We are interested in conditions on j; l, and h that ensure that, properly standardized, will still have an asymptotic normal distribution. To put this scenario in the framework of our theorem we may pretend that the owner reports an entire sequence of length d = hlj of partly fake tosses to the professor. The rst lj reported tosses will be the outcome of the rst j real tosses, repeated l times possibly in some permuted order. The second lj reported tosses will be the outcome of the second j real tosses, and so on. Denote the i-th reported toss by the Bernoulli variable Y n;i. We then dene n;i = Y n;i? p. Thus, in the notation of the theorem, m = lj and d = hlj. We are interested in the rates at which j; l and h can tend to innity to ensure that the reported proportion of tosses ^p still has a limiting normal distribution. Of course, we dene ^p = =d. Since j n;i j 1, we can choose n 1, independent of n and for all. Next we note that But also, Therefore, we can pick B n;m;1 = l j p (1? p) l p (1? p) = m 1+ (lj) 1+ (lj) : Bn dm = hb n;m;1 hm 1+ K n = L n = = B n;m;1 l p (1? p) = m1+ (lj) : l p(1? p) (lj) ; and condition (4) will be satised trivially. Since n 1 we need L n to be bounded away from zero to satisfy condition (5). This will be implied by (lj) l = O(1): (7) Clearly, (7) is immediately satised for 0. In case > 0, we can be more specic if we know more about the relationship between j and l. Two particular choices will be discussed below. Finally, condition (6) simplies to m 1+(1?)(1+=) d = m1+(1?)(1+=) mh = m(1?)(1+=) h = (lj)(1?)(1+=) h! 0: (8) 3

5 Note that we can choose as large as we like, so if > 0 it will be enough to have To derive the limiting distribution exactly, we note that lj h! 0: (9) B n = hb n;m;1 = hl jp (1? p): The theorem therefore tells us that if we let ^p = =d, d 1 (^p? p) =) N(0; p (1? p)); (10) l as long as (7) and (8) are met. We can be somewhat more specic for two particular choices of the relationship between l and j. 1. If l behaves like a power of j, then a sucient condition for the CLT (10) to hold is simply that lj=h! 0. Assume l / j, for some > 0. Indeed, to check (7) we note that (lj) l / j (1 + ) j ; which will be satised for = =(1 + ) > 0. As noted before, with > 0, (8) is implied by lj=h! 0.. The case where we keep l xed and let only j tend to innity is not very interesting. The reversal, keeping j xed and letting l tend to innity, however, is. In that case, (7) will be satised for any < 1 and therefore it is again sucient to have lj=h! 0, which simplies to l=h! 0. The CLT (10) again follows. Example 3. (Running mean) Suppose we are using the running mean (a special case of a moving average) to smooth an observed sequence. Denote the original sequence by fy i g and the smoothed sequence by f i g. A very simple model is that the Y i are i.i.d. from a distribution with mean zero and nite + moment, for some > 0. Denote the variance of Y i by Y. It is natural to let the amount of smoothing increase with the sample size of the observed sequence. The question we ask is at what rate we can do this if we want the CLT to hold for the smoothed sequence. To be more precise dene n;i = 1 m i+m= j=i?m=+1 Y j ; 1 i d; assuming without loss of generality that m is even. This example can obviously be generalized to the case where the n;i are a weighted average of the Y j. As always, we think 4

6 of d and m as functions of the row index n, without making it obvious in the notation. By our denition, m corresponds to the amount of smoothing. By the triangle inequality, we can choose n E jy 1 j + as a very crude bound, independent of n. We note that V ar( m n;i ) Y m?1 h=0 (m? h) m Y m: Therefore, condition () is satised with = 0 and K n Y m. Analogously, V ar( d n;i ) Y m?1 h=0 (d? h)(m? h) m Const: d; so condition (3) is satised with = 0 and L n 1 Y for large n. Condition (6) then requires that m 1+= =d! 0, which tells us at what rate we may increase the amount of smoothing with the sample size of the observed sequence. The answer depends on the in general unknown. If we admit moments of any order, the condition becomes m=d! 0. In any case however, a constant rate, like \10% of the sample size", is clearly not allowed by the theorem. Finally, conditions (4) and (5) are trivially satised, since neither n nor K n nor L n depend upon n, at least for large n. To derive the explicit limiting distribution, note that we can write B n = V ar( d d?1 n;i ) = s n;0 + s n;l where s n;l l=1 n?l Cov( n;i ; n;i+l ): Due to the i.i.d. structure of the underlying fy i g sequence, it is easily seen that Cov( n;i ; n;i+l ) = m? l m Y for l m, and 0 otherwise: Therefore, as long as m=d! 0, we have B n = d Y ( 1 m + m m? l ) + o(1) m l=1 = dy + o(1): The theorem thus tells us that, given m 1+= =d! 0, we have d 1= n; =) N(0; Y ). Example 3.3 (Variance of the Sample Mean and Spectral Estimation) Consider the problem of estimating the variance of the (standardized) sample mean of a strictly stationary time series. Assume that f: : :; Z?1 ; Z 0 ; Z 1 ; : : :g is an innite sequence of strictly stationary random 5

7 variables having, without loss of generality, mean zero. The sample mean based on observing the nite segment Z 1 ; : : :; Z n is dened as Z n = n?1 P n Z i. The variance of n 1= Zn is given by n V ar(n1= Zn ) = (0) + n?1 (1? i n ) (i); where (i) = Cov(Z 1 ; Z 1+i ) is the i-th autocovariance of the underlying time series. Note that n! f(0), where f() is the spectral density function of the Z i process. Thus, the problem of estimating n is asymptotically equivalent to estimating the spectral density function at zero. The considerations below apply equally well to the problem of spectral density estimation. A naive estimate of n would be given by n?1 ^ n;naiv = ^(0) + (1? i n ) ^(i); where ^(i) is an estimate of the i-th autocovariance dened as ^(i) = 1 n n?i j=1 Z j Z j+i : However, it is well known that the naive estimator ^ n;naiv is inconsistent, e.g., Rosenblatt (1985). This is basically due to the fact that estimates ^(i) for i close to n? 1 are not reliable, since they are based on very few observations. To ensure consistency, the higher-order autocovariance estimates need to be downweighted in a sucient way. A very simple downweighting scheme is to assign weight zero to all ^(i) for i > m, where m is a truncation point depending on the sample size n, and weight one to all ^(i) for all i m. As before, we implicitly assume the dependence of m on n to be understood rather than using the notation m n. The truncation estimator of n is then dened as ^ n;t runc = ^(0) + m ^(i): (11) It is well known that under suitable conditions this estimator will be consistent and asymptotically normal for general stationary sequences; for example, see Rosenblatt (1985). The point of this example is to give a very simple alternative proof using Theorem.1 for the special case of i.i.d. random variables fz i g. Of course, this may be considered a toy example. The point is that even in the simple i.i.d. case there are certain restrictions at what rate the truncation point m may increase with the sample size n. Naturally, we assume that the practitioner trying to estimate the variance of the sample mean assumes stationarity of the underlying sequence, not knowing about the actual independence structure. Obviously, in this case we have n = = V ar(z 1 ) for all n. Note that this example could be easily extended to the somewhat 6

8 more interesting case where fz i g is a sequence of ~m-dependent random variables. This would come at the expense of an additional computational burden only. To apply the theorem it will be helpful to rearrange the terms in (11) in the following way. n ^ n;t runc = n;1 + n; + : : : + n;n ; (1) where n;i = Zi + Z iz i+1 + Z i Z i+ + : : : + Z i Z i+m : We have multiplied ^ n;t runc by n here for notational convenience later on. Note that we dene Z j 0 for j > n. This arrangement is useful for our purposes, since the n;i are easily seen to be m-dependent random variables even in case the Z i are independent. Note that if we considered the case of ~m-dependent Z i, the n;i would be m + ~m-dependent random variables. Proposition 3.1 Let Z i be a sequence of i.i.d. random variables with mean zero and variance. Assume that for some > 0, max(e jz i j + ; E jz i j 4+ )? < 1; m 1+= =d! 0: Then ( n m ) 1 (^ n;t runc? ) =) N(0; 4 4 ). Proof: We need to check the conditions of Theorem.1. Note that for this example d = n in the notation of the theorem. Remember from (1) that our set-up is for n ^ n;t runc, rather than ^ n;t runc. With the denition ky k p = (E jy j p ) 1=p, for p > 0, we rst note that k n;i k + = Zi + Z i Z i+1 + : : : + Z i Z i+m + Z i + + kz i k + kz i+1 + : : : + Z i+m k + (due to independence)? 1 + +? 1 + C +? 1 + (m? 1) 1 : The last inequality follows from Lemma A.1 which states a moment bound for independent random variables and is given in the appendix. Here, C + is a constant that only depends on +. If we assume, without loss of generality, that both? 1 and C + 1, we therefore have The important fact is that n = O(m + ). E j n;i j + + C + +? m + n : (13) Next, we have to look at the B n;k;a, the variance of n;a + : : : + n;a+k?1. Since the time series fz i g is assumed to be i.i.d., we may take without loss of generality a = 1. Dene s k;l = k?l Cov( n;i ; n;i+l ): 7

9 Looking at the variance of a block of size k m we then have B n;k;1 = V ar( k k?1 n;i ) = s k;0 + s k;l : Since the Z i are i.i.d. mean zero random variables, it is easy to see that Cov( n;i ; n;i+l ) = 0 for l 1 which implies that s k;l = 0 for l 1. On the other hand, l=1 Cov( n;i ; n;i ) = + 4 min(m? 1; n? i) 4 ; where = V ar(z 1 ) and = V ar(z 1 ). With 1 = max( ; 4 4 ) we therefore get B n;k;1 = s k;0 mk 1 : (14) Hence, for 0, we may choose K n = m 1? 1. By the same reasoning we nd, with = min( ; 4 4 ), that B n = B n;n;1 = s n;0 1 mn : (15) To be more precise, since we need this result for the limiting variance of ^ n;t runc, B n = nm( m + 4 m? 1 m 4 ) + o(1): (16) And so we can choose L n = 1 m1?. Since both K n and L n are O(m 1? ) as long as 0, condition (4) is trivially satised. To check condition (5), we note that + n =Ln = Const: m + + (1?) m For 0, this ratio will be O(1). Conditions (4) and (5) together therefore will be met for = 0. Thus, nally, condition (6) will be satised for m 1+= =n! 0. Remember that d = n for this example. From (16) we then know that the proper normalizing constant for n ^ n;t runc is (nm)?1=, so the proper normalizing constant for ^ n;t runc has to be (n=m)1=. (16) also tells us that the limiting variance is 4 4. : Example 3.4 (Moving Blocks Bootstrap) The moving blocks bootstrap was introduced by Kunsch (1989) and Liu and Singh (199) as an extension of the classical bootstrap by Efron (1979). It resamples blocks of data at a time, rather than single data points. The extension is needed in the case of dependent observations, where Efron's bootstrap fails. Conditional on observing a sequence Z 1 ; : : :; Z n, the moving blocks method generates pseudo sequences Z 1 ; : : :; Z n by sampling l blocks of size b and concatenating them. Here, we assume without loss of generality that n = lb. The observed sequence Z 1 ; : : :; Z n yields n? b + 1 such blocks. The 8

10 rst one is Z 1 ; : : :; Z b, the last one is Z n?b+1. The moving blocks distribution P n places mass 1=(n? b + 1) on each block, so the sampling of the blocks is done randomly with replacement. The point of this example is that, conditional on Z 1 ; : : :; Z n, a pseudo sequence Z 1 ; : : :; Z n clearly will be b-dependent. It is well known that to ensure asymptotic consistency of the moving blocks method, the block size b has to tend to innity with the sample size n. This is exactly the set-up of our theorem. Since the properties of the moving blocks bootstrap have already been analyzed elsewhere, we will not give any alternative proofs here. Here, we merely point out the structure of the moving blocks process fz i g, thereby validating the utility of our results. 9

11 A Proof of the theorem Proof of Theorem.1: In the proof we will need a result for bounding moments of m- dependent sequences. We will state it as a Corollary of the following Lemma, which implicitly is given in Chow and Teicher (1978) and deals with independent sequences. Lemma A.1 Let fy i g be an independent sequence of mean zero random variables. E jy i j q for some q 1 and all i. Then E n Y i q Cqn q q ; where C q is a positive constant depending only upon q. Assume Proof: See Theorem and Corollary in Section 10.3 of Chow and Teicher (1978). Corollary A.1 Let f i g be an m-dependent sequence of mean zero random variables. Assume E j i j q for some q 1 and all i. Then, for all n m, E n i q Cq q (16mn) q ; where C q is a positive constant depending only upon q. Proof: Dene t to be the integer part of n m, i.e., t = b n m c: Now split n into t blocks of size m and a remainder block: n A A t + A t+1. Due to m-dependency the odd-numbered blocks are independent of each other, as are the even-numbered blocks. This allows us to apply Lemma A.1: n i q A i + i odd A i i q even q C q (m) q 1 t (by Minkowski) (by Lemma A.1 and Minkowski) But, this is equivalent to n E i q Cq q q m(t= + 1) q C q q4 q m(t=) q C q q (16mn) q : 10

12 We are now able to prove the theorem. The main idea of the proof follows Berk (1973), but we need some modications, since our theorem is more general. For each n we choose an integer p = p n > m so that lim n!1 m=p = 0; lim n!1 p 1+(1?)(1+=) =d = 0: (17) This can be done, for example, by remembering assumption (6) and choosing p to be the smallest integer greater than m and greater than m 1 d 1, where is equal to 1 + (1? )(1 + =). Next dene integers t = t n and q = q n by d = pt + q; 0 q < p: The main idea of the proof is to split the sum n;1 + : : :+ n;d into alternate blocks of length p? m (the big blocks) and m (the little blocks). This is a common approach to proving central limit theorems for dependent random variables, and is attributed to Markov and Bernstein (197). Let U n;i = n;(i?1)p n;ip?m ; 1 i t; V n;i = n;ip?m n;ip ; 1 i t; U n;t+1 = n;tp+1 + : : : + n;d : By denition, n;1 + : : : + n;d = P t+1 U n;i + P t V n;i. Since the n;i are m-dependent and p > m, fu n;i g and fv n;i g are each independent sequences. It is easily seen that the dierence between B n?1( n;1 + + n;d P ) and B n?1 t+1 U n;i has variance approaching zero. Indeed, V ar(b?1 n t V n;i ) = B? n B? n t V ar(v n;i ) t[sup i V ar(v n;i)] B? n tk nm 1+ (by assumption ()) B n? (d=p)k n m 1+ K n m! 0 (by assumptions (3) and (4)): L n p P d n;i Hence, provided they exist, the asymptotic distributions of the two quantities B n P?1 and B n?1 t+1 P U n;i are the same, and the goal now is to show that B n?1 t+1 U n;i =) N(0; 1): In order to apply assumption (3) again, Cov(P we will rst P ar(p establish that B n?v t+1 U n;i) tends to one, or, equivalently, that B n? t+1 U t n;i; V n;i ) tends to zero. Note rst that Cov(U n;i ; V n;j ) = 0 unless j = i or j = i? 1. Furthermore, jcov(u n;i ; V n;j )j = je(u n;i V n;j )j [V ar(u n;i )V ar(v n;i )] 1 (by Cauchy-Schwarz) K n (mp) 1+ (by assumption ()): 11

13 Combining these two facts, we obtain and nally, B? n t+1 Cov( U n;i ; t t+1 Cov( U n;i ; V n;i ) K n L n t K n 1 L n pm = K n L n m p V n;i ) tk n(mp) 1+ ; t 1+ (mp) dm 1? 1+ (mp) (by assumption (3))! 0 (by assumption (4) and since < 1): P t+1 By Lyapounov's theorem it will now suce to verify that E ju n;ij + =Bn + tends to zero. By Corollary A.1 we have and therefore t+1 By assumption (3) nally, E ju n;i j + C + + n(16pm) + ; 1 i t + 1; E ju n;i j + =B + n Const. n (d=p + 1)(pm) + =B + n : n (d=p)(pm) + =B + n n L? + n n L? + n d p p pm + dm m (1?) + d = n L? + n p + +(1?) d? m p = O(1) A B (by assumption (5)); (1?) + where A p + +(1?) d? + (1?) and B (m=p). The second condition on p in (17) implies that A tends to zero. The rst condition on p in (17) together with the fact that < 1 imply that B tends to zero as well. 1

14 References Bergstrom, H. (1970). A comparison method for distribution functions of sums of independent and dependent random variables. Theor. Probability Appl {457. Berk, K. N. (1973). A central limit theorem for m-dependent random variables with unbounded m. Ann. Probability 1, 35{354. Bernstein, S. (197). Sur l'extension du theoreme du calcul des probabilites aux sommes de quantites dependantes. Mathematische Annalen 97, 1{59. Chow, Y. S. and Teicher, H. (1978). Probability Theory, Springer, New York. Diananda, P. H. (1955). The central limit theorem for m-dependent variables. Proc. Comb. Phil. Soc. 51, 9{95. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. Annals of Statistics 7, 1{6. Kunsch, H.R. (1989). The jackknife and the bootstrap for general stationary observations. Annals of Statistics 17, 117{141. Liu, R.Y. and Singh, K. (199). Moving blocks jackknife and bootstrap capture weak dependence. In Exploring the limits of bootstrap, ed. by LePage and Billard, John Wiley, New York. Orey, S. A. (1958). Central limit theorems for m-dependent random variables. Duke Math. J {546. Rosenblatt, M. (1985). Stationary sequences and random elds, Birkhauser, Boston. 13

Weak convergence of dependent empirical measures with application to subsampling in function spaces

Journal of Statistical Planning and Inference 79 (1999) 179 190 www.elsevier.com/locate/jspi Weak convergence of dependent empirical measures with application to subsampling in function spaces Dimitris