A Central Limit Theorem for the Sum of Generalized Linear and Quadratic Forms by R. L. Eubank and Suojin Wang Texas A&M University ABSTRACT. A central limit theorem is established for the sum of stochastically dependent generalized linear and quadratic forms which is often used in statistical inference. It is shown that a mild correlation type condition is sufficient to ensure that Liapounoff type conditions on the linear and quadratic forms individually will imply asymptotic normality of their sum. Key words: central limit theorem, dependent sum, generalized quadratic forms, linear forms Running headline: A Central Limit Theorem 1. Introduction and main results Statistical testing and inference problems occasionally lead to the study of random variables that are the sum of generalized linear and quadratic forms. While central limit theorems are available for analyzing either a quadratic or linear form separately, such results are not sufficient to imply asymptotic normality for their sum unless the two forms are independent. In this paper we show that, with the addition of a mild correlation condition, standard Liapounoff type conditions on the generalized linear and quadratic forms separately will ensure asymptotic normality of their sum. 1
The basic problem to be considered extends that in de Jong (1987). While de Jong s asymptotic normality results on generalized quadratic forms are used extensively, they don t apply to the sum of stochastically dependent generalized linear and quadratic forms which is often used in statistical inference. It is precisely the purpose of this note to provide a generalized asymptotic result that covers even wider applicability. Assume that we have independent random variables X 1,..., X n, and associated transformed variables W kj = W kjn (X k, X j ) and L k = L kn (X k ) for function W kjn (, ) and L kn ( ) satisfying EL k = 0 and E(W kj X k ) = 0 a.s. for k, j = 1,..., n. (1.1) Assume, without loss of generality, that W kj = W jk and define W(n) = 1 j<k n W kj and L(n) = L k. Note that the second part of condition (1.1) implies that W kk = 0 a.s. With this condition, W(n) is said to be clean as defined in de Jong (1987). Then, our goal is to give conditions under which T(n) = W(n) + L(n) is asymptotically normal. We are interested primarily in the case where both W(n) and L(n) are of the same stochastic order since T(n) can be treated using existing techniques otherwise. For example, if W(n) is of smaller stochastic order than L(n), then the standard CLT applies 2
to L(n) and thus to T(n). Likewise, if W(n) is the only dominating term then de Jong s results may be applied directly to T(n). Thus, writing σ 2 W(n) = Var W(n), σ 2 L(n) = Var L(n) and σ 2 (n) = Var T(n), we assume that 0 < lim σ2 W (n) σ 2 (n), lim σ2 L (n) σ 2 (n) < 1. (1.2) By (1.1), σ 2 (n) = σ 2 W (n) + σ2 L (n). We can now state our main results as follows: Theorem. Assume that (1.1) (1.2) hold and that as n, (a) max 1 k n Var W kj /σ 2 (n) 0, (1.3) (b) EW(n) 4 /σ 4 W(n) 3, (1.4) (c) and (d) EL 4 k/σ 4 (n) 0, (1.5) k 1 E E (W kj L k X 1,..., X k 1 ) 2 /σ 4 (n) 0. (1.6) Then T(n)/σ(n) d N(0,1). Note that if we set C j = k=j+1 E(W kjl k X j ), then C i and C j (i j) are uncorrelated. Thus, condition (d) above is easily seen to be equivalent to (d ) var (C j )/σ 4 (n). 3
Condition (d ) may often be easier to check in practice than condition (d). Due to (1.2), condition (1.5) is seen to be a standard Liapounoff condition for asymptotic normality of the linear form L(n). Conditions (1.3) (1.4) are sufficient to imply that W(n)/σ W (n) d N(0,1) as shown in de Jong (1987). Thus the implication of the Theorem is that the correlation type condition (1.6) is sufficient to allow us to combine individual asymptotic normality of W(n) and L(n) to conclude asymptotic normality of their sum. An important special case occurs when W kj = e k e j w kjn and L k = e k l kn for e 1,..., e n being independent and identically distributed (iid) random variables and nonstochastic sequences {w kjn } and {l kn } with w kjn = w jkn. Note that in this special case X k may be seen as e k.the conditions of the Theorem simplify substantially in this case and we therefore state it explicitly as a corollary. Condition (1.9) of the Corollary is obtained from condition (d) of the Theorem using summation by parts and is a reduced form of condition (d ). Corollary. Assume that T(n) = W(n) + L(n) for W(n) = k 1 e ke j w kjn and L(n) = e kl kn with iid random variables e 1,..., e n having Ee 1 = 0, Ee 2 1 = σ 2 and Ee 4 1 <. Then, σw 2 (n) = σ4 n k 1 w2 kjn, σ2 L (n) = σ2 n l2 kn and T(n)/σ(n) d N(0,1) for σ 2 (n) = σw 2 (n) + σ2 L (n) if in addition to (1.2) and (1.4) we have and max 1 k n wkjn/σ 2 2 (n) 0, (1.7) l 4 kn/σ 4 (n) 0, (1.8) 4
k=j+1 2 w kjn l kn /σ 4 (n) 0. (1.9) 2. Applications As an illustrative example of the Corollary, let w kjn = ( k j + 1) 1 and l kn = 1 for 0 < j, k n to obtain W(n) = k 1 ( k j + 1) 1 e k e j and L(n) = e k. Thus, σl 2 (n) = nσ2 and σw 2 (n) = σ4 n k 1 ( k j + 1) 2 n, i.e., σw 2 (n) is exactly of order n. Therefore, conditions (1.2) and (1.8) are clearly satisfied. Moreover, max 1 k n w2 kjn is bounded so that condition (1.7) is satisfied. Thus, it remains to check conditions (1.4) and (1.9). It is readily obtained that ( n k=j+1 w kjnl kn ) 2 = ( n k=j+1 ( k j + 1) 1 ) 2 (log j)2 = O(n(log n) 2 ), leading to (1.9). Finally, to check (1.4) which is necessary to ensure the asymptotic normality of the quadratic term, one may use de Jong s (1987) partition of EW(n) 4. With the notation in his Table 1 (p. 266), it is readily seen that G I, G II, G III are of order n and G IV = O(n(log n) 2 ), all of which are of lower order than G V which is asymptotic to σw 4 (n)/2. Therefore, EW(n) 4 lim n σw 4 (n) = lim n 6G V = 3, (n) σ 4 W and by the Corollary, (W(n) + L(n))/σ(n) d N(0,1). Other examples of applications where the Corollary can be useful include certain inference problems in nonparametric smoothing for which a basic pivotal quantity of interest is asymptotically distributed as T(n)/σ(n) for T(n) as in the Corollary. Condition (1.9) is often quite easy to check in such cases. For example, one may readily show that 5
(1.9) is satisfied for the nonparametric regression confidence region problem studied in Eubank and Wang (1994). This allows us to conclude that their major result, Theorem 2, remains valid if their assumption that e 1 has moments of all orders is replaced by Ee 4 1 <. Note also that the iid assumption for e i s in the Corollary above is for simplicity and is not necessary. It is possible to generalize the result to non-iid cases under certain fourth moment restrictions. 3. Proof of the Theorem To prove the Theorem we will apply results from Heyde and Brown (1970). Define U k = U 1k + U 2k for U 1k = k 1 W kj/σ(n) and U 2k = L k /σ(n), k = 1,..., n. Then, the Theorem follows from the Heyde/Brown work once we show that and EUk 4 = o(1) (2.1) { E E ( } 2 Uk 2 ) X 1,..., X k 1 1 = o(1). (2.2) For condition (2.1) we have that EU 4 k 24 [ ( EU 4 1k + EU 4 2k)]. Condition (1.5) ensures that EU 4 2k = o(1). On the other hand, by partitioning EU 4 1k into five different sums one can show that ( k 1 W kj) 4 /σ 4 W (n) = o(1), as in Propositions 3.2 3.3 of de Jong (1987). Then using (1.2) we obtain that EU 4 1k = o(1). To verify (2.2) we begin by using (1.1) along with Lemma 2.1 of de Jong (1987) to rewrite the condition as E E ( Uk 2 ) X 1,..., X k 1 6 2 = 1 + o(1).
Now E ( U 2 k X 1,..., X k 1 ) = a11k + 2a 12k + a 22k with a ijk = E (U ik U jk X 1,..., X k 1 ) (In particular, a 22k = EL 2 k /σ2 (n)) and, hence, { n { n } 2 E E ( U 2 k X 1,..., X k 1 ) } 2 = E (a 11k + a 22k ) { + 4E (a 11k + a 22k ) } ( n ) 2 a 12k + 4E a 12k. The last term on the right hand side of this expression is o(1) as a result of (1.6). This also entails that the second term is o(1), provided that E { (a 11k + a 22k )} 2 = O(1) which is seen below. By showing that under (1.3) (1.4) Var( a 11k) 0 as n and E ( a 11k) = k 1 VarW kj/σ 2 (n) = σw 2 (n)/σ2 (n) as in Propositions 3.1 3.2 of de Jong (1987) we conclude that ( σ 4 n ) 2 (n) σw 4 (n)e a 11k 1. (2.3) Thus, using a 22k = EL2 k /σ2 (n) = σ 2 L (n)/σ2 (n) and E ( a 11k) = σ 2 W (n)/ σ 2 (n) along with (2.3) gives { n 2 ( E (a 11k + a 22k )} = E a 11k ) 2 + 2 σ2 W (n)σ2 L (n) σ 4 (n) + σ4 L (n) σ 4 (n) and the proof is complete. = [{ σ 4 W(n) + 2σ 2 W(n)σ 2 L(n) + σ 4 L(n) } /σ 4 (n) ] + o(1) = 1 + o(1), 7
Acknowledgments The research of R. L. Eubank was partially supported by the National Science Foundation. The research of S. Wang was partially supported by the National Science Foundation, the National Security Agency, and the National Cancer Institute (CA-57030). References Eubank, R. L. and Wang, S. (1994). Confidence regions in nonparametric regression. Scand. J. Statist. 21, 147 158. Heyde, C. C. and Brown, B. M. (1970). On the departure from normality of a certain class of martingales. Ann. Math. Statist. 41, 2161 2165. de Jong, P. (1987). A central limit theorem for generalized quadratic forms. Probab. Th. Rel. Fields 75, 261 277. R. L. Eubank and Suojin Wang, Department of Statistics, Texas A&M University, College Station, TX 77843, U.S.A. 8