STRONG CONSISTENCY OF LEAST SQUARES ESTIMATE IN MULTIPLE REGRESSION WHEN THE ERROR VARIANCE IS INFINITE

Transcription

1 Statstca Sca 9(1999), STRONG CONSISTENCY OF LEAST SQUARES ESTIMATE IN MULTIPLE REGRESSION WHEN THE ERROR VARIANCE IS INFINITE J Mgzhog ad Che Xru GuZhou Natoal College ad Graduate School, Chese Academy Abstract: Let Y = x β + e,1, S = xx. Suppose that the radom errors e 1,e 2,... are..d., wth a commo dstrbuto F belogg to the class F r = {F : xdf =0, 0 < x r df < } for some r [1, 2). I ths paper we obta a suffcet codto for the strog cosstecy of the Least Sequares Estmate (LSE) ˆβ of β. The codto s ecessary the followg sese: If the codto s ot satsfed, the for some F F r, ˆβ rals to coverge a.s. to β. Key words ad phrases: Least squares estmate, lear models, strog cosstecy. 1. Itroducto ad Ma Results Cosder the lear model Y = x β + e, 1, 1. (1.1) I ths paper we assume that x 1,x 2,... are kow o-radom p-vectors, ad e s the radom error the th observato, =1, 2,... The LSE of β wll be deoted by ˆβ. May statstcas have cosdered the problem of strog cosstecy of ˆβ. I earler days the problem was studed uder the assumpto that the e possess a fte varace. Ths case was fally solved a mportat paper of La, Robbs ad We (1979) whch they showed that f e 1,e 2,... are..d., Ee 1 =0ad 0 <Ee 2 1 <, a suffcet codto for the strog cosstecy of ˆβ s that ( = 1 x x ) 0, as. (1.2) Sce the ecessty of (1.2) was earler proved by Drygas (1976), (1.2) s both ecessary ad suffcet. Later, a umber of authors cosdered the case where e does ot have a fte varace. A stadard formulato s as follows: e 1,e 2,... are..d. wth a commo dstrbuto F belogg to the famly F r = { F : xdf =0, 0 < } x r df <, 1 r<2. (1.3)

2 290 JIN MINGZHONG AND CHEN XIRU Uder ths codto Che (1981) showed that for geeral p, fs 1 = O( (2 r)/r (log ) a )forsomea>1 ad some addtoal codtos are met, the ˆβ β a.s.. Almost at the same tme Che, La ad We (1981) showed that strog cosstecy holds uder the codto S 1 = O( (2 r)/r (log ) 2/r ε) for some ε>0. I hs upublshed doctoral dssertato, Zhu (1989) made a essetal mprovemet: for geeral p the codto S 1 = O( (2 r)/r )salreadysuffcet for ˆβ β a.s.. Ths codto, though far weaker the earler-metoed codtos, s stll ot the best (see Remark 2 below). The best codto for the case of p = 1 was obtaed by Che, Zhu ad Fag (1996) as a by-product of a more geeral result. The purpose of ths paper s to gve a soluto for geeral p. Assume that S 1 exsts for large. Wrtea = S 1 x. For small such that does ot exst, a ca be defed arbtrarly. Defe N(K) =#{ : 1, a K 1 }. Let {(, 1),...,(, )} be a permutato of {1, 2,...,} satsfyg a (,1) a (,).Put V (, j)= V ()= max 1 j x I( a a (,j) ), 1 j, V (, j). Now we ca formulate the ma result of ths paper. Theorem. Suppose model (1.1) that the e 1,e 2,... are..d., wth a commo dstrbuto F belogg to F r. The a suffcet codto for the strog cosstecy of ˆβ s: For 1 <r<2:(1, 2) holds ad N(K) =O(K r )ask. (1.4) For r = 1:(1.2) holds N(K) = O(K) adv () = O(1). (1.5) The codto s also ecessary the followg sese: f (1.3) ad (1.4) are ot satsfed, the for some F F r, ˆβ fals to coverge to β almost surely. Remark 1. Cosder model (1.1). Assume that the radom errors e 1,e 2,... are..d., ad F s ther commo dstrbuto. As metoed earler, for F F 2 the ecessary ad suffcet codto for the strog cosstecy of ˆβ s that 0. Ths s true for every F F 2, ad we may smply say that the codto 0 s a ecessary ad suffcet codto for the class F 2. For 1 r<2 the stuato s more complcated. The above theorem does ot mply that the codto C r (1.3) or (1.4) s a ecessary ad suffcet codto for the class F r. For accordg to the above theorem, whe codto

3 LEAST SQUARES ESTIMATE IN MULTIPLE REGRESSION 291 C r s ot satsfed, we ca oly assert that there exsts at least oe F F r such that f F s the commo dstrbuto of the e s,the ˆβ β a.s. does ot hold. Ca we fd a codto D r (volvg oly {x }) whch s a ecessary ad suffcet codto for the strog cosstecy of ˆβ for the whole class F r?evdetly ths s mpossble. For sce F 2 F r for ay r [1, 2), f such a codto D r exsts, t must be (1.2). But accordg to our theorem, (1.2) aloe s evdetly ot suffcet: A smple example shows that the codto N(K) =O(K r )sot a cosequece of (1.2). The meag of our theorem ca also be uderstood the followg way: f oe wats a codto depedg solely upo {x } whch s suffcet for the strog cosstecy of ˆβ for the whole class F r, the the codto stated the theorem s the best possble. Remark 2. A smple example shows that Zhu s (1989) codto = O( (2 r)/r ) s stroger tha ours. Cosder model (1.1) whch β s oedmesoal ad { 1, whe =2 x = 1, 2 2, 2 3,... (1.6) 0, otherwse. The S = O(log ) ad Zhu s codto fals. We caot assert from Zhu s theorem that ˆβ β, a.s., but ˆβ β a.s. accordg to Kolmogorov s strog law of large umbers. O the other had t s easy to verfy that sequece (1.5) satsfes (1.2)-(1.4). So the strog cosstecy of ˆβ follows from our theorem. 2. The Necessty of (1.2) Ths follows drectly from the followg lemma of Che (1981): f {e 1,e 2,...} s a sequece of depedet radom varables cotag o asymptotcally degeerate subsequece (.e. a subsequece {e } such that e c 0pr. for some costat sequece {c }), ad {c, 1, 1} s a array of costats, the c e 0 pr. etals c The Necessty of N(K) =O(K r ) For a matrx A =(a j ), defe the matrx orm A =max,j a j. The t ca easly be show that S 1S 1 = O(1). If S 1 0, the lm a =0, max 1 S 1 x 0. Remember a = x. Now suppose N(K) soto(k r ). We shall fd a sequece {e 1,e 2,...} of..d. r.v.s wth commo dstrbuto belogg to the

4 292 JIN MINGZHONG AND CHEN XIRU famly F r such that x e 0, a.s. (3.1) We ca fd a sequece of postve tegers 1 < 2 < such that N( k )/ r k as k. Therefore there exsts {p k,k 1}, such that p k > 0, p k =1, r k p k <, ad k=1 k=1 p k N( k )=. Let {e 1,e 2,...} be a..d. sequece wth a commo dstrbuto F : k=1 P (e 1 = k )=p(e 1 = k )=p k /2, k 1. The F belogs to F r. Sce a 0, we ca rearrage { a, 1} a decreasg order: a (1) a (2).Notethat{(1), (2),...} s a permutato of {1, 2,...} ad, by defto of N(K), t follows that { e 1 a () 1 } {N( e 1 ) ()}. Therefore = p k N( k )=E(N( e 1 )) = P (N( e 1 ) ) = P (N( e 1 ) ()) k=1 P ( e 1 a () 1 )= P ( e 1 a 1 )= P ( a e 1), (3.2) whch etals P ( a e 1,.o.) = 1. The (3.1) s proved ad hece the ecessty of N(K) =O(K r ) follows. 4. Suffcecy: 1 <r<2 Lemma 1. Let a be defed as earler. The covergece of a e etals x e 0. Proof. Wrte T o =0,T j = j a e,adt = a e.wehave S x e = 1 x j x j(t T j 1 ) = S 1 x j x j(t T (T j 1 T )) k T T + δ j T j 1 T + δ j T j 1 T, (4.1) j=k+1 where δ j = x j S 1 x j ad k s a large teger for whch T T ad T l T are small for l>k. Sce δ j 0forfxedj (whch follows from S 1 0), ad δ j = p, each term the rght had sde of (4.1) ca be made arbtrarly small, ad the lemma follows.

5 LEAST SQUARES ESTIMATE IN MULTIPLE REGRESSION 293 Now fx j {1,...,p}. Deote by d the jth compoet of a. Defe e = e I( e < d 1 ). To prove the strog cosstecy of ˆβ,byLemma1we eed oly show d e coverges a.s.. (4.2) The, applyg Kolmogorov s three seres theorem, we eed to verfy P ( e d 1) <, (4.3) Sce Ed e I [ e d <1] coverges, (4.4) Ed 2 e2 I [ e d <1] <. (4.5) P ( e d 1) = P ( e d 1 )=P( e I( e d 1 ) d 1 ) d E( e I( e d 1 )), the proof of (4.3) follows from the argumet below for (4.4). Also, the proof of (4.5) s smlar to (4.4). So we proceed wth (4.4). Let q = P ( 1 e 1 <), =1, 2,... Sce Ee =0,wehaveEe = E(e I( e d 1 )). If k 1 d 1 <k,wehave E d e d E e I( e d 1 ) (k 1) 1 j=k 1 jq j, k 2. Further, otcg that #{ : 1,k 1 < d 1 k} = Ñ(k) Ñ(k 1), where Ñ(k) =#{ : 1, d 1 k}, wehave Ed e Ñ(1) sup d E e 1 + (Ñ(k) Ñ(k 1))(k 1) 1 jq j 1 k=2 j=k 1 J 1 + J 2. (4.6) Sce a 0, J 1 remas bouded as. O the other had, we have ( J 2 = j 1 j ) Ñ(j +1)jq j Ñ(1)jq j + Ñ(k)((k 1) 1 k 1 ) jq j H 1 H 2 + H 3. From Ñ(j +1) c(j +1) r (for some c) ade e 1 r <, t follows that H 1 <. Lkewse for H 2.AsforH 3, Ñ(k) =O(k r ), (k 1) 1 k 1 = O(k 2 ), ad r>1, k=2

6 294 JIN MINGZHONG AND CHEN XIRU so j k=2 Ñ(k)((k 1) 1 k 1 )=O(j r 1 )adh 3 < follows from the fact that E e 1 r <. Summg up, ad otcg (4.6), we have (4.4), cocludg ths part of the proof. 5. Suffcecy: r =1 The above argumet breaks dow for the case r = 1, sce ths case we ca oly get H 3 = O(log ) ad ot O(1). Ths s the reaso for mposg the addtoal codto V () = O(1). Uder the codto V () =O(1), apply the argumets Secto 4 to the sequece {e Ee I [ e < a 1 ]} to set S 1 x (e Ee ) 0, a.s. ( where,e = e I( e < a 1 )). (5.1) Therefore we eed oly show x Ee 0. (5.2) To ths ed, defe t = xdf, a(,) 1 x < a (,+1) 1 1, wth the coveto a (,+1) 1 =. Wehave E( x e )=S 1 = x(, ) xdf x a (,) 1 x (,) j= t j = j t j x (,). Two cases are possble: the frst case s a (,j) = a (,j+1). The t j = 0 ad t j S 1 j x (,) = t j V (, j). The secod case s a (,j) > a (,j+1). The we have S 1 j x (,) = V (, j) by defto. Hece we always have t j S 1 j x (,) = t j V (, j). It follows that h E(S 1 x e )= t j V (, j) = t j V (, j)+ t j V (, j) J 1 + J 2, j=h+1 (5.3) where h s fxed. Wthout loss of geeralty assume x 0forall 1, ad f S 1 does ot exst choose a 0. Thea 0forall 1. Sce lm a =0,

7 LEAST SQUARES ESTIMATE IN MULTIPLE REGRESSION 295 we have lm (, ) =(). Hece, cosderg lm sup V (, j) lm sup j 0, we have x () =0, (5.4) where j =max(l : a (l) = a (j) ). From (5.4) we get lm J 1 =0forfxed h. Further, by assumpto (1.4), V (, j) V () = O(1). Hece J 2 O(1) j=h+1 t j O(1) x df O(1) x df. x a (,h+1) 1 x a (h+1) 1 The last tegral ca be made arbtrarly small by cloosg h large eough. Summg up ad otcg (5.3), we obta (5.1). The result (5.2) ca be proved smlarly (ad wthout the assumpto V () = O(1)). 6. The Necessty of V () =O(1) for r =1 Suppose that ˆβ s strogly cosstet, so (3.1) holds. From Secto 2 ad Secto 3, we have S 1 0adN(K) =O(K). As poted out at the ed of Secto 5, these two facts etal (5.2). Therefore S 1 x (e Ee ) 0, a.s.. These two facts, together wth (3.1), etal S 1 x e 0, a.s.. Summg up, we get S 1 x Ee 0. (6.1) Therefore to prove the ecessty of the codto V () = O(1)wehavetoshow that, f V () s ot bouded, we ca costruct a sequece of..d. radom varables {e } wth Ee 1 = 0 such that (6.1) s ot true. Ths ca be doe as Secto 2 of Che (1995); the detals are omtted. Ackowledgemet The authors very much apprecate the work of a aoymous referee who suggested smplfed proofs for some pots the orgal text. Especally, he poted out that a addtoal codto the orgal verso of the paper was ot eeded, thus provdg a essetal mprovemet of the orgal result. The authors receved support from Guzhou provce ad the NNSF of Cha. Refereces Che, X. R. (1981). Aga o the cosstecy of least squares estmates lear models. Acta Math. Sca 24, ( Chese). Che, X. R. (1981). O a problem of weak cosstecy of lear estmates lear models. Ch. Aals Math. 2, ( Chese).

8 296 JIN MINGZHONG AND CHEN XIRU Che, X. R. (1995). Cosstecy of LS estmates of multple regresso uder a lower order momet codto. Scece Cha (Ser. A) 38, Che, X. R., Zhu, L. X. ad Fag, K. T. (1996). Almost sure covergece of weghted sums. Statst. Sca 6, Che, G. J., La, T. L. ad We, C. Z. (1981). Covergeces systems ad strog cosstecy of least squares estmates regresso models. J. Multvarate Aal. 11, Drygas, H. (1976). Weak ad strog cosstecy of the least squares estmates regresso models. Z. Wahrsch. verw. Gebete 34, La, T. L., Robbs, H. ad We, C. Z. (1979). Strog cosstecy of least squares estmates multple regresso II. J. Multvarate Aal. 9, Zhu, L. X. (1989). Doctoral dssertato. Publshed by the Isttute of Systems Scece. The Chese Academy ( Chese). Departmet of Mathematcs, Guzhou Natoal College, Huax, Guyag E-mal: wlcjw@xhfo.com Graduate School, Chese Academy, Bejg (Receved December 1995; accepted Aprl 1998)