THE STRONG LAW OF LARGE NUMBERS FOR STATIONARY SEQUENCES Debdeep Pati Idia Statistical Istitute, Kolkata Jue 26, 2006 Abstract The traditioal proof of the strog law of large umbers usig idepedet ad idetically distributed radom variables 3, was developed by Kolmogorov i the 1930 s, ad explais well what happes ad much more, at this level of geerality. I this short article, I have tried to give a simple proof of the strog law of large umbers with the assumptio of oly statioarity of the give sequece of radom variables. The proof is essetially alog the lies of the proof of the ergodic theorem i 1. 1 Itroductio The law of large umbers, ot really a law but a mathematical theorem, is at the same time a justificatio for applicatio of statistics ad a essetial tool for the mathematical theory of probability. Jakob Beroulli, i the cocludig chapter of his Ars cojectadi 2 proposed ad proved the first versio of the law of large umbers ad qualified the problem he was facig as oe of exceedig difficulty ad usefuless whose solutio would icrease importace ad meaig of all of the theory. First, I would try to motivate the essece of the basic law of large umbers by cosiderig, very simply, a ifiite sequece x 0, x 1, x 2,... each of whose elemets is either 0 or 1. Perhaps it will help to thik of x as the result of the th trial of a ucertai experimet with x = 1 with some 1
probability p desigig success ad x idicatig failure. Let = 0 with the remaiig probability x = x 0 + x 1 + x 2 + + x 1 deote the average umber of success upto time. mathematically that for some sequece x, lim x ( 1) It is very easy to see exists, while for other sequeces x this is ot the case. But othig impedes the averages x from oscillatig more ad more slowly as grows. I our day to day life the society makes seemigly uderstadable statemets about the percetage of dyig smokers, the probability of rai tomorrow or a idustrial average yield. I the ext sectio, we formally state ad prove the strog law of large umbers for a statioary sequece. 2 The Strog Law of large umbers I this sectio, I will prove three theorems, the first of which shows that the lim x exists assumig oly statioarity of the give sequece ad the ext theorem helps us to idetify the limit oly i the case of i.i.d sequeces havig fiite secod momet. The limit ca be idetifed i the most geeral case ad ca be foud i advaced probability texts. The proof essetially follows the same lies as that of Kolmogorov s classical strog law of large umbers ad hece ot discussed here. The last theorem proves the L 1 covergece of x. Theorem 1 {X i } i 1 be a statioary sequece of radom variables such that E X 1 <. Let The lim X exists, i.e., X = X 1 + X 2 + X 3 + + X ( 1) lim sup X = lim if X Proof: Fix α > 0, β > 0. Let X = lim sup X ad X = lim if X. We will show that X = X (2.1) I order to prove 2.1, we will first show E X 1 E X (2.2) 2
Let Xα = mi{x, α} ad X β k = max{x k, β} k 1. We will later prove that 2.2 is true if we ca show for ay α > 0, β > 0 E X β 1 E Xα (2.3) We have Xα X ad X β k X k k 1. For ay give ɛ > 0, defie for each k 1, { } N β k = mi r 1 1 r X β k+i 1 r X α ɛ k 1 Next cosider the followig two lemmas. { } Lemma 1 N β k as defied above are idepedet ad idetically distributed fiite radom variables. Proof of the Lemma: For each k 1, ɛ > 0, sice Xα X ad X β k X k k 1, the set { } r 1 1 r X β k+i 1 r X α ɛ is o-empty by the defiitio of lim sup. Hece { } N β k k 1 are fiite radom variables. Now sice X β k = max{x k, β} k 1, {X β k } k 1 is also a statioary sequece sice a cotiuous fuctio of a statioary sequece is also statioary. The i.i.d property follows simply from the statioarity of the sequece {X β k } k 1. Lemma 2 Suppose for each ɛ > 0, there exists M 1 such that k 1, N β k M a.s., the we have E X β 1 E Xα Proof of the Lemma: Obtai a partitio of {1, 2, 3,..., } as: A 1 = {1, 2, 3,..., N β 1 }, A 2 = {N β 1 +1, N β 1 +2,..., N β 2 },..., A l = {N β l 1 +1, N β 1 + 2,..., N β l }, A l+1 = {N β l + 1,..., } such that N β l+1 >. Clearly A l+1 < M so that A 1 A 2 A l > M. Hece we have X β k = X β k + X β k ( M)(Xα ɛ) Mβ (2.4) k=1 k A 1 A l k A l+1 Takig expectatio of 2.4 ad dividig by we get ( ) M E X β 1 (E X α ɛ) Mβ 3
Now lettig ad ɛ 0, we have E { } Sice N β k X β 1 E Xα as defied above are i.i.d radom variables ad fiite w.p 1, we k 1 { } are certaily ot goig to get such bouds for N β k. Still fiiteess of the k 1 aforesaid i.i.d radom variables esures) that give ɛ > 0, we are able to get a M large eough such that P (N βk > M < ɛ k 1. Defie k 1, X β k = Xβ k 1 N β k M + max{α, Xβ k }1 N β k >M (2.5) Therefore, X β k Xβ k β k 1. For ay give ɛ > 0, defie for each k 1, { } N β k = mi r 1 1 r X β k+i 1 r X α ɛ Sice, X β k Xβ k, Ñ β k N β k, k 1. Note that if N β k > M, Ñ β k = 1 sice i that case X β k α X α. Therefore Ñ β k M k 1. Proceedig as i lemma 2 we get E But from 2.5 we get, X β 1 = X β 1 + Xβ 1 ( M ( max{α, X β 1 } X β 1 ) (E X α ɛ) Mβ (2.6) ) 1 N β 1 >M Xβ 1 + (α + β) 1 N β 1 >M (2.7) Takig expectatio of 2.7 ad by our choice of M, we get from 2.6, ( ) M E X β 1 E Xβ 1 (α + β) ɛ (E X α ɛ) Mβ (α + β) ɛ Now lettig ad ɛ 0, we have E X β 1 E Xα (2.8) Also we have X β 1 X 1 as β ad X β 1 X 1 ad E X 1 <. Hece by DCT, E X β 1 E X 1 as β. Agai we have, X α = X α1 X α <0 + X α1 X α 0 = X 1 X <0 + X α1 X 0 Now X α1 X 0 X 1 X 0 as α. So by MCT, E X α1 X 0 E X 1 X 0 Now X k X k 4
X 1 k=1 X k lim if X lim if 1 k=1 X k E X lim if E 1 k=1 X k E X E X 1 < E X > E X 1 X <0 >. Therefore we have Hece as α E X α = E X 1 X <0 + E X α1 X 0 So as α ad β we fially have E X α E X (2.9) E X 1 E X (2.10) Now replacig X k by X k i the above(ote that all the assumptios of statioarity ad fiiteess of expectatio of the modulus of the radom variables remai valid) we have E X 1 E X (2.11) Combiig 2.10 ad 2.11 we have E X = E X = E X 1 (2.12) Agai sice X X 0 ad E X X = 0, we observe that X = X = S(say) which implies that X a.e S thus provig the theorem. Theorem 2 {X i } i 1 be a i.i.d sequece of radom variables such that E X 2 1 <. The lim sup X = lim if X = E X 1 Proof: The first equality i.e., lim sup X = lim if X ca be proved exactly i the same maer as i the case of statioary sequeces. Let lim sup X = lim if X = lim X = S. We will show that S = E X 1 = µ(say) (2.13) Let V ar X 1 = σ 2 < (by the assumptio). Observe that E X µ 2 = σ 2 5
Therefore, lim E X µ 2 = 0 (2.14) We have already proved i Theorem 1 that E S = E X 1 < (2.15) Now E S 2 = E 2 lim X = E lim 2 2 X = E lim if X lim if E 2 X = lim if 1 2 E X2 i + i j E X ix j = lim if 1 2 ((µ 2 + σ 2 ) + ( 1)µ 2 ) = µ 2 <. Therefore, E S 2 < (2.16) Sice X a.e. S, give ɛ > 0, we ca fid a 0 such that 0, we have X (S ɛ, S + ɛ). Therefore ( X µ ) 2 max { (S ɛ µ) 2, (S + ɛ µ) 2} 0. From 2.15 ad 2.16 we get, E max { (S ɛ µ) 2, (S + ɛ µ) 2} <. Also we have ( X µ ) 2 a.e. (S µ) 2 (2.17) Thus from 2.14 ad 2.17 ad applyig DCT, we get lim E X µ 2 = E S µ 2 = 0 S = µ a.s. Theorem 3 {X i } i 1 be a statioary sequece of radom variables such that E X 1 <. Let X = X 1 + X 2 + X 3 + + X ad lim sup X = lim if X = S(say).The ( 1) L X 1 S Proof: The proof ecessarily rests upo the Egorov s theorem, which we are goig to state as a lemma the proof of which ca be foud i stadard probability texts. a.s Lemma 3 {X } 1 ad X are fiite valued radom variables ad X X, the for every ɛ > 0 a set E with P ( E ) < ɛ ad X X uiformly o E i.e., sup X (ω) X (ω) 0 as ω E 6
We cotiue with the proof of the theorem. Let ɛ > 0 be give. Sice sup ω E X (ω) X (ω) 0 as, 0 such that 0, Note that 0, sup X (ω) X (ω) < ɛ ω E E X S = E X S 1 E + E X S 1 E ɛp (E) + E X S 1 E Now, E X S 1 E E 1 X i S 1 E 1 Hece from 2.18 ad 2.19 we get, E X S ɛp (E) + E X 1 S 1 E X1 S λ + E (2.18) E X i S 1 E = E X 1 S 1 E (2.19) X 1 S 1 E X1 S >λ Therefore, E X S ɛp (E) + λp ( E ) + E X 1 S 1 X1 S >λ (2.20) Now ote that E X 1 S E X 1 + E S ad E S = E lim X = E lim X = E lim if X lim if E X lim if 1 E X i = lim if E X 1 = E X 1 <. Therefore we have E X 1 S <. Also ad Hece by DCT, we have X 1 S 1 X1 S >λ 0 as λ X 1 S 1 X1 S >λ X 1 S with E X 1 S < E X 1 S 1 X1 S >λ 0 as λ From 2.20, we get E X S ɛ (P (E) + λ) + E X 1 S 1 X1 S >λ Now let ɛ 0 ad λ to get 0, E X S 0 which shows L X 1 S cocludig the proof. 7
Refereces 1 Y.Katzelso, B.Weiss: A simple proof of some ergodic theorems. Isr.J.Math. 42(1982), 291-296. 2 J.Beroulli: Ars cojectadi. Basel, 1713 (published posthumously). 3 A.N.Kolmogorov: Grudbegriffe der Wahrscheilichkeitsrechug. Spriger Verlag, Berli 1933. 8