Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series theorem We will give a secod proof of the SLLN. This proof is based o radom series ad gives more precise iformatio i some cases. EX 5. Note that ad + = m 0 diverges, m + = m + m 0 i the sese that the partial sums coverge. How about Z, mm + coverges, where Z {±} are IID uiform? Note that, with positive probability, you have log stretches of oes. O the other had, you have +s ad s roughly half of the time. To aswer the questio, we prove: THM 5. Three-series theorem Let X, X,... be idepedet. Let A > 0 ad Y = X { X A}. I order for X to coverge a.s., it is ecessary ad sufficiet that: P X > A < +, =
Lecture 5: More o the a.s. covergece of sums ad EY coverges, = VarY < +. 3 = EX 5.3 Cotiued Take A >. The X = Y = Z ad EY = 0 for all. Therefore it suffices to check 3. Note that VarX = EX = < +.. Sufficiecy Proof: We claim it suffices to prove that Y EY coverges. Ideed, suppose that this is the case, the by it follows that Y coverges. By ad BC, So we prove: X coverges. THM 5.4 Zero-mea series theorem Let X, X,... be idepedet with mea 0. If VarX < + the S = k X k coverges a.s. Proof: For M > 0, let w M sup S S m w.,m M By a Cauchy-type argumet it suffices to prove that, for all ε > 0, Pw > ε = P M {w M > ε} = lim M Pw M > ε = 0, by cotiuity. To argue about the sup we prove the followig:
Lecture 5: More o the a.s. covergece of sums 3 THM 5.5 Kolmogorov s maximal iequality Let X, X,... be idepedet with 0 mea ad fiite variace. The P max S k x VarS k x. Note that this is a stregtheig of Chebyshev s iequality for sums of idepedet RVs. Proof: The trick is to divide the evet of iterest ito whe it first occurs. Let A k = { S k x but S j < x for j < k}. The, usig idepedece of S k Ak ad S S k ad the fact that ES S k = 0, ES ES; A k = k= ESk + S ks S k + S S k ; A k k= ESk ; A k k= x PA k k= = x P max S k x. k Goig back to the previous proof, ote that Kolmogorov s maximal iequality implies P max S m S M > ε VarS N S M m M+ M m N ε VarX m ε, so takig a limit whe N + the M + ad usig cotiuity Fially, ote that P sup S m S M > ε m M Pw M > ε P 0, usig S S m S S M + S m S M. sup S m S M > ε m M as M +. 0,
Lecture 5: More o the a.s. covergece of sums 4.3 Necessity Proof: Suppose X coverges. The X 0 ad by BC, for all A > 0, holds ad Y coverges as well. We use symmetrizatio. If Z is a RV ad Z is a idepedet copy. The we let Z = Z Z. We will show below that the covergece of a zero-mea, uiformly bouded radom series implies the covergece of the secod momets. The Y coverges = VarY = VarY < + = Y EY coverges = EY coverges. The statemet above will follow from the followig maximal iequality: THM 5.6 Let X, X,... be idepedet with zero mea, fiite variace ad PX i c = for some c > 0. The P max S c + x k x k VarS. Proof: Let A be the evet above ad defie A k as before. O the oe had, otig that o A k we have S k c + x ad arguig as before, O the other had, ES ; A = k PA ES k + S S k ; A k c + x + EXk k= = PA c + x + ES. ES ; A = ES ES ; A c ES x PA c. Rearragig gives the result. Fially if we had VarY = +, the previous iequality used as before would give for all M > 0 a cotradictio. P sup S S M > ε k M =,
Lecture 5: More o the a.s. covergece of sums 5.4 Applicatios.4. A secod proof of the SLLN To see the coectio betwee the covergece of series ad the law of large umbers, we begi with the followig lemma whose proof is i the appedix. LEM 5.7 Kroecker s lemma If a + ad x /a coverges the a m= x m 0. THM 5.8 Strog law of large umbers Let X, X,... be pairwise idepedet IID with E X < +. Let S = k X k ad µ = EX. The S µ, a.s. Proof: We already proved that it suffices to show T µ where Y k = X k { Xk k} ad T = k Y k, that EY k µ, ad that + i= VarY i i E X < +. That implies, from Theorem 5.4 or from the Three-Series Theorem with A >, k Y k EY k k coverges a.s..4. Rates of covergece Uder a stroger assumptio, we have the followig: THM 5.9 Rate of covergece Let X, X,... be IID with EX = µ < + ad VarX = σ < +. The S = k X k satisfies S µ / log /+ε = / S log /+ε µ 0, a.s.
Lecture 5: More o the a.s. covergece of sums 6 Proof: Take µ = 0 w.l.o.g. Defie a = / log /+ε. The Recallig that Var X a = σ log +ε. xlog x α dx = α + C, log x α for α > we get that the previous series is fiite. Hece, from Theorem 5.4, X /a coverges ad a k X k goes to 0. Law of the iterated logarithm Let X be IID with EX = 0 ad VarX = σ < + ad let S = k X k. Usig radom series results, we proved that S log /+ε 0, a.s. O the other had, we will prove later that the distributio of / S coverges to a o-trivial limit. I fact: THM 5.0 Law of Iterated Logarithm LIL Let X be IID with EX = 0 ad VarX = σ < + ad let S = k X k. The lim sup S σ log log =, a.s. We prove the result i a special case: assume from ow o that X N0,. The more geeral case uses Browia motio ad is described i Dur0, Sectio 8.8. But first:. Remider: Normal distributio Recall: DEF 5. Desity A RV X has a probability desity fuctio PDF f X : R 0 + if B BR, PX B = f X ydy. B
Lecture 5: More o the a.s. covergece of sums 7 The, by Exercise.6.8, if E hx < + the EhX = hxf X xdx. The: DEF 5. Gaussia A RV X has a ormal distributio with mea µ ad variace σ deoted X Nµ, σ if it has desity f X x = exp x µ πσ σ. Note i particular that f X xdx = = π = π = π π =, π 0 0 0 πσ exp x µ + y µ σ exp x + ỹ d xdỹ exp r rdrdθ exp u du where we used polar coordiates o the third lie. Also, EX µ = = σ = 0, x µf X xdx z π exp z dz dxdy by symmetry ad, by liearity of expectatio ad usig polar coordiates agai, VarX = EX µ = σ = σ 0 r exp = σ u exp u du, = σ, 0 x + ỹ π exp x + ỹ r rdr d xdỹ
Lecture 5: More o the a.s. covergece of sums 8 by itegratio by parts. That is, EX = µ ad VarX = σ. Let X Nµ, σ ad Y Nµ, σ be idepedet ormal RVs. By our covolutio formula, the PDF of X + Y is f X+Y z = f X z yf Y ydy = = = exp z y µ πσ σ σ + y µ σ πσ + σ exp z µ µ σ + σ, dy that is, X + Y Nµ + µ, σ + σ. See Dur0, Example..4 for the computatios.. Proof of LIL i Gaussia case We start with two lemmas of idepedet iterest. First, because we are dealig with a lim sup we will eed a maximal iequality. LEM 5.3 Let X be idepedet ad symmetric that is, X ad X have the same distributio. The for all a R P max S > a PS > a. k This is similar to Kolmogorov s maximal iequality. However, we will get a stroger boud i the Gaussia case below. First, we prove the lemma. Proof: Let A be the set i bracket o the LHS ad B be the set i bracket o the RHS. Defie A k = {S k > a but S j a, j < k}. Note that Moreover, PB k PA k B. 4 PA k B PA k {S S k } = PA k PS S k 0 PA k, by symmetry. Pluggig back ito 4 gives the result.
Lecture 5: More o the a.s. covergece of sums 9 LEM 5.4 For x > 0, x x 3 e x / x e y / dy x e x /. Proof: By the chage of variable y = x + z ad usig e z / x e y / dy e x / For the other directio, by differetiatio x 0 e xz dz = e x / x. 3y 4 e y / dy = x x 3 e x /. We come back to the proof of the mai theorem. Proof: Defie h = log log. Upper boud. The upper boud follows from BC. Let K > close to ad c = KhK. I words we wat to show that S k /hk is smaller tha K evetually. By the lemmas above Note that P max S k K k > c PS K > c SK = P K / > c K / K / exp π c c K c K = K log log K = Klog +log log K = log K +loglog K K. For large eough P max S k K > c exp c K =. K log K K, which is summable. By BC, evetually for K k < K S k max k K S c = KhK Khk.
Lecture 5: More o the a.s. covergece of sums 0 Hece, lim sup k S k hk K. Lower boud. For the other directio, we use a trick similar to the head rus problem. We divide time ito big idepedet blocks. Let N > 0 large. By the above lemma, for ε > 0 P S N + S N > εhn + N π y where y = ε log logn + N, y 3 e y /, 5 so that the RHS i 5 is of order ε which sums to +. By BC by idepedece the evet i bracket i 5 occurs i.o. Moreover, by the upper boud argumet S N > hn, a.s. for large eough. Hece for ifiitely may so that S N + > εhn + N hn, lim sup k up to logarithmic factors. S k lim sup hk ε S N + hn + N N, Refereces Dur0 Rick Durrett. Probability: theory ad examples. Cambridge Series i Statistical ad Probabilistic Mathematics. Cambridge Uiversity Press, Cambridge, 00. Shi96 A. N. Shiryaev. Probability, volume 95 of Graduate Texts i Mathematics. Spriger-Verlag, New York, 996. Wil9 David Williams. Probability with martigales. Cambridge Mathematical Textbooks. Cambridge Uiversity Press, Cambridge, 99.
Lecture 5: More o the a.s. covergece of sums A Proof of Kroecker s lemma Proof: Let b m = m k= x k/a k. Notig that x m = a m b m b m, we have settig a 0 = b 0 = 0 x m = a m b m a m b m a a m= m= m= = a b + a m b m a m b m a m= m= a m a m = b b m. a m= Sice b b < + ad the a s are o-decreasig, the average o the RHS coverges to b. Exercise. B St-Petersburg paradox A importat example: EX 5.5 St-Petersburg paradox Cosider a IID sequece with Clearly EX = +. Note that P X = j = j, j. P X = Θ, ideed it is a geometric series ad the sum is domiated by the first term ad therefore we caot apply the WLLN. Istead we apply the WLLN for triagular arrays to a properly ormalized sum. We take X,k = X k ad b = log. We check the two coditios. First P X,k > b = Θ 0. log k= To check the secod oe, let X,k = X,k X,k b ad ote EX,k = log +log log j= j j log +log log = log.
Lecture 5: More o the a.s. covergece of sums So Fially, b k= EX,k = log log 0. a = so that ad k= log +log log EX,k = EX, = j j = log +log log, j= S a b P 0, S log P. O the other had, ote that P X K log = Ω K log which is ot summable. By BC, sice K is arbitrary,, lim sup S log = +, a.s. More geerally, usig the radom series results above see D: THM 5.6 Let X be IID with E X = + ad S = k X k. Let a be a sequece with a / icreasig. The lim sup S /a = 0 or + accordig as P X a < + or = +.