Steve R. Dubar Departmet of Mathematics 203 Avery Hall Uiversity of Nebraska-Licol Licol, NE 68588-0130 http://www.math.ul.edu Voice: 402-472-3731 Fax: 402-472-8466 Topics i Probability Theory ad Stochastic Processes Steve R. Dubar Law of the Iterated Logarithm Ratig Mathematicias Oly: prologed scees of itese rigor. 1
Sectio Starter Questio Key Cocepts 1. The Law of the Iterated Logarithm tells very precisely how far the umber of successes i a coi-tossig game will make excursios from the average value. 2. The Law of the Iterated Logarithm is a high-poit amog icreasigly precise limit theorems characterizig how far the umber of successes i a coi-tossig game will make excursios from the average value. The theorems start with the Strog Law of Large Numbers ad the Cetral Limit Theorem, to Hausdorff s Estimate, ad the Hardy-Littlewood Estimate leadig to the Law of the Iterated Logarithm. 3. Khichi s Law of the Iterated Logarithm says: Almost surely, for all ɛ > 0, there exist ifiitely may such that S p > (1 ɛ) 2p(1 p) l(l()) ad furthermore, almost surely, for all ɛ > 0, for every larger tha a threshold value N S p < (1 + ɛ) 2p(1 p) l(l()). 2
Vocabulary 1. The limsup, abbreviatio for limit superior is a refied ad geeralized otio of limit, beig the largest depedet-variable subsequece limit. That is, amog all subsequeces of idepedet-variable values tedig to some idepedet-variable value, usually ifiity, there will be a correspodig depedet-variable subsequece. Some of these depedet-variable sequeces will have limits, ad amog all these, the largest is the limsup. 2. The limif, abbreviatio for limit iferior is aalogous, it is the least of all depedet-variable subsequece limits. 3. Khichi s Law of the Iterated Logarithm says: Almost surely, for all ɛ > 0 there exist ifiitely may such that S p > (1 ɛ) 2p(1 p) l(l()) ad furthermore, almost surely, for all ɛ > 0, for every larger tha a threshold value N S p < (1 + ɛ) 2p(1 p) l(l()). Mathematical Ideas Overview We agai cosider the umber of successes i a coi-tossig game. That is, we cosider the sum S where the idepedet, idetically distributed radom variables i the sum S = X 1 + + X are the Beroulli radom variables X i = +1 with probability p ad X i = 0 with probability q = 1 p. Note that the mea µ = p is ad the variace is σ 2 = p(1 p) for each of the summads X i. 3
The Strog Law of Large Numbers says that S p lim with probability 1 i the sample space of all possible coi flips. This says the deomiator is too strog, it codeses out all variatio i the sum S. The Cetral Limit Theorem applied to this sequece of coi flips says lim = 0 S p p(1 p) = Z where Z N(0, 1) is a ormal radom variable ad the limit is iterpreted as covergece i distributio. I fact, this implies that for large about 68% of the poits i the sample space of all possible coi flips satisfy S p 1 p(1 p) ad about 95% of the poits i the sample space of all possible coi flips satisfy S p 2. p(1 p) This says the deomiator is too weak, it does t codese out eough iformatio. I fact, usig the Kolmogorov zero-oe law ad the Cetral Limit Theorem, almost surely ad almost surely lim if lim sup S p p(1 p) = S p p(1 p) = +. The Strog Law ad the Cetral Limit Theorem together suggest that somewhere i betwee ad we might be able to make stroger statemets about covergece ad the variatio i the sequece S. 4
I fact, Hausdorff s estimate tells us: lim S p 1/2+ɛ = 0 with probability 1 i the sample space of all possible coi flips for all values of ɛ > 0. This says the deomiator 1/2+ɛ is still too strog, it codeses out too much iformatio. Eve better, Hardy ad Littlewood s estimate tells us: lim S p costat l with probability 1 i the sample space of all possible coi flips for all values of ɛ > 0. I a way, this says l ) is still a little too strog, it codeses out most iformatio. Khichi s Law of the Iterated Logarithm has a deomiator that is just right. It tells us very precisely how the deviatios of the sums from the mea vary with. Usig a method due to Erdös, it is possible to refie the law eve more, but for these otes a refiemet is probably past the poit of dimiishig returs. Like the Cetral Limit Theorem, the Law of the Iterated Logarithm illustrates i a astoishigly precise way that eve completely radom sequeces obey precise mathematical laws. Khichi s Law of the Iterated Logarithm says that: Almost surely, for all ɛ > 0, there exist ifiitely may such that S p > (1 ɛ) p(1 p) 2 l(l()) ad furthermore, almost surely, for all ɛ > 0, for every larger tha a threshold value N depedig o ɛ S p < (1 + ɛ) p(1 p) 2 l(l()). These appear i a slightly o-stadard way, with the additioal factor 2 l l times the stadard deviatio from the Cetral Limit Theorem to emphasize the similarity to ad the differece from the Cetral Limit Theorem. 5
Theorem 1 (Law of the Iterated Logarithm). With probability 1: lim sup S p 2p(1 p) l(l()) = 1. This meas that with probability 1 for ay ɛ > 0, oly fiitely may of the evets: S p > (1 + ɛ) 2p(1 p) l(l()) occur; o the other had, with probability 1, occurs for ifiitely may. S p > (1 ɛ) 2p(1 p) l(l()) For reasos of symmetry, for ɛ > 0, the iequality S p < (1 + ɛ) 2p(1 p) l(l()) ca oly occur for fiitely may ; while S p < (1 ɛ) 2p(1 p) l(l()) must occur for ifiitely may. That is, lim if S p 2p(1 p) l(l()) = 1 with probability 1. Compare the Law of the Iterated Logarithm to the Cetral Limit Theorem. The Cetral Limit Theorem, says that (S p)/ p(1 p) is approximately distributed as a N(0, 1) radom variable for large. Therefore, for a large but fixed, there is probability about 1/6 that the values of (S p)/ p(1 p) ca exceed the stadard deviatio 1, or S p > p(1 p). For a fixed but large, with probability about 0.025, (S p)/ p(1 p) ca exceed twice the stadard deviatio 2, or (S p) > 2 p(1 p). The Law of the Iterated Logarithm tells us the more precise iformatio that there are ifiitely may such that S p > (1 ɛ) 2p(1 p) l(l()) 6
for ay ɛ > 0. The Law of the Iterated Logarithm does ot tell us how log we will have to wait betwee such repeated crossigs however, ad the wait ca be very, very log ideed, although it must (with probability 1) evetually occur agai. Moreover, the Law of the Iterated Logarithm tells us i additio that S p < (1 ɛ) 2p(1 p) l(l()) must occur for ifiitely may. Khichi s Law of the Iterated Logarithm also applies to the cumulative fortue i a coi-tossig game, or equivaletly, the positio i a radom walk. Cosider the idepedet Beroulli radom variables Y i = +1 with probability p ad Y i = 1 with probability q = 1 p. The mea is µ = 2p 1 ad the variace is σ 2 = 4p(1 p) for each of the summads Y i. The cosider the sum T = Y 1 + + Y with mea (2p 1) ad variace 4p(1 p). Sice Y = 2X 1, the T = 2S ad S = 1T 2 +. The applyig 2 the Law of the Iterated Logarithm says that with probability 1 for ay ɛ > 0, oly fiitely may of the evets: T (2p 1) > (1 + ɛ)2 2p(1 p) l(l()) occur; o the other had, with probability 1, T (2p 1) > (1 ɛ)2 2p(1 p) l(l()) occurs for ifiitely may. This meas that the fortue must (with probability 1) oscillate back ad forth across the et zero axis ifiitely ofte, crossig the upper ad lower boudaries: ±(1 ɛ)2 2p(1 p) l(l()) The statemet puts some stregth behid a uderstadig of the log-term swigs backs ad forth i value of a radom process. It also implies a form of recurrece, that is, a radom walk must visit every iteger value. The Law of the Iterated Logarithm for Beroulli trials stated here is a special case of a eve more geeral theorem first formulated by Kolmogorov i 1929. It is also possible to formulate eve stroger ad more geeral theorems! The proof here uses the Large Deviatios ad Moderate Deviatios results with the Borel-Catelli Lemmas. I aother directio, the Law of the 7
Iterated Logarithm ca be proved usig ivariace theorems, so it is distatly related to the Cetral Limit Theorem. Figure 1 gives a impressio of the growth of the fuctio i the Law of the Iterated Logarithm compared to the square root fuctio. Figure 2 gives a impressio of the Law of the Iterated Logarithm by showig a piecewise liearly coected graph of 2000 steps of S p with p = q = 1/2. I this figure, the radom walk must retur agai to cross the blue curves with (1 ɛ) = 0.9 ifiitely may times, but may oly cross the red curve with 1+ɛ = 1.1 fiitely may times. Of course, this is oly a schematic impressio sice a sigle radom walk (possibly atypical, from the egligible set!) o the fiite iterval 0 2000 ca oly suggest the almost sure ifiitely may crossigs of (1 ɛ)α(x) for ay ɛ > 0. Figure 3 gives a compariso of impressios of four of the limit theorems. The idividual figures deliberately are spaghetti graphs to give a impressio of the esemble of sample paths. Each figure shows a differet scalig of the same 15 sample paths for a sequece of 100,000 fair coi flips, each path a differet color. Note that the steps axis has a logarithmic scale, meaig that the shape of the paths is distorted although it still gives a impressio of the radom sums. The top left figure shows S / p covergig to 0 for all paths i accord with the Strog Law of Large Numbers. The top right figure plots the scalig (S p)/ 2p(1 p). For large values of steps the values over all paths is a distributio ragig from about 2 to 2, cosistet with the Cetral Limit Theorem. The lower left figure plots (S p)/ 0.6 as a illustratio of Hausdorff s Estimate with ɛ = 0.1. It appears that the scaled paths are very slowly covergig to 0, the rage for = 100,000 is withi 0.5, 0.5. The lower right figure shows (S p)/ 2p(1 p) l(l(x)) alog with lies ±1 to suggest the coclusios of the Law of the Iterated Logarithm. It suggests that all paths are usually i the rage 1, 1 but with each path makig a few excursios outside the rage. Hausdorff s Estimate Theorem 2 (Hausdorff s Estimate). Almost surely, for ay ɛ > 0, S p = o ( ɛ+1/2) as. Proof. The proof resembles the proof of the Strog Law of Large Numbers for idepedet radom variables with mea 0 ad uiformly bouded 4th 8
y 50 40 30 20 10 0 0 1000 2000 x Figure 1: The Iterated Logarithm fuctio α(x) = 2p(1 p)x l(l(x)) i gree alog with fuctios (1 + ɛ)α(x) i red ad (1 ɛ)α(x) i blue, with ɛ = 0.1. For compariso, the square root fuctio 2p(1 p)x is i black. 9
y 50 40 30 20 10 0 0 1000 2000 x Figure 2: A impressio of the Law of the Iterated Logarithm usig a piecewise liearly coected graph of 2000 steps of S p with p = q = 1/2 with the blue curve with (1 ɛ)α(x) ad the red curve with (1 + ɛ)α(x) for ɛ = 0.1 ad α(x) = 2p(1 p)x l(l(x)). 10
Figure 3: A compariso of four of the limit theorems. The idividual figures deliberately are spaghetti graphs to give a impressio of the esemble of sample paths. Each figure shows a differet scalig of the same 15 sample paths for a sequece of 100,000 fair coi flips, each path a differet color. Note that the steps axis has a logarithmic scale, meaig that the shape of the sample paths is distorted. 11
momets. That proof showed that usig the idepedece ad idetical distributio assumptios E S 4 = E X 4 1 + 3( 1)E X 2 1 2 C + 3 2 σ 4 C 1 2. The adaptig the proof of the Markov ad Chebyshev iequalities with fourth momets E S 4 C 1 2 4. 4 Use the Corollary that if E X coverges, the the sequece (X ) 1 =1 coverges almost surely to 0. By compariso, S / 0 a.s. Usig the same set of ideas E S 4 C 1 2 α 4α =1 E S 4 4 coverges so that provided that α > 3/4. The usig the same lemma S / α 0 for α > 3/4 for a simple versio of Hausdorff s Estimate. Now adapt this proof to higher momets. Let k be a fixed positive iteger. Recall the defiitio R (ω) = S (ω) p = (X k p) = (X k where X k = k=1 k=1. Expadig the product R 2k X k p ad cosider E R 2k results i a sum of products of the idividual radom variables X i of the form X i 1 X i 2 X i 2k. Each product X i 1 X i 2 X i 2k results from a selectio or mappig from idices {1, 2,..., 2k} to the set {1,..., }. Note that if a idex j {1, 2,..., } is selected oly oce so that X j appears oly oce i the product X i 1 X i 2 X i k, the E X i 1 X i 2 X i k = 0 by idepedece. Further otice that for all sets of idices E X i 1 X i 2 X i k 1. Thus E R 2k = E X i 1 X i 2 X i 2k N(k, ), 1 i 1,...,i 2k where N(k, ) is the umber of fuctios from {1,..., 2k} to {1,..., } that take each value at least twice. Let M(k) be the umber of partitios of {1,..., 2k} ito subsets each cotaiig at least two elemets. If P is such a partitio, the P has at most k elemets. The umber of fuctios N(k, ) 12
that are costat o each elemet of P is at most k. Thus, N(k, ) k M(k). Now let ɛ > 0 ad cosider ( ) E ɛ 1/2 2k R 2kɛ k N(k, ) 2kɛ M(k). Choose k > 1 2ɛ. The 1 ( ) E ɛ 1/2 2k R <. Recall the Corollary 2 appearig i the sectio o the Borel-Catelli Lemma: Let (X ) 0 be a sequece of radom variables. If =1 E X coverges, the X coverges to zero, almost surely. By this corollary, the sequece of radom variables ( ɛ 1/2 R ) 0 almost surely as. This meas that for each ɛ > 0, there is a egligible evet (depedig o ɛ) outside of which ɛ 1/2 R coverges to 0. Now cosider a coutable set of values of ɛ tedig to 0. Sice a coutable uio of egligible evets is egligible, the for each ɛ > 0, ɛ 1/2 R coverges to 0 almost surely. Hardy-Littlewood Estimate Theorem 3 (Hardy-Littlewood Estimate). ( ) S p = O l a.s. for. Remark. The proof shows that S p l a.s. for. Proof. The proof uses the Large Deviatios Estimate as well as the Borel- Catelli Lemma. Recall the Large Deviatios Estimate says S P p + ɛ e h +(ɛ), 13
where ( ) ( ) p + ɛ 1 p ɛ h + (ɛ) = (p + ɛ) l + (1 p ɛ) l. p 1 p Note that as ɛ 0, h + (ɛ) = Note that ad take ɛ = The l P S p + ɛ ad ote that ɛ2 + 2p(1 p) O(ɛ3 ). = P S p ɛ, P S p l e h + ( ) l h + = l 2p(1 p) + o ( l ). ( ) 1, ( ( sice O l ) ) 3/2 = o ( 1 ). Thus, the probability is less tha or equal to the followig: exp ( h + ( l ( Hece exp ( h + )) is coverget because of the followig iequalities ( = exp = exp 1 2p(1 p) ( l 2p(1 p) = 1 2p(1 p) exp (o (1)). )) l 1 2p(1 p), ad ) l + o (1) ) exp (o (1)) 1 1 2p(1 p) p(1 p) 1 4 2p(1 p) 1 2 1 2p(1 p) 2 1 2p(1 p) 2 1 2p(1 p) 2. 14
Thus, ad so 1 P S p > l <, P S p l i.o. = 1. Proof of Khichi s Law of Iterated Logarithm Theorem 4 (Khichi s Law of Iterated Logarithm). Almost surely, lim sup S p 2p(1 p) l (l ) = 1, ad lim if S p 2p(1 p) l (l ) = 1. First establish a two lemmas, ad for coveiece, let α() = 2p(1 p) l (l ). Lemma 5. For all positive a ad δ ad large eough, (l ) a2 (1+δ) < P S p > aα() < (l ) a2 (1 δ). Proof of Lemma 5. Recall that the Large Deviatios Estimate gives Note that α() P R aα() = P S p aα() S = P p aα() ( ( )) aα()a exp h +. 0 as. The ( ) aα() h + = a 2 2p(1 p) ( ) ( 2 (α() ) ) 3 α() + O, 15
ad so ( ) ( ) aα() α() h + = a 2 3 l (l ) + O a 2 (1 δ) l (l ) 2 for large eough. This meas that S P p aα() exp ( a 2 (1 δ) l (l ) ) = (l ) a2 (1 δ). Sice l (l ) = o ( 1/6), the results of the Moderate Deviatios Theorem apply to give S P p aα() S p(1 p) = P p a 2 l (l ) 1 exp ( a 2 l (l ) ) 2πa 2 l (l ) 1 = 2a (l ) a2. π l (l ) Sice ( ) l (l ) = o (l ) a2 δ, S P p aα() (l ) a2 (1+δ) for large eough. Lemma 6 (12.5, Kolmogorov Maximal Iequality). Suppose (Y ) 1 are idepedet radom variables. Suppose further that E Y = 0 ad Var(Y ) = σ 2. Defie T := Y 1 + + Y. The P max T k b 4 1 k 3 P T b 2σ. Remark. Lemma 6 is a example of a class of lemmas called maximal iequalities. Here are two more examples of maximal iequalities. Lemma 7 (Karli ad Taylor, page 280). Let (Y ) 1 be idetical idepedetly distributed radom variables with E Y = 0 ad Var(Y ) = σ 2 <. Defie T = k=1 Y k. The ɛ 2 P max T k > ɛ σ 2. 0 k 16
Lemma 8 (Karli ad Taylor, page 280). Let (X ) 0 be a submartigale for which X 0. For λ > 0, λp max X k > λ E X. 0 k Proof of Lemma 6. Sice the Y k s are idepedet, the Var(T T k ) = ( k)σ 2 for 1 k. Chebyshev s Iequality tells us that P T T k 2σ 1 Var(T T k ) 4σ 2 Note that P max T k b = 0 k = 1 k 4 3 4. P T 1 < b,..., T k 1 < b, ad T k b k=1 P T 1 < b,..., T k 1 < b, ad T k b 4 3 P T T k 2σ k=1 = 4 P T 1 < b,..., T k 1 < b, ad T k b ad T T k 2σ 3 4 3 k=1 P T 1 < b,..., T k 1 < b, ad T k b ad T b 2σ k=1 4 3 P T b 2σ. Remark. Note that the secod part of the Law of the Iterated Logarithm, S lim if p = 1 follows by symmetry from the first part by replacig α() p with (1 p) ad S with S. Remark. The proof of the Law of the Iterated Logarithm proceeds i two parts. First it suffices to show that lim sup S p α() The secod part of the proof is to establish that lim sup S p α() < 1 + η for η > 0, a.s.. (1) > 1 η for η > 0, a.s. (2) 17
It will oly take a subsequece to prove (2). However this will ot be easy because it will use the secod Borel-Catelli Lemma, which requires idepedece. Remark. The followig is a simplified proof givig a partial result for iteger sequeces with expoetial growth. This simplified proof illustrates the basic ideas of the proof. Fix γ > 1 ad let k := γ k. The P S k p k (1 + η)α( k ) < (l k ) (1+η)2 (1 δ) ( ) = O k (1+η)2 (1 δ). Choose δ so that (1 + η) 2 (1 δ) < 1. The P S k k p (1 + η)α( k ) <. k 1 By the first Borel-Catelli lemma, ad so or P S k k p (1 + η)α( k ) i.o. = 0, S k k p P lim sup k α( k ) (1 + η) = 1, S k k p (1 + η) a.s. α( k ) The full proof of the Law of the Iterated Logarithm takes more work to complete. Proof of (1) i the Law of the Iterated Logarithm. Fix η > 0 ad let γ > 1 be a costat chose later. For k Z, let k = γ k. The proof cosists of showig that P max (S p) (1 + η)α( k ) <. k+1 k 1 From Lemma 6 P max (S p) (1 + η)α( k ) k+1 k 1 4 R 3 P k+1 (1 + η)α( k ) 2 k+1 p(1 p), 18
where R = S p. We do kow that P max (S p) (1 + η)α( k ) k+1 4 3 P R k+1 (1 + η)α( k ) 2 k+1 p(1 p) Note that k+1 = o (α( k )) because this is approximately γ k+1 compared to c 1 γk l(l γ γ ) = c 1 γ k/2 l(l γ) + l(k),. (3) which is the same as γ 1/2 compared to c 1 c2 + l(k). The 2 k+1 p(1 p) < 1ηα( 2 k) for large eough k. Usig this iequality i the right had side of Equatio (3), we get P max S p (1 + η)α( k ) 4 k+1 3 P S k+1 k+1 p (1 + η/2)α( k ). Now, α( k+1 ) γα( k ). Choose γ so that 1 + η/2 > (1 + η/4) γ. The for large eough k, we have (1 + η/2)α( k ) > (1 + η/4)α( k+1 ). Now P max S p (1 + η)α( k ) 4 k+1 3 P S k+1 k+1 p (1 + η/4)α( k+1 ). Use Lemma 5 with a = (1 δ) 1 = (1 + η/4). The we get P max S p (1 + η)α( k ) 4 k+1 3 (l k+1) (1+η/4) for k large. Note that (l k+1 ) (1+η/4) (l γ) (1+η/4) k (1+η/4), which is the geeral term of a coverget series so P max R (1 + η)α( k ) <. k+1 k 1 19
The the first Borel-Catelli Lemma implies that or equivaletly The i particular max R (1 + η)α( k ) i.o. with probability 0. k+1 max S p < (1 + η)α( k ) a.s. for large eough k. k+1 max S p < (1 + η)α( k ) a.s. for large eough k. k < k+1 This i tur implies that almost surely S p < (1 + η)α(). for > k ad large eough k which establishes (1). Proof of (2) i the Law of the Iterated Logarithm. Cotiue with the proof of Equatio (2). To prove the secod part, it suffices to show that there exists k so that R k (1 η)α( k ) i.o. almost surely. Let k = γ k for γ chose later with γ Z sufficietly large. The proof will show ( P R γ R γ 1 1 η ) α(γ ) =, (4) 2 ad also 1 R γ 1 η 2 α(γ ) a.s. for large eough. (5) dist. Note that R γ R γ 1 = R γ γ 1. It suffices to cosider ( P R γ γ 1 1 η ) α(γ ). 2 Note that α(γ γ 1 ) α(γ ) = = c (γ γ 1 ) l (l (γ γ 1 )) cγ l (l γ ) ( ( )) 1 1 ) l ( l γ + l 1 1 γ γ l ( l γ) 1 1 γ. 20
Choose γ Z so that 1 η 2 1 η < 1 1 γ. 4 The ote that we ca choose large eough so that ( ) 1 η ( 2) 1 η < α(γ γ 1 ) α(γ ) or ( 1 η ) ( α(γ ) < 1 η ) α(γ γ 1 ). 2 4 Now cosiderig equatio (4), ad the iequality above P R γ R γ 1 ( 1 η 2 4 ) α(γ ) P R γ γ 1 ( 1 η 4 Now usig Lemma 5, with a = (1 + δ) 1 = ( 1 η 4), we get ) α(γ γ 1 ). ( P R γ R γ 1 1 η ) α(γ ) 2 l ( γ γ 1) ( (1 η 4) = ( l γ + l 1 1 )) (1 η 4). γ The series with this as its terms diverges. Thus, we see that Equatio (4) has bee prove. Now otice that ad so which meas that α(γ ) = cγ l (l γ ) = cγ (l + l l γ) α(γ 1 ) = cγ 1 (l( 1) + l l γ), γα(γ 1 ) = cγ (l( 1) + l l γ). Thus, α(γ ) γα(γ 1 ). Now choose γ so that η γ > 4. The ηα(γ ) η γα(γ 1 ) > 4α(γ 1 ) for large eough. 21
Thus, we have R γ 1 η 2 α(γ ) R γ 1 2α(γ 1 ) sice R γ 1 η 2 α(γ ) R γ 1 2α(γ 1 ). Now use (4) ad we see that R γ 1 < 2α(γ 1 ) happes almost surely for large eough. Now R γ R γ 1 is a sequece of idepedet radom variables, ad so the secod Borel-Catelli Lemma says that almost surely ( R γ R γ 1 > 1 η ) α(γ ) i.o. 2 Addig this with Equatio (5), we get that This is eough to show that R γ > (1 η) α(γ ) i.o. lim if S p α() > 1 η a.s., which is eough to show the oly remaiig part of Khichi s Law of the Iterated Logarithm. Sources This sectio is adapted from: W. Feller, i Itroductio to Probability Theory ad Volume I, Chapter III, ad Chapter VIII, ad also E. Lesige, Heads or Tails: A Itroductio to Limit Theorems i Probability, Chapter 12, America Mathematical Society, Studet Mathematical Library, Volume 28, 2005. Some of the ideas i the proof of Hausdorff s Estimate are adapted from J. Lamperti, Probability: A Survey of the Mathematical Theory, Secod Editio, Chapter 8. Figure 3 is a recreatio of a figure i the Wikipedia article o the Law of the Iterated Law. 22
Algorithms, Scripts, Simulatios Algorithm Scripts R script for compariso figures. 1 p <- 0.5 2 k <- 15 3 <- 100000 4 coiflips <- array ( ruif (*k) <= p, dim =c(,k)) 5 S <- apply ( coiflips, 2, cumsum ) 6 steps <- c (1: ) 7 8 steps2 <- steps 2: 9 S2 <- S 2:, 10 steps3 <- steps 3: 11 S3 <- S 3:, 12 13 oes <- cbid ( matrix (1, -2,1), matrix (-1, -2,1) ) 14 15 par ( mfrow = c (2,2) ) 16 17 matplot ((S- steps *p)/ steps, 18 log ="x", type ="l", lty = 1, ylab ="", mai =" Strog Law ") 19 matplot ((S- steps *p)/ sqrt (2*p*(1 -p)* steps ), 20 log ="x", type ="l", lty = 1, ylab ="", mai =" Cetral Limit Theorem ") 21 matplot ((S- steps *p)/( steps ^(0.6) ), 22 log ="x", type ="l", lty = 1, ylab ="", mai =" Hausdorff s Estimate ") 23 ## matplot ((S2 - steps2 *p)/ sqrt ( steps2 * log ( steps2 )), log ="x", xlim =c(1,), type ="l", lty = 1) 24 matplot ( steps3, (S3 - steps3 *p)/ sqrt (2*p*(1 -p)* steps3 * log ( log ( steps3 ))), 25 log ="x", xlim =c(1,), type ="l", lty = 1, ylab ="", mai =" Law of Iterated Logarithm ") 26 matlies ( steps3, oes, type ="l", col =" black ") 23
Problems to Work for Uderstadig 1. The multiplier of the variace 2p(1 p) fuctio l(l()) grows very slowly. To uderstad how very slowly, calculate a table with = 10 j ad 2 l(l()) for j = 10, 20, 30,..., 100. (Remember, i mathematical work above calculus, l(x) is the atural logarithm, base e, ofte writte l(x) i calculus ad below to distiguish it from the commo or base-10 logarithm. Be careful, some software ad techology caot directly calculate with magitudes this large.) 2. Cosider the sequece a = ( 1) /2 + ( 1) for = 1, 2, 3.... Here x is the floor fuctio, the greatest iteger less tha or equal to x, so 1 = 1, 3/2 = 1, 8/3 = 2, 3/2 = 2, etc. Fid lim sup a ad Does the sequece a have a limit? lim if a. 3. Show that the secod part of the Law of the Iterated Logarithm, S lim if p = 1 follows by symmetry from the first part by α() replacig p with (1 p) ad S with S. Solutios to Problems 24
Readig Suggestio: Refereces 1 William Feller. A Itroductio to Probability Theory ad Its Applicatios, Volume I, volume I. Joh Wiley ad Sos, third editio, 1973. QA 273 F3712. 2 Joh W. Lamperti. Probability: A Survey of the Mathematical Theory. Wiley Series i Probability ad Statistics. Wiley, secod editio editio, 1996. 3 Emmauel Lesige. Heads or Tails: A Itroductio to Limit Theorems i Probability, volume 28 of Studet Mathematical Library. America Mathematical Society, 2005. Outside Readigs ad Liks: 1. 2. 3. 4. I check all the iformatio o each page for correctess ad typographical errors. Nevertheless, some errors may occur ad I would be grateful if you would alert me to such errors. I make every reasoable effort to preset curret ad accurate iformatio for public use, however I do ot guaratee the accuracy or timeliess of iformatio o this website. Your use of the iformatio from this website is strictly volutary ad at your risk. 25
I have checked the liks to exteral sites for usefuless. Liks to exteral websites are provided as a coveiece. I do ot edorse, cotrol, moitor, or guaratee the iformatio cotaied i ay exteral website. I do t guaratee that the liks are active at all times. Use the liks here with the same cautio as you would all iformatio o the Iteret. This website reflects the thoughts, iterests ad opiios of its author. They do ot explicitly represet official positios or policies of my employer. Iformatio o this website is subject to chage without otice. Steve Dubar s Home Page, http://www.math.ul.edu/~sdubar1 Email to Steve Dubar, sdubar1 at ul dot edu Last modified: Processed from L A TEX source o March 22, 2018 26