Topics in Probability Theory and Stochastic Processes Steven R. Dunbar. Law of the Iterated Logarithm
|
|
- Bernard Russell
- 5 years ago
- Views:
Transcription
1 Steve R. Dubar Departmet of Mathematics 203 Avery Hall Uiversity of Nebraska-Licol Licol, NE Voice: Fax: Topics i Probability Theory ad Stochastic Processes Steve R. Dubar Law of the Iterated Logarithm Ratig Mathematicias Oly: prologed scees of itese rigor. 1
2 Sectio Starter Questio Key Cocepts 1. The Law of the Iterated Logarithm tells very precisely how far the umber of successes i a coi-tossig game will make excursios from the average value. 2. The Law of the Iterated Logarithm is a high-poit amog icreasigly precise limit theorems characterizig how far the umber of successes i a coi-tossig game will make excursios from the average value. The theorems start with the Strog Law of Large Numbers ad the Cetral Limit Theorem, to Hausdorff s Estimate, ad the Hardy-Littlewood Estimate leadig to the Law of the Iterated Logarithm. 3. Khichi s Law of the Iterated Logarithm says: Almost surely, for all ɛ > 0, there exist ifiitely may such that S p > (1 ɛ) 2p(1 p) l(l()) ad furthermore, almost surely, for all ɛ > 0, for every larger tha a threshold value N S p < (1 + ɛ) 2p(1 p) l(l()). 2
3 Vocabulary 1. The limsup, abbreviatio for limit superior is a refied ad geeralized otio of limit, beig the largest depedet-variable subsequece limit. That is, amog all subsequeces of idepedet-variable values tedig to some idepedet-variable value, usually ifiity, there will be a correspodig depedet-variable subsequece. Some of these depedet-variable sequeces will have limits, ad amog all these, the largest is the limsup. 2. The limif, abbreviatio for limit iferior is aalogous, it is the least of all depedet-variable subsequece limits. 3. Khichi s Law of the Iterated Logarithm says: Almost surely, for all ɛ > 0 there exist ifiitely may such that S p > (1 ɛ) 2p(1 p) l(l()) ad furthermore, almost surely, for all ɛ > 0, for every larger tha a threshold value N S p < (1 + ɛ) 2p(1 p) l(l()). Mathematical Ideas Overview We agai cosider the umber of successes i a coi-tossig game. That is, we cosider the sum S where the idepedet, idetically distributed radom variables i the sum S = X X are the Beroulli radom variables X i = +1 with probability p ad X i = 0 with probability q = 1 p. Note that the mea µ = p is ad the variace is σ 2 = p(1 p) for each of the summads X i. 3
4 The Strog Law of Large Numbers says that S p lim with probability 1 i the sample space of all possible coi flips. This says the deomiator is too strog, it codeses out all variatio i the sum S. The Cetral Limit Theorem applied to this sequece of coi flips says lim = 0 S p p(1 p) = Z where Z N(0, 1) is a ormal radom variable ad the limit is iterpreted as covergece i distributio. I fact, this implies that for large about 68% of the poits i the sample space of all possible coi flips satisfy S p 1 p(1 p) ad about 95% of the poits i the sample space of all possible coi flips satisfy S p 2. p(1 p) This says the deomiator is too weak, it does t codese out eough iformatio. I fact, usig the Kolmogorov zero-oe law ad the Cetral Limit Theorem, almost surely ad almost surely lim if lim sup S p p(1 p) = S p p(1 p) = +. The Strog Law ad the Cetral Limit Theorem together suggest that somewhere i betwee ad we might be able to make stroger statemets about covergece ad the variatio i the sequece S. 4
5 I fact, Hausdorff s estimate tells us: lim S p 1/2+ɛ = 0 with probability 1 i the sample space of all possible coi flips for all values of ɛ > 0. This says the deomiator 1/2+ɛ is still too strog, it codeses out too much iformatio. Eve better, Hardy ad Littlewood s estimate tells us: lim S p costat l with probability 1 i the sample space of all possible coi flips for all values of ɛ > 0. I a way, this says l ) is still a little too strog, it codeses out most iformatio. Khichi s Law of the Iterated Logarithm has a deomiator that is just right. It tells us very precisely how the deviatios of the sums from the mea vary with. Usig a method due to Erdös, it is possible to refie the law eve more, but for these otes a refiemet is probably past the poit of dimiishig returs. Like the Cetral Limit Theorem, the Law of the Iterated Logarithm illustrates i a astoishigly precise way that eve completely radom sequeces obey precise mathematical laws. Khichi s Law of the Iterated Logarithm says that: Almost surely, for all ɛ > 0, there exist ifiitely may such that S p > (1 ɛ) p(1 p) 2 l(l()) ad furthermore, almost surely, for all ɛ > 0, for every larger tha a threshold value N depedig o ɛ S p < (1 + ɛ) p(1 p) 2 l(l()). These appear i a slightly o-stadard way, with the additioal factor 2 l l times the stadard deviatio from the Cetral Limit Theorem to emphasize the similarity to ad the differece from the Cetral Limit Theorem. 5
6 Theorem 1 (Law of the Iterated Logarithm). With probability 1: lim sup S p 2p(1 p) l(l()) = 1. This meas that with probability 1 for ay ɛ > 0, oly fiitely may of the evets: S p > (1 + ɛ) 2p(1 p) l(l()) occur; o the other had, with probability 1, occurs for ifiitely may. S p > (1 ɛ) 2p(1 p) l(l()) For reasos of symmetry, for ɛ > 0, the iequality S p < (1 + ɛ) 2p(1 p) l(l()) ca oly occur for fiitely may ; while S p < (1 ɛ) 2p(1 p) l(l()) must occur for ifiitely may. That is, lim if S p 2p(1 p) l(l()) = 1 with probability 1. Compare the Law of the Iterated Logarithm to the Cetral Limit Theorem. The Cetral Limit Theorem, says that (S p)/ p(1 p) is approximately distributed as a N(0, 1) radom variable for large. Therefore, for a large but fixed, there is probability about 1/6 that the values of (S p)/ p(1 p) ca exceed the stadard deviatio 1, or S p > p(1 p). For a fixed but large, with probability about 0.025, (S p)/ p(1 p) ca exceed twice the stadard deviatio 2, or (S p) > 2 p(1 p). The Law of the Iterated Logarithm tells us the more precise iformatio that there are ifiitely may such that S p > (1 ɛ) 2p(1 p) l(l()) 6
7 for ay ɛ > 0. The Law of the Iterated Logarithm does ot tell us how log we will have to wait betwee such repeated crossigs however, ad the wait ca be very, very log ideed, although it must (with probability 1) evetually occur agai. Moreover, the Law of the Iterated Logarithm tells us i additio that S p < (1 ɛ) 2p(1 p) l(l()) must occur for ifiitely may. Khichi s Law of the Iterated Logarithm also applies to the cumulative fortue i a coi-tossig game, or equivaletly, the positio i a radom walk. Cosider the idepedet Beroulli radom variables Y i = +1 with probability p ad Y i = 1 with probability q = 1 p. The mea is µ = 2p 1 ad the variace is σ 2 = 4p(1 p) for each of the summads Y i. The cosider the sum T = Y Y with mea (2p 1) ad variace 4p(1 p). Sice Y = 2X 1, the T = 2S ad S = 1T 2 +. The applyig 2 the Law of the Iterated Logarithm says that with probability 1 for ay ɛ > 0, oly fiitely may of the evets: T (2p 1) > (1 + ɛ)2 2p(1 p) l(l()) occur; o the other had, with probability 1, T (2p 1) > (1 ɛ)2 2p(1 p) l(l()) occurs for ifiitely may. This meas that the fortue must (with probability 1) oscillate back ad forth across the et zero axis ifiitely ofte, crossig the upper ad lower boudaries: ±(1 ɛ)2 2p(1 p) l(l()) The statemet puts some stregth behid a uderstadig of the log-term swigs backs ad forth i value of a radom process. It also implies a form of recurrece, that is, a radom walk must visit every iteger value. The Law of the Iterated Logarithm for Beroulli trials stated here is a special case of a eve more geeral theorem first formulated by Kolmogorov i It is also possible to formulate eve stroger ad more geeral theorems! The proof here uses the Large Deviatios ad Moderate Deviatios results with the Borel-Catelli Lemmas. I aother directio, the Law of the 7
8 Iterated Logarithm ca be proved usig ivariace theorems, so it is distatly related to the Cetral Limit Theorem. Figure 1 gives a impressio of the growth of the fuctio i the Law of the Iterated Logarithm compared to the square root fuctio. Figure 2 gives a impressio of the Law of the Iterated Logarithm by showig a piecewise liearly coected graph of 2000 steps of S p with p = q = 1/2. I this figure, the radom walk must retur agai to cross the blue curves with (1 ɛ) = 0.9 ifiitely may times, but may oly cross the red curve with 1+ɛ = 1.1 fiitely may times. Of course, this is oly a schematic impressio sice a sigle radom walk (possibly atypical, from the egligible set!) o the fiite iterval ca oly suggest the almost sure ifiitely may crossigs of (1 ɛ)α(x) for ay ɛ > 0. Figure 3 gives a compariso of impressios of four of the limit theorems. The idividual figures deliberately are spaghetti graphs to give a impressio of the esemble of sample paths. Each figure shows a differet scalig of the same 15 sample paths for a sequece of 100,000 fair coi flips, each path a differet color. Note that the steps axis has a logarithmic scale, meaig that the shape of the paths is distorted although it still gives a impressio of the radom sums. The top left figure shows S / p covergig to 0 for all paths i accord with the Strog Law of Large Numbers. The top right figure plots the scalig (S p)/ 2p(1 p). For large values of steps the values over all paths is a distributio ragig from about 2 to 2, cosistet with the Cetral Limit Theorem. The lower left figure plots (S p)/ 0.6 as a illustratio of Hausdorff s Estimate with ɛ = 0.1. It appears that the scaled paths are very slowly covergig to 0, the rage for = 100,000 is withi 0.5, 0.5. The lower right figure shows (S p)/ 2p(1 p) l(l(x)) alog with lies ±1 to suggest the coclusios of the Law of the Iterated Logarithm. It suggests that all paths are usually i the rage 1, 1 but with each path makig a few excursios outside the rage. Hausdorff s Estimate Theorem 2 (Hausdorff s Estimate). Almost surely, for ay ɛ > 0, S p = o ( ɛ+1/2) as. Proof. The proof resembles the proof of the Strog Law of Large Numbers for idepedet radom variables with mea 0 ad uiformly bouded 4th 8
9 y x Figure 1: The Iterated Logarithm fuctio α(x) = 2p(1 p)x l(l(x)) i gree alog with fuctios (1 + ɛ)α(x) i red ad (1 ɛ)α(x) i blue, with ɛ = 0.1. For compariso, the square root fuctio 2p(1 p)x is i black. 9
10 y x Figure 2: A impressio of the Law of the Iterated Logarithm usig a piecewise liearly coected graph of 2000 steps of S p with p = q = 1/2 with the blue curve with (1 ɛ)α(x) ad the red curve with (1 + ɛ)α(x) for ɛ = 0.1 ad α(x) = 2p(1 p)x l(l(x)). 10
11 Figure 3: A compariso of four of the limit theorems. The idividual figures deliberately are spaghetti graphs to give a impressio of the esemble of sample paths. Each figure shows a differet scalig of the same 15 sample paths for a sequece of 100,000 fair coi flips, each path a differet color. Note that the steps axis has a logarithmic scale, meaig that the shape of the sample paths is distorted. 11
12 momets. That proof showed that usig the idepedece ad idetical distributio assumptios E S 4 = E X ( 1)E X C σ 4 C 1 2. The adaptig the proof of the Markov ad Chebyshev iequalities with fourth momets E S 4 C Use the Corollary that if E X coverges, the the sequece (X ) 1 =1 coverges almost surely to 0. By compariso, S / 0 a.s. Usig the same set of ideas E S 4 C 1 2 α 4α =1 E S 4 4 coverges so that provided that α > 3/4. The usig the same lemma S / α 0 for α > 3/4 for a simple versio of Hausdorff s Estimate. Now adapt this proof to higher momets. Let k be a fixed positive iteger. Recall the defiitio R (ω) = S (ω) p = (X k p) = (X k where X k = k=1 k=1. Expadig the product R 2k X k p ad cosider E R 2k results i a sum of products of the idividual radom variables X i of the form X i 1 X i 2 X i 2k. Each product X i 1 X i 2 X i 2k results from a selectio or mappig from idices {1, 2,..., 2k} to the set {1,..., }. Note that if a idex j {1, 2,..., } is selected oly oce so that X j appears oly oce i the product X i 1 X i 2 X i k, the E X i 1 X i 2 X i k = 0 by idepedece. Further otice that for all sets of idices E X i 1 X i 2 X i k 1. Thus E R 2k = E X i 1 X i 2 X i 2k N(k, ), 1 i 1,...,i 2k where N(k, ) is the umber of fuctios from {1,..., 2k} to {1,..., } that take each value at least twice. Let M(k) be the umber of partitios of {1,..., 2k} ito subsets each cotaiig at least two elemets. If P is such a partitio, the P has at most k elemets. The umber of fuctios N(k, ) 12
13 that are costat o each elemet of P is at most k. Thus, N(k, ) k M(k). Now let ɛ > 0 ad cosider ( ) E ɛ 1/2 2k R 2kɛ k N(k, ) 2kɛ M(k). Choose k > 1 2ɛ. The 1 ( ) E ɛ 1/2 2k R <. Recall the Corollary 2 appearig i the sectio o the Borel-Catelli Lemma: Let (X ) 0 be a sequece of radom variables. If =1 E X coverges, the X coverges to zero, almost surely. By this corollary, the sequece of radom variables ( ɛ 1/2 R ) 0 almost surely as. This meas that for each ɛ > 0, there is a egligible evet (depedig o ɛ) outside of which ɛ 1/2 R coverges to 0. Now cosider a coutable set of values of ɛ tedig to 0. Sice a coutable uio of egligible evets is egligible, the for each ɛ > 0, ɛ 1/2 R coverges to 0 almost surely. Hardy-Littlewood Estimate Theorem 3 (Hardy-Littlewood Estimate). ( ) S p = O l a.s. for. Remark. The proof shows that S p l a.s. for. Proof. The proof uses the Large Deviatios Estimate as well as the Borel- Catelli Lemma. Recall the Large Deviatios Estimate says S P p + ɛ e h +(ɛ), 13
14 where ( ) ( ) p + ɛ 1 p ɛ h + (ɛ) = (p + ɛ) l + (1 p ɛ) l. p 1 p Note that as ɛ 0, h + (ɛ) = Note that ad take ɛ = The l P S p + ɛ ad ote that ɛ2 + 2p(1 p) O(ɛ3 ). = P S p ɛ, P S p l e h + ( ) l h + = l 2p(1 p) + o ( l ). ( ) 1, ( ( sice O l ) ) 3/2 = o ( 1 ). Thus, the probability is less tha or equal to the followig: exp ( h + ( l ( Hece exp ( h + )) is coverget because of the followig iequalities ( = exp = exp 1 2p(1 p) ( l 2p(1 p) = 1 2p(1 p) exp (o (1)). )) l 1 2p(1 p), ad ) l + o (1) ) exp (o (1)) 1 1 2p(1 p) p(1 p) 1 4 2p(1 p) p(1 p) 2 1 2p(1 p) 2 1 2p(1 p) 2. 14
15 Thus, ad so 1 P S p > l <, P S p l i.o. = 1. Proof of Khichi s Law of Iterated Logarithm Theorem 4 (Khichi s Law of Iterated Logarithm). Almost surely, lim sup S p 2p(1 p) l (l ) = 1, ad lim if S p 2p(1 p) l (l ) = 1. First establish a two lemmas, ad for coveiece, let α() = 2p(1 p) l (l ). Lemma 5. For all positive a ad δ ad large eough, (l ) a2 (1+δ) < P S p > aα() < (l ) a2 (1 δ). Proof of Lemma 5. Recall that the Large Deviatios Estimate gives Note that α() P R aα() = P S p aα() S = P p aα() ( ( )) aα()a exp h +. 0 as. The ( ) aα() h + = a 2 2p(1 p) ( ) ( 2 (α() ) ) 3 α() + O, 15
16 ad so ( ) ( ) aα() α() h + = a 2 3 l (l ) + O a 2 (1 δ) l (l ) 2 for large eough. This meas that S P p aα() exp ( a 2 (1 δ) l (l ) ) = (l ) a2 (1 δ). Sice l (l ) = o ( 1/6), the results of the Moderate Deviatios Theorem apply to give S P p aα() S p(1 p) = P p a 2 l (l ) 1 exp ( a 2 l (l ) ) 2πa 2 l (l ) 1 = 2a (l ) a2. π l (l ) Sice ( ) l (l ) = o (l ) a2 δ, S P p aα() (l ) a2 (1+δ) for large eough. Lemma 6 (12.5, Kolmogorov Maximal Iequality). Suppose (Y ) 1 are idepedet radom variables. Suppose further that E Y = 0 ad Var(Y ) = σ 2. Defie T := Y Y. The P max T k b 4 1 k 3 P T b 2σ. Remark. Lemma 6 is a example of a class of lemmas called maximal iequalities. Here are two more examples of maximal iequalities. Lemma 7 (Karli ad Taylor, page 280). Let (Y ) 1 be idetical idepedetly distributed radom variables with E Y = 0 ad Var(Y ) = σ 2 <. Defie T = k=1 Y k. The ɛ 2 P max T k > ɛ σ 2. 0 k 16
17 Lemma 8 (Karli ad Taylor, page 280). Let (X ) 0 be a submartigale for which X 0. For λ > 0, λp max X k > λ E X. 0 k Proof of Lemma 6. Sice the Y k s are idepedet, the Var(T T k ) = ( k)σ 2 for 1 k. Chebyshev s Iequality tells us that P T T k 2σ 1 Var(T T k ) 4σ 2 Note that P max T k b = 0 k = 1 k P T 1 < b,..., T k 1 < b, ad T k b k=1 P T 1 < b,..., T k 1 < b, ad T k b 4 3 P T T k 2σ k=1 = 4 P T 1 < b,..., T k 1 < b, ad T k b ad T T k 2σ k=1 P T 1 < b,..., T k 1 < b, ad T k b ad T b 2σ k=1 4 3 P T b 2σ. Remark. Note that the secod part of the Law of the Iterated Logarithm, S lim if p = 1 follows by symmetry from the first part by replacig α() p with (1 p) ad S with S. Remark. The proof of the Law of the Iterated Logarithm proceeds i two parts. First it suffices to show that lim sup S p α() The secod part of the proof is to establish that lim sup S p α() < 1 + η for η > 0, a.s.. (1) > 1 η for η > 0, a.s. (2) 17
18 It will oly take a subsequece to prove (2). However this will ot be easy because it will use the secod Borel-Catelli Lemma, which requires idepedece. Remark. The followig is a simplified proof givig a partial result for iteger sequeces with expoetial growth. This simplified proof illustrates the basic ideas of the proof. Fix γ > 1 ad let k := γ k. The P S k p k (1 + η)α( k ) < (l k ) (1+η)2 (1 δ) ( ) = O k (1+η)2 (1 δ). Choose δ so that (1 + η) 2 (1 δ) < 1. The P S k k p (1 + η)α( k ) <. k 1 By the first Borel-Catelli lemma, ad so or P S k k p (1 + η)α( k ) i.o. = 0, S k k p P lim sup k α( k ) (1 + η) = 1, S k k p (1 + η) a.s. α( k ) The full proof of the Law of the Iterated Logarithm takes more work to complete. Proof of (1) i the Law of the Iterated Logarithm. Fix η > 0 ad let γ > 1 be a costat chose later. For k Z, let k = γ k. The proof cosists of showig that P max (S p) (1 + η)α( k ) <. k+1 k 1 From Lemma 6 P max (S p) (1 + η)α( k ) k+1 k 1 4 R 3 P k+1 (1 + η)α( k ) 2 k+1 p(1 p), 18
19 where R = S p. We do kow that P max (S p) (1 + η)α( k ) k P R k+1 (1 + η)α( k ) 2 k+1 p(1 p) Note that k+1 = o (α( k )) because this is approximately γ k+1 compared to c 1 γk l(l γ γ ) = c 1 γ k/2 l(l γ) + l(k),. (3) which is the same as γ 1/2 compared to c 1 c2 + l(k). The 2 k+1 p(1 p) < 1ηα( 2 k) for large eough k. Usig this iequality i the right had side of Equatio (3), we get P max S p (1 + η)α( k ) 4 k+1 3 P S k+1 k+1 p (1 + η/2)α( k ). Now, α( k+1 ) γα( k ). Choose γ so that 1 + η/2 > (1 + η/4) γ. The for large eough k, we have (1 + η/2)α( k ) > (1 + η/4)α( k+1 ). Now P max S p (1 + η)α( k ) 4 k+1 3 P S k+1 k+1 p (1 + η/4)α( k+1 ). Use Lemma 5 with a = (1 δ) 1 = (1 + η/4). The we get P max S p (1 + η)α( k ) 4 k+1 3 (l k+1) (1+η/4) for k large. Note that (l k+1 ) (1+η/4) (l γ) (1+η/4) k (1+η/4), which is the geeral term of a coverget series so P max R (1 + η)α( k ) <. k+1 k 1 19
20 The the first Borel-Catelli Lemma implies that or equivaletly The i particular max R (1 + η)α( k ) i.o. with probability 0. k+1 max S p < (1 + η)α( k ) a.s. for large eough k. k+1 max S p < (1 + η)α( k ) a.s. for large eough k. k < k+1 This i tur implies that almost surely S p < (1 + η)α(). for > k ad large eough k which establishes (1). Proof of (2) i the Law of the Iterated Logarithm. Cotiue with the proof of Equatio (2). To prove the secod part, it suffices to show that there exists k so that R k (1 η)α( k ) i.o. almost surely. Let k = γ k for γ chose later with γ Z sufficietly large. The proof will show ( P R γ R γ 1 1 η ) α(γ ) =, (4) 2 ad also 1 R γ 1 η 2 α(γ ) a.s. for large eough. (5) dist. Note that R γ R γ 1 = R γ γ 1. It suffices to cosider ( P R γ γ 1 1 η ) α(γ ). 2 Note that α(γ γ 1 ) α(γ ) = = c (γ γ 1 ) l (l (γ γ 1 )) cγ l (l γ ) ( ( )) 1 1 ) l ( l γ + l 1 1 γ γ l ( l γ) 1 1 γ. 20
21 Choose γ Z so that 1 η 2 1 η < 1 1 γ. 4 The ote that we ca choose large eough so that ( ) 1 η ( 2) 1 η < α(γ γ 1 ) α(γ ) or ( 1 η ) ( α(γ ) < 1 η ) α(γ γ 1 ). 2 4 Now cosiderig equatio (4), ad the iequality above P R γ R γ 1 ( 1 η 2 4 ) α(γ ) P R γ γ 1 ( 1 η 4 Now usig Lemma 5, with a = (1 + δ) 1 = ( 1 η 4), we get ) α(γ γ 1 ). ( P R γ R γ 1 1 η ) α(γ ) 2 l ( γ γ 1) ( (1 η 4) = ( l γ + l 1 1 )) (1 η 4). γ The series with this as its terms diverges. Thus, we see that Equatio (4) has bee prove. Now otice that ad so which meas that α(γ ) = cγ l (l γ ) = cγ (l + l l γ) α(γ 1 ) = cγ 1 (l( 1) + l l γ), γα(γ 1 ) = cγ (l( 1) + l l γ). Thus, α(γ ) γα(γ 1 ). Now choose γ so that η γ > 4. The ηα(γ ) η γα(γ 1 ) > 4α(γ 1 ) for large eough. 21
22 Thus, we have R γ 1 η 2 α(γ ) R γ 1 2α(γ 1 ) sice R γ 1 η 2 α(γ ) R γ 1 2α(γ 1 ). Now use (4) ad we see that R γ 1 < 2α(γ 1 ) happes almost surely for large eough. Now R γ R γ 1 is a sequece of idepedet radom variables, ad so the secod Borel-Catelli Lemma says that almost surely ( R γ R γ 1 > 1 η ) α(γ ) i.o. 2 Addig this with Equatio (5), we get that This is eough to show that R γ > (1 η) α(γ ) i.o. lim if S p α() > 1 η a.s., which is eough to show the oly remaiig part of Khichi s Law of the Iterated Logarithm. Sources This sectio is adapted from: W. Feller, i Itroductio to Probability Theory ad Volume I, Chapter III, ad Chapter VIII, ad also E. Lesige, Heads or Tails: A Itroductio to Limit Theorems i Probability, Chapter 12, America Mathematical Society, Studet Mathematical Library, Volume 28, Some of the ideas i the proof of Hausdorff s Estimate are adapted from J. Lamperti, Probability: A Survey of the Mathematical Theory, Secod Editio, Chapter 8. Figure 3 is a recreatio of a figure i the Wikipedia article o the Law of the Iterated Law. 22
23 Algorithms, Scripts, Simulatios Algorithm Scripts R script for compariso figures. 1 p < k < < coiflips <- array ( ruif (*k) <= p, dim =c(,k)) 5 S <- apply ( coiflips, 2, cumsum ) 6 steps <- c (1: ) 7 8 steps2 <- steps 2: 9 S2 <- S 2:, 10 steps3 <- steps 3: 11 S3 <- S 3:, oes <- cbid ( matrix (1, -2,1), matrix (-1, -2,1) ) par ( mfrow = c (2,2) ) matplot ((S- steps *p)/ steps, 18 log ="x", type ="l", lty = 1, ylab ="", mai =" Strog Law ") 19 matplot ((S- steps *p)/ sqrt (2*p*(1 -p)* steps ), 20 log ="x", type ="l", lty = 1, ylab ="", mai =" Cetral Limit Theorem ") 21 matplot ((S- steps *p)/( steps ^(0.6) ), 22 log ="x", type ="l", lty = 1, ylab ="", mai =" Hausdorff s Estimate ") 23 ## matplot ((S2 - steps2 *p)/ sqrt ( steps2 * log ( steps2 )), log ="x", xlim =c(1,), type ="l", lty = 1) 24 matplot ( steps3, (S3 - steps3 *p)/ sqrt (2*p*(1 -p)* steps3 * log ( log ( steps3 ))), 25 log ="x", xlim =c(1,), type ="l", lty = 1, ylab ="", mai =" Law of Iterated Logarithm ") 26 matlies ( steps3, oes, type ="l", col =" black ") 23
24 Problems to Work for Uderstadig 1. The multiplier of the variace 2p(1 p) fuctio l(l()) grows very slowly. To uderstad how very slowly, calculate a table with = 10 j ad 2 l(l()) for j = 10, 20, 30,..., 100. (Remember, i mathematical work above calculus, l(x) is the atural logarithm, base e, ofte writte l(x) i calculus ad below to distiguish it from the commo or base-10 logarithm. Be careful, some software ad techology caot directly calculate with magitudes this large.) 2. Cosider the sequece a = ( 1) /2 + ( 1) for = 1, 2, Here x is the floor fuctio, the greatest iteger less tha or equal to x, so 1 = 1, 3/2 = 1, 8/3 = 2, 3/2 = 2, etc. Fid lim sup a ad Does the sequece a have a limit? lim if a. 3. Show that the secod part of the Law of the Iterated Logarithm, S lim if p = 1 follows by symmetry from the first part by α() replacig p with (1 p) ad S with S. Solutios to Problems 24
25 Readig Suggestio: Refereces 1 William Feller. A Itroductio to Probability Theory ad Its Applicatios, Volume I, volume I. Joh Wiley ad Sos, third editio, QA 273 F Joh W. Lamperti. Probability: A Survey of the Mathematical Theory. Wiley Series i Probability ad Statistics. Wiley, secod editio editio, Emmauel Lesige. Heads or Tails: A Itroductio to Limit Theorems i Probability, volume 28 of Studet Mathematical Library. America Mathematical Society, Outside Readigs ad Liks: I check all the iformatio o each page for correctess ad typographical errors. Nevertheless, some errors may occur ad I would be grateful if you would alert me to such errors. I make every reasoable effort to preset curret ad accurate iformatio for public use, however I do ot guaratee the accuracy or timeliess of iformatio o this website. Your use of the iformatio from this website is strictly volutary ad at your risk. 25
26 I have checked the liks to exteral sites for usefuless. Liks to exteral websites are provided as a coveiece. I do ot edorse, cotrol, moitor, or guaratee the iformatio cotaied i ay exteral website. I do t guaratee that the liks are active at all times. Use the liks here with the same cautio as you would all iformatio o the Iteret. This website reflects the thoughts, iterests ad opiios of its author. They do ot explicitly represet official positios or policies of my employer. Iformatio o this website is subject to chage without otice. Steve Dubar s Home Page, to Steve Dubar, sdubar1 at ul dot edu Last modified: Processed from L A TEX source o March 22,
Topics in Probability Theory and Stochastic Processes Steven R. Dunbar. Stirling s Formula from the Sum of Average Differences
Steve R Dubar Departmet of Mathematics 03 Avery Hall Uiversity of Nebraska-Licol Licol, NE 68588-030 http://wwwmathuledu Voice: 40-47-373 Fax: 40-47-8466 Topics i Probability Theory ad Stochastic Processes
More informationTopics in Probability Theory and Stochastic Processes Steven R. Dunbar. Stirling s Formula Derived from the Gamma Function
Steve R. Dubar Departmet of Mathematics 23 Avery Hall Uiversity of Nebraska-Licol Licol, NE 68588-3 http://www.math.ul.edu Voice: 42-472-373 Fax: 42-472-8466 Topics i Probability Theory ad Stochastic Processes
More informationTopics in Probability Theory and Stochastic Processes Steven R. Dunbar. Evaluation of the Gaussian Density Integral
Steve R. Dubar Departmet of Mathematics 3 Avery Hall Uiversity of Nebraska-Licol Licol, NE 68588-13 http://www.math.ul.edu Voice: 4-47-3731 Fax: 4-47-8466 Topics i Probability Theory ad Stochastic Processes
More informationTopics in Probability Theory and Stochastic Processes Steven R. Dunbar. Almost Sure Events
Steve R. Dubar Departmet of Mathematics 203 Avery Hall Uiversity of Nebraska-Licol Licol, NE 68588-0130 http://www.math.ul.edu Voice: 402-472-3731 Fax: 402-472-8466 Topics i Probability Theory ad Stochastic
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationMA131 - Analysis 1. Workbook 3 Sequences II
MA3 - Aalysis Workbook 3 Sequeces II Autum 2004 Cotets 2.8 Coverget Sequeces........................ 2.9 Algebra of Limits......................... 2 2.0 Further Useful Results........................
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationSequences. Notation. Convergence of a Sequence
Sequeces A sequece is essetially just a list. Defiitio (Sequece of Real Numbers). A sequece of real umbers is a fuctio Z (, ) R for some real umber. Do t let the descriptio of the domai cofuse you; it
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More informationMath 216A Notes, Week 5
Math 6A Notes, Week 5 Scribe: Ayastassia Sebolt Disclaimer: These otes are ot early as polished (ad quite possibly ot early as correct) as a published paper. Please use them at your ow risk.. Thresholds
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationSequences I. Chapter Introduction
Chapter 2 Sequeces I 2. Itroductio A sequece is a list of umbers i a defiite order so that we kow which umber is i the first place, which umber is i the secod place ad, for ay atural umber, we kow which
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationMA131 - Analysis 1. Workbook 2 Sequences I
MA3 - Aalysis Workbook 2 Sequeces I Autum 203 Cotets 2 Sequeces I 2. Itroductio.............................. 2.2 Icreasig ad Decreasig Sequeces................ 2 2.3 Bouded Sequeces..........................
More information4.3 Growth Rates of Solutions to Recurrences
4.3. GROWTH RATES OF SOLUTIONS TO RECURRENCES 81 4.3 Growth Rates of Solutios to Recurreces 4.3.1 Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer.
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationLecture 2. The Lovász Local Lemma
Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationAssignment 5: Solutions
McGill Uiversity Departmet of Mathematics ad Statistics MATH 54 Aalysis, Fall 05 Assigmet 5: Solutios. Let y be a ubouded sequece of positive umbers satisfyig y + > y for all N. Let x be aother sequece
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationProbability for mathematicians INDEPENDENCE TAU
Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet
More informationLecture Notes for Analysis Class
Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios
More informationMath 113 Exam 3 Practice
Math Exam Practice Exam will cover.-.9. This sheet has three sectios. The first sectio will remid you about techiques ad formulas that you should kow. The secod gives a umber of practice questios for you
More informationMath 2784 (or 2794W) University of Connecticut
ORDERS OF GROWTH PAT SMITH Math 2784 (or 2794W) Uiversity of Coecticut Date: Mar. 2, 22. ORDERS OF GROWTH. Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationLaw of the sum of Bernoulli random variables
Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationFACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
More informationLesson 10: Limits and Continuity
www.scimsacademy.com Lesso 10: Limits ad Cotiuity SCIMS Academy 1 Limit of a fuctio The cocept of limit of a fuctio is cetral to all other cocepts i calculus (like cotiuity, derivative, defiite itegrals
More informationLimit superior and limit inferior c Prof. Philip Pennance 1 -Draft: April 17, 2017
Limit erior ad limit iferior c Prof. Philip Peace -Draft: April 7, 207. Defiitio. The limit erior of a sequece a is the exteded real umber defied by lim a = lim a k k Similarly, the limit iferior of a
More informationMa 530 Introduction to Power Series
Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power
More informationNotes #3 Sequences Limit Theorems Monotone and Subsequences Bolzano-WeierstraßTheorem Limsup & Liminf of Sequences Cauchy Sequences and Completeness
Notes #3 Sequeces Limit Theorems Mootoe ad Subsequeces Bolzao-WeierstraßTheorem Limsup & Limif of Sequeces Cauchy Sequeces ad Completeess This sectio of otes focuses o some of the basics of sequeces of
More information6 Infinite random sequences
Tel Aviv Uiversity, 2006 Probability theory 55 6 Ifiite radom sequeces 6a Itroductory remarks; almost certaity There are two mai reasos for eterig cotiuous probability: ifiitely high resolutio; edless
More informationCHAPTER 10 INFINITE SEQUENCES AND SERIES
CHAPTER 10 INFINITE SEQUENCES AND SERIES 10.1 Sequeces 10.2 Ifiite Series 10.3 The Itegral Tests 10.4 Compariso Tests 10.5 The Ratio ad Root Tests 10.6 Alteratig Series: Absolute ad Coditioal Covergece
More informationThe Random Walk For Dummies
The Radom Walk For Dummies Richard A Mote Abstract We look at the priciples goverig the oe-dimesioal discrete radom walk First we review five basic cocepts of probability theory The we cosider the Beroulli
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationAdvanced Analysis. Min Yan Department of Mathematics Hong Kong University of Science and Technology
Advaced Aalysis Mi Ya Departmet of Mathematics Hog Kog Uiversity of Sciece ad Techology September 3, 009 Cotets Limit ad Cotiuity 7 Limit of Sequece 8 Defiitio 8 Property 3 3 Ifiity ad Ifiitesimal 8 4
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationChapter 0. Review of set theory. 0.1 Sets
Chapter 0 Review of set theory Set theory plays a cetral role i the theory of probability. Thus, we will ope this course with a quick review of those otios of set theory which will be used repeatedly.
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationCS / MCS 401 Homework 3 grader solutions
CS / MCS 401 Homework 3 grader solutios assigmet due July 6, 016 writte by Jāis Lazovskis maximum poits: 33 Some questios from CLRS. Questios marked with a asterisk were ot graded. 1 Use the defiitio of
More informationn outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe
More informationRandomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018)
Radomized Algorithms I, Sprig 08, Departmet of Computer Sciece, Uiversity of Helsiki Homework : Solutios Discussed Jauary 5, 08). Exercise.: Cosider the followig balls-ad-bi game. We start with oe black
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationRandom Models. Tusheng Zhang. February 14, 2013
Radom Models Tusheg Zhag February 14, 013 1 Radom Walks Let me describe the model. Radom walks are used to describe the motio of a movig particle (object). Suppose that a particle (object) moves alog the
More informationIntroduction to Probability and Statistics Twelfth Edition
Itroductio to Probability ad Statistics Twelfth Editio Robert J. Beaver Barbara M. Beaver William Medehall Presetatio desiged ad writte by: Barbara M. Beaver Itroductio to Probability ad Statistics Twelfth
More informationThe natural exponential function
The atural expoetial fuctio Attila Máté Brookly College of the City Uiversity of New York December, 205 Cotets The atural expoetial fuctio for real x. Beroulli s iequality.....................................2
More informationTHE STRONG LAW OF LARGE NUMBERS FOR STATIONARY SEQUENCES
THE STRONG LAW OF LARGE NUMBERS FOR STATIONARY SEQUENCES Debdeep Pati Idia Statistical Istitute, Kolkata Jue 26, 2006 Abstract The traditioal proof of the strog law of large umbers usig idepedet ad idetically
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationM17 MAT25-21 HOMEWORK 5 SOLUTIONS
M17 MAT5-1 HOMEWORK 5 SOLUTIONS 1. To Had I Cauchy Codesatio Test. Exercise 1: Applicatio of the Cauchy Codesatio Test Use the Cauchy Codesatio Test to prove that 1 diverges. Solutio 1. Give the series
More informationLecture 1 Probability and Statistics
Wikipedia: Lecture 1 Probability ad Statistics Bejami Disraeli, British statesma ad literary figure (1804 1881): There are three kids of lies: lies, damed lies, ad statistics. popularized i US by Mark
More informationENGI Series Page 6-01
ENGI 3425 6 Series Page 6-01 6. Series Cotets: 6.01 Sequeces; geeral term, limits, covergece 6.02 Series; summatio otatio, covergece, divergece test 6.03 Stadard Series; telescopig series, geometric series,
More informationOptimally Sparse SVMs
A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but
More informationA Note on the Kolmogorov-Feller Weak Law of Large Numbers
Joural of Mathematical Research with Applicatios Mar., 015, Vol. 35, No., pp. 3 8 DOI:10.3770/j.iss:095-651.015.0.013 Http://jmre.dlut.edu.c A Note o the Kolmogorov-Feller Weak Law of Large Numbers Yachu
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationBIRKHOFF ERGODIC THEOREM
BIRKHOFF ERGODIC THEOREM Abstract. We will give a proof of the poitwise ergodic theorem, which was first proved by Birkhoff. May improvemets have bee made sice Birkhoff s orgial proof. The versio we give
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationWe are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n
Review of Power Series, Power Series Solutios A power series i x - a is a ifiite series of the form c (x a) =c +c (x a)+(x a) +... We also call this a power series cetered at a. Ex. (x+) is cetered at
More informationApplication to Random Graphs
A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let
More informationOn Random Line Segments in the Unit Square
O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,
More informationMath 155 (Lecture 3)
Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,
More informationCHAPTER I: Vector Spaces
CHAPTER I: Vector Spaces Sectio 1: Itroductio ad Examples This first chapter is largely a review of topics you probably saw i your liear algebra course. So why cover it? (1) Not everyoe remembers everythig
More information7 Sequences of real numbers
40 7 Sequeces of real umbers 7. Defiitios ad examples Defiitio 7... A sequece of real umbers is a real fuctio whose domai is the set N of atural umbers. Let s : N R be a sequece. The the values of s are
More informationLecture 12: November 13, 2018
Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,
More informationLecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More informationIntroduction to Probability. Ariel Yadin. Lecture 7
Itroductio to Probability Ariel Yadi Lecture 7 1. Idepedece Revisited 1.1. Some remiders. Let (Ω, F, P) be a probability space. Give a collectio of subsets K F, recall that the σ-algebra geerated by K,
More informationChapter 10: Power Series
Chapter : Power Series 57 Chapter Overview: Power Series The reaso series are part of a Calculus course is that there are fuctios which caot be itegrated. All power series, though, ca be itegrated because
More informationSieve Estimators: Consistency and Rates of Convergence
EECS 598: Statistical Learig Theory, Witer 2014 Topic 6 Sieve Estimators: Cosistecy ad Rates of Covergece Lecturer: Clayto Scott Scribe: Julia Katz-Samuels, Brado Oselio, Pi-Yu Che Disclaimer: These otes
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationSequences and Series
Sequeces ad Series Sequeces of real umbers. Real umber system We are familiar with atural umbers ad to some extet the ratioal umbers. While fidig roots of algebraic equatios we see that ratioal umbers
More information