Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet sequeces 3a Idepedet evets Cotiuous probability spaces are eeded here; triagle arrays do ot help. 3a1 Defiitio. (a Evets A 1,A 2,... are idepedet if for every the evets A 1,...,A are idepedet; (b radom variables X 1,X 2,... are idepedet if for every the radom variables X 1,...,X are idepedet; (c σ-algebras A 1,A 2,... are idepedet if for every the σ-algebras A 1,...,A are idepedet. The relatio P ( X 1 B 1,X 2 B 2,... = P ( X 1 B 1 P ( X2 B 2... holds for idepedet radom variables X ad Borel sets B R, but is of little use. 3a2 Exercise. Let(Ω,F,Pbe(0,1withLebesguemeasure, adβ 1,β 2, : Ω {0,1} biary digits; ω = =1 β (ω, limifβ 2 (ω = 0. The β are idepedet radom variables; also, β = 1l A, ad A are idepedet evets of probability 0.5 each ( a fair coi tossed edlessly. Treatig β as radom variables we observe that the radom variable U = 2 β is distributed uiformly o (0,1, that is, F U (u = u for 0 u 1. We itroduce radom variables U 1 = 2 β 2 1, U 2 = 2 β 2. =1 =1
Probability for mathematicias INDEPENDENCE TAU 2013 29 Foreachtheradomvector(β 1,β 3,...,β 2 1 isdistributedlike(β 1,β 2,...,β ; therefore k=1 2 k β 2k 1 is distributed like k=1 2 k β k, that is, F U1 (u = F U (u wheever u is dyadic (that is, of the form k/2 ; it follows that F U1 = F U. We see that U 1 is distributed uiformly o (0,1; the same holds for U 2. For each the radom vectors (β 1,β 3,...,β 2 1 ad (β 2,β 4,...,β 2 are idepedet (thik, why, therefore F U1,U 2 (u 1,u 2 = F U1 (u 1 F U2 (u 2 for all dyadic u 1,u 2, ad for arbitrary u 1,u 2 as well. We see that U 1,U 2 are idepedet. 1 Graph of U 1 Approximatig the curve {(U 1 (ω,u 2 (ω : 0 < ω < 1} Similarly we may itroduce U 1,U 2,... by U = 2 k β 2 1 (2k 1 k=1 ad check that these are a ifiite sequece of idepedet radom variables, each distributed uiformly o (0,1. 2 Now, give p 1,p 2, [0,1], we may cosider evets A = {U p } ad check that they are idepedet, ad P ( A = p. Let evets A 1,A 2,... be idepedet. The sum S = k=1 1l A k, the radom umber of occurred evets, ca be fiite or ifiite. 3a3 Theorem. 3 (a If k=1 P( A k < the S < almost surely; (b if k=1 P( A k = the S = almost surely. These (a ad (b are called Borel-Catelli lemmas. Idepedece matters for (b but ot (a. For idepedet evets, P ( S < is either 0 or 1, which is a special case of Kolmogorov s 0 1 law. 3a4 Exercise. Let U 1,U 2,... be idepedet radom variables, each distributed uiformly o ( 1, 1. The 1 Istead of dyadic umbers ad CDF we could use dyadic algebra; it geerates the Borel σ-algebra. 2 [W, Sect. 4.6]. 3 [KS, Sect. 7.1, Lemmas 7.3, 7.4]; [D, Sect. 1.6, (6.1 ad (6.6].
Probability for mathematicias INDEPENDENCE TAU 2013 30 (a the sequece (U =1 is dese i R a.s.; (b the sequece ( 2 U =1 is ot dese, ad moreover, 2 U a.s. If A k are idepedet, P ( A k 0 but k P( A k =, the the idicators X k = 1l Ak coverge to 0 i L 2 (Ω but ot almost surely; moreover, limsup k X k (ω = 1 for almost all ω Ω. (There is a simpler, oprobabilistic example o Ω = (0, 1. S = Proof of 3a3(a (the first Borel-Catelli lemma 1l Ak ; ES = p 1 + +p, p k = P ( A k ; k=1 P ( S > M P ( S > M as ; P ( S > M ES M = p 1 + +p M P ( S > M 1 M k (wrog for! 1 P ( A k ; M P ( A k 0 as M. Aother proof: the sequece S is icreasig ad ES is bouded, therefore S S < a.s. Ed of proof of 3a3(a (the first Borel-Catelli lemma k Proof of 3a3(b (the secod Borel-Catelli lemma (Clearly ES =, but we eed much more... P ( S M = P ( e S e M Ee S = em e M ( e M exp k=1 ( pk e 1 +(1 p k 1 }{{} 1 (1 e 1 p k k=1 p k (1 e 1 0 as ; (sice 1 ε e ε P ( S M = 0 for all M. Aother proof: Ee S 0 (as before; Ee S Ee S ; Ee S = 0. Ed of proof of 3a3(b (the secod Borel-Catelli lemma
Probability for mathematicias INDEPENDENCE TAU 2013 31 Let A k be equiprobable, of probability p each ( ufair coi. 3a5 Propositio. 1 ( 1lA1 + +1l A p (as almost surely. This is a special case of the Strog Law of Large Numbers (see 3b2, but also of the followig fact (less geeral ad much simpler to prove. 3a6 Propositio. (Borel s strog law of large umbers Let X be idepedet, idetically distributed radom variables such that EX1 4 <, the 1 (X 1 + +X EX 1 almost surely. 3a7 Exercise. (a It is sufficiet to prove 3a6 for EX 1 = 0. Let X be as i 3a6. (b If EX 1 = 0 the E(X 1 + +X 4 3 2( EX 2 1 2. Proof of 3a6. Assumig EX 1 = 0 ad deotig S = X 1 + +X we have E ( S 4 = O( 1 ; 2 E( S 4 < ; ( S 4 < a.s.; ( S 4 0 a.s.; S 0 a.s. 3a8 Exercise. For S as i 1a5 (the simple radom walk, S = o( a.s. Compare it with 1a5. Covergece i L 2 is rather evidet, but almost everywhere covergece is ot. Arealumberx (0,1iscalled10-ormal,ifitsdecimaldigitsα 1,α 2,... defied by x = α 1 10 + α 2 10 2 +...; α 1,α 2, {0,1,2,3,4,5,6,7,8,9} have equal frequecies, that is, #{k [1,] : α k = a} 1 10 as (for all a ad moreover, their combiatios have equal frequecies, that is, #{k [1,] : α k = a 1,α k+1 = a 2,...,α k+l 1 = a l } 1 10 l as for all a 1,...,a l ad all l. Similarly, p-ormal umbers are defied for ay p = 2,3,... Fially, x is called ormal, if it is p-ormal for all p.
Probability for mathematicias INDEPENDENCE TAU 2013 32 3a9 Propositio. Normal umbers exist. Propositio 3a9 follows from Propositio 3a10. 3a10 Propositio. 1 Almost all umbers are ormal. That is, the set of all ormal umbers is Lebesgue measurable, ad its Lebesgue measure is equal to 1. This is Borel s ormal umber theorem (1909. Proof. It suffices to treat a sigle base, for istace 10, ad a sigle combiatio of digits, for istace 71 : #{k [1,] : α k = 7,α k+1 = 1} 1 100 as. (? Splittig these k ito eve ad odd umbers we ote that it suffices to treat the two cases separately; for istace, the odd case: or equivaletly, #{k : 2k 1,α 2k 1 = 7,α 2k = 1} #{k : 2k 1,α 2k 1 = 7,α 2k = 1} #{k : 2k 1 } which is a special case of 3a3. 1 200, (? 1 100 as, Do ot thik that the ormality exhausts probabilistic properties of (digits of real umbers. 3a11 Propositio. The series =1 2β 1 coverges for almost all x (0,1. (Here β 1,β 2,... are the biary digits of x. By the way, the sum of the series, f(x = ( 1 β, is a terrible (but measurable fuctio. Especially, mes{x (a,b : f(x (c,d} > 0 for all itervals (a,b (0,1, (c,d R (here mes stads for the Lebesgue measure. Surely we caot draw its graph! Here is a probabilistic couterpart of 3a11. 1 [D, Sect. 6.2, Example 2.5].
Probability for mathematicias INDEPENDENCE TAU 2013 33 3a12 Propositio. The series X 1 1 + X 2 2 + X 3 3 +... coverges almost surely. (Here X 1,X 2,... are idepedet radom sigs. Covergece i L 2 is rather evidet, but almost everywhere covergece is ot. See also 3b3 ad 3b8. Propositios 3a11 ad 3a12 will be proved i Sectio 3b. 3a13 Propositio. A measurable fuctio f : R R satisfyig f(x + 2 = f(x for all x R ad = 1,2,... is costat almost everywhere. I other words: there exists a R such that f(x = a for almost all x. (It eed ot hold for all x. This is a aalytical couterpart of the followig probabilistic fact. 3a14 Propositio. Let X 1,X 2,... be idepedet radom sigs, ad a radom variable Y be of the form Y = f (X,X +1,... for all. The Y is costat a.s. This is a special case of Kolmogorov s 0 1 law (see 3b7. 3b Idepedet radom variables Let F ad F be as i 1c2. 3b1 Theorem. 1 sup F (x F(x 0 amost surely, as. x R This is the (strog form of Gliveko-Catelli theorem. Proof. By 1c3, for every ε > 0 there exist m ad t 1 < < t m such that µ ( (,t 1 ε, µ ( (t 1,t 2 ε,..., µ ( (t m 1,t m ε, µ ( (t m,+ ε. Similarly to the proof of 1c2, if µ ( (,tk ] µ ( (,t k ] ε ad µ ( (,tk µ ( (,t k ε for all k the sup t F (t F(t 3ε. By 3a5, this happes evetually (almost surely, for every ε > 0 separately. Therefore, almost surely it holds for all ε > 0 simultaeously. 1 [D, Sect. 1.7, (7.4].
Probability for mathematicias INDEPENDENCE TAU 2013 34 Strog law of large umbers 3b2 Theorem. 1 Let X 1,X 2,... be idepedet idetically distributed radom variables. If E X 1 < the X 1 + +X EX 1 a.s. as. This is the Strog Law of Large Numbers. Compare it with 1c1, 3a5 ad 3a6. It appears that 3b2 is much harder to prove. 3b3 Propositio. 2 (Kolmogorov Suppose X 1,X 2,... are idepedet radom variables with EX = 0. If Var(X < the the series X coverges almost surely. Postpoig the proof of 3b3 we first show that it implies 3b2. Before treatig radom series, recall covergece of series a of real umbers (a R, ad do ot cofuse it with covergece of positive series (a > 0; do ot write a < istead of a coverges, ad ote that a ca coverge while b diverge eve if a /b 1. 3b4 Lemma. (Kroecker If x R are such that x coverges the x 1 + +x 0. Proof. (sketch I terms of y = x, x = y, it takes the form if y coverges the 1 y 1 + 2 y 2 + + y 0. I terms of S = y 1 + +y, y = S S 1, it takes the form which is easy to check. if S S the S 1 (S 1 + +S 1 0, Proof of 3b2 (strog law of large umbers assumig 3b3 (to be proved later By the first Borel-Catelly lemma 3a3(a, X evetually, sice =1 P( X > = =1 P( X 1 > E X 1 <. 1 [D, Sect. 1.7, Items (7.1 ad (8.6]; [KS, Sect. 7.2, Th. 7.7]. 2 [D, Sect. 1.8, Th. (8.3].
Probability for mathematicias INDEPENDENCE TAU 2013 35 WeitroduceY = X 1l [,] (X adotethat Y 1+ +Y X 1+ +X 0 almost surely, sice X Y 0 almost surely. Thus it is sufficiet to prove that Y 1+ +Y EX 1 a.s. We itroduce Z = Y EY ad ote that EY = E ( X 1 1l [,] (X 1 EX 1, therefore Y 1+ +Y Z 1+ +Z = EY 1+ +EY EX 1. Thus it is sufficiet to prove that Z 1+ +Z 0 a.s. By 3b4, it is sufficiet to prove that Z coverges almost surely. By 3b3, it is sufficiet to prove that Var ( Z <. We have VarZ = VarY EY; 2 it remais to prove that 1 EY 2 2 <. I fact, 1 =1 EY 2 2 2E X 1, sice y y 2 k=1 1l k 2 [ k,k] (y 2 y. Ideed, for y ( 1,] we have k= 1 k 2y2 2 y = 1 + 2 k=+1 1 k 2 1 2 + k= 1 k 2 2 2 y = dx x 2 = 1 2 + 1 2. Ed of proof of 3b2 assumig 3b3 Kolmogorov s maximal iequality The followig result is eeded for 3b3. 3b5 Propositio. LetX 1,...,X beidepedet radomvariables, EX k = 0 ad EXk 2 < for k = 1,...,. The, for every c > 0, (Here S = X 1 + +X. ( P max S k c k=1,..., ES2 c 2. 3b6 Remark. Evidetly, P ( S k c ES2 k ES2 c 2, thus, max c 2 k P ( S k c ES2. However, Kolmogorov s result is much stroger! Also, evidetly, c 2 P ( max k S k c k P( S k c 1 c k 2 ES2 k, but it does ot help: the latter may grow as (try X 2 = X 3 = = 0.
Probability for mathematicias INDEPENDENCE TAU 2013 36 Here is the first proof, for the discrete case; it shows the idea 1 used afterwards i the secod, geeral proof. (We do it for the quadratic fuctio, but the proofs work for every covex fuctio. E ( S X1,...,X k = Sk, thus E ( S 2 X1,...,X k S 2 k ; (by coditioal Jese, or just coditioal EX 2 (EX 2 0 itroduce disjoit evets A k = { S 1 < c,..., S k 1 < c, S k c}; E ( S 2 Ak c 2 ; E ( S 2 1l A k c 2 P ( A k ; E ( S 2 1l A 1 A c 2 P ( A 1 A ; Here is the secod (fial proof. ES 2 c2 P ( max S k c. k Proof of 3b5 We itroduce disjoit evets A k as before ad prove that E ( S 2 1l A k c 2 P ( A k as follows. We have where E ( S 2 1l Ak = B k R k (x 1 + +x 2 µ 1 (dx 1...µ (dx, B k = {(x 1,...,x k : x 1 < c,..., x 1 + +x k 1 < c, x 1 + +x k c}; we rewrite the itegral as µ 1 (dx 1...µ k (dx k (x 1 + +x 2 µ k+1 (dx k+1...µ (dx ; B k R k takig ito accout that (for every a (a+x k+1 + +x 2 µ R }{{} k+1(dx k+1...µ (dx a 2 k =a 2 +2a(x k+1 + +x +(x k+1 + +x 2 we get B k µ 1 (dx 1...µ k (dx k (x 1 + +x k 2 }{{} c 2 ( c 2 P Ak. Ed of proof of 3b5 1 This idea, stoppig, will be the teor of Part 2 of the course.
Probability for mathematicias INDEPENDENCE TAU 2013 37 Proof of 3b3 We ll prove that the partial sums S are a Cauchy sequece a.s., that is, lim sup S k S l = 0 a.s. k,l These suprema, beig a decreasig (i sequece, coverge a.s.; i order to prove that their limit vaishes a.s. it is sufficiet to prove that We have, usig 3b5, ε > 0 P ( sup S k S l > 2ε 0. k,l ( ( P sup S k S l > 2ε P sup S k S > ε = k,l k = limp ( max S k S > ε 1 m k=,...,+m ε }{{} 2 1 ε 2E(X2 +1 + +X2 +m Ed of proof of 3b3 k=+1 VarX k 0. The proof of 3b2 (strog law of large umbers is ow complete. Zero-oe law 3b7 Propositio. Let X 1,X 2,... be idepedet radom variables, ad a radom variable Y be of the form Y = f (X,X +1,... for all. The Y is costat a.s. This is a form of Kolmogorov s 0 1 law. (See also 3a14. Basically, it holds because every measurable fuctio of X 1,X 2,... is approximately a measurable fuctio of X 1,...,X (see 3b14. 3b8 Exercise. Let X 1,X 2,... be idepedet radom variables, ad S = X 1 + +X. The the followig evets are of probability 0 or 1 each: S coverge; S are bouded; S are bouded from above; S are bouded from below. Deduce it from 3b7.
Probability for mathematicias INDEPENDENCE TAU 2013 38 See also 3a11, 3a12. Recall the σ-algebras geerated by radom variables: σ(x, σ(x,y etc.; σ(x,y cosists of sets of the form {ω : (X(ω,Y(ω B} for Borel B R 2. Rewritig (X(ω,Y(ω B as 1l B (X(ω,Y(ω = 1 we see that a σ(x,y-measurable idicator fuctio is of the form ϕ(x,y where ϕ is a Borel measurable idicator fuctio o R 2. It follows (but ot immediately that the same holds for R-valued (rather tha {0, 1}-valued fuctios (the Doob-Dyki lemma; this is why σ(x, Y-measurable fuctios are ofte called measurable fuctios of X,Y. Similarly, σ(x 1,X 2,...-measurable fuctios are ofte called measurable fuctios of X 1,X 2,... Here σ(x 1,X 2,... is the least σ-algebra makig all X k measurable. Deotig F = σ(x,x +1,... we have F (the tail σ-algebra = F. Measurability w.r.t. the tail σ-algebra is measurability w.r.t F It holds for Y of 3b7 ad 3a14. for every. 3b9 Propositio (Kolmogorov s 0-1 law. 1 If X are idepedet the the tail σ-algebra is trivial. Deotig F 1 = σ(x 1,...,X we have F 1 σ(x 1,X 2,... = F 1 i the sesethatf 1 istheleastσ-algebrathatcotaisallf 1. Thatis, F 1 = σ(e where E = F 1. 3b10 Exercise. (a E is a algebra; (b E eed ot be a σ-algebra. Hit: (b try biary digits. (3b11 By 1b6, E is dese i σ(e, that is, if P(A E = 0 for all A σ(e E E wheever E is a algebra (ot just σ(x 1,...,X. 3b12 Exercise. If a σ-algebra is idepedet of (all evets of a algebra E the it is idepedet of σ(e. Proof of Kolmogorov s 0-1 law. Idepedece of F1 ad F +1 implies idepedece of F1 ad the tail σ-algebra for every. By 3b12 the tail σ-algebra is idepedet of F1, therefore, of itself! 1 [D, Sect. 1.8, (8.1]; [W, Th. 4.11].
Probability for mathematicias INDEPENDENCE TAU 2013 39 3a13, 3a14 ad 3b7 follow. Here is aother useful cosequece of (3b11. 3b13 Exercise. Let F 1 F 2 F be sub-σ-algebras, ad F = σ(f 1,F 2,... The L 2 (F is the closure of L 2 (F. Hit: for a idicator fuctio i L 2 (F use (3b11; their liear combiatios approximate every bouded fuctio. (3b14 I particular, L 2 ( σ(x1,x 2,... is the closure of L 2 ( σ(x1,...,x wheever X 1,X 2,... are radom variables (ot just idepedet. Some more applicatios of zero-oe law (ad CLT. 3b15 Exercise. For the simple radom walk (S, (a sup S = a.s.; (b limif S = ad limsup S = a.s.; (c sup{ : S = 0} = a.s. Hit: (a max k (S k+ S k = ; (b use (a ad 3b8; (c use (b. 3b16 Exercise. For the simple radom walk (S, lim if S = ad limsup Hit: sup k S 2 k+1 S 2 k 2 k/2 = (usig 2a1. S = a.s.