Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A} F for all A S Oe ca write {X A} or (X A) as a shorthad for {ω : X(ω) A} = X 1 (A). If (S, S) = (R d, R d ), the X is called a d-dimesioal radom vector. Here, R is the Borel σ field or the σ-field geerated by the ope subsets of R, ad R d is the d-fold product σ-algebra of R with itself, which will be defied shortly. Perhaps the simplest example of a measurable fuctio is a idicator fuctio of a measurable set. The idicator fuctio of a set F F is defied as 1 F (ω) = { 1 if ω F 0 if ω / F 3.2 Geeratio of σ-field Let A be a collectio of subsets of Ω. The σ-field geerated by A, deoted by σ(a), is the smallest σ-field o Ω which cotais A, which is the itersectio of all σ-fields cotaiig A. Let {X i } i I be a family of mappigs of Ω ito measurable spaces (S i, S i ), i I. Here, I is a arbitrary (possibly ucoutable) idex set. The σ-field geerated by {X i } i I, deoted by σ({x i } i I ), is the smallest σ field o Ω with respect to which each X i is measurable. Takig A = i I {X 1 i (S) : S S i }, we ca see that this case reduces to the previous oe. We ow itroduce product spaces ad product σ fields. Give measurable spaces (S i, S i ) ad idex set I, defie the product sample space as Ω = i S i = {(ω i, i I) : ω i S i }, the Cartesia product of the S i. Give S i S i, let S i be the set of poits i ω Ω for which the the i th coordiate lies i S i, that is, S i = {ω Ω : ω i S i }. 3-1
Lecture 3: Radom variables ad their distributios 3-2 Let A = { i F S i : F S is fiite}, that is, A is the set of fiite itersectios of S i. The product σ-field is defied to be σ(a). 3.3 Checkig measurability Theorem 3.3.1 Let (Ω, F) be a measurable space ad X : Ω S. If S has the σ- field σ(a) for a arbitrary collectio of sets A, the X is measurable iff (X A) F for A A. Proof: We first prove the reverse directio. Sice {X A} = {ω : X(ω) A} = X 1 (A), we have Thus, X 1 (σ(a)) = σ(x 1 (A)). X 1 (A c ) = (X 1 (A)) c ( ) X 1 A i = X 1 (A i ) i i ( ) X 1 A i = X 1 (A i ) i i To prove the forward directio, ote that the collectio C of subsets of S give by C = {B S : X 1 (B) F} is a σ-field which cotais A ad hece σ(a) which is the σ-field geerated by A. Similarly, if S has the σ-field σ(y i, i I), X is measurable iff each Y i X is measurable. Fact: The compositio of two measurable maps is measurable. 3.4 Real ad exteded real radom variables Let S be a topological space. The Borel σ-field o S, deoted by B(S), is the σ-field geerated by ope subsets of S. If f : S T is a cotiuous fuctio, the f is measurable from (S, B(S)) to (T, B(T )) by the previous theorem. If (S, S) = (R, R), the some possible choices of A are {(, x] : x R} or {(, x) : x Q} where Q = the ratioals. For the real lie R = (, ) ad exteded real lie R = [, ], the Borel σ-fields ca be defied as follows. B(R) = σ{(, x], x R} B( R) = σ{[, x], x R}
Lecture 3: Radom variables ad their distributios 3-3 Defiitio 3.4.1 (Real Radom Variable) Let (Ω, F) be a measurable space. A real radom variable (r.r.v.) is a measurable map from Ω to R. Thus a fuctio X with rage R is a r.v. iff (X x) F for all x R (by theorem 2.1). Similarly, exteded real radom variables (e.r.r.v.) ca be defied o rage R. Operatios o real umbers are performed poitwise o real-valued fuctios, e.g., Z = X + Y meas Z(ω) = X(ω) + Y (ω) for all ω Ω ad Z = lim Z meas Z(ω) = lim Z (ω) for all ω Ω Notatio for real umbers: x y = max(x, y), x y = mi(x, y), x + = x 0, x = (x 0). Note that x = x + + x ad x = x + x. Theorem 3.4.2 If X 1, X 2,... are e.r.r.v. s o (Ω, F), the they are closed uder all limitig operatios, i.e., are also e.r.r.v. if X, sup X, lim if X, lim sup X Proof: Sice the ifimum of a sequece is < a iff some term is < a, we have { } if X < a = {X < a} F The proof for supremum follows similarly. For limit iferior of X, we have lim if X := sup{ if X m} m Now ote that Y = if m X m is a e.r.r.v. for each ad so sup Y is also a e.r.r.v. The proof for limit superior follows similarly. From the above proof we see that Ω 0 { } ω : lim X exists = { } ω : lim sup X lim if X = 0 is a measurable set. If X (ω) coverges for almost all ω, i.e., P(Ω 0 ) = 1, we say that X coverges almost surely to a limit X which is defied o Ω 0. X ca be defied arbitrarily o Ω \ Ω 0, with differet authors preferrig differet covetios.
Defiitio 3.4.3 (Simple Radom Variable) X is a simple radom variable iff X is a fiite liear combiatio of idicators, i.e., X ca be expressed as X(ω) = i=1 c i1 Ai (ω) where c i R ad A i F. A simple r.v. ca oly take fiitely may values. Theorem 3.4.4 Every real r.v. X is a poitwise limit of a sequece of simple r.v. s, which ca be take to be icreasig if X 0. Proof: For X 0 let, X = { k 1 2 o { k 1 2 X < k 2 }, 0 k 2 o {X } The X X. For geeral X use the decompositio X = X + X. Corollary 3.4.5 Let X ad Y be real valued r.v. s. The so are XY, X +Y, X Y, mi(x, Y ), max(x, Y ). Proof: Cosider X X ad Y Y. This implies X Y XY. Similarly, use the previous theorem to pass from simple case to the more geeral cases. 3-4
Lecture 4: Expected Value 4-1 Lecture 4 : Expected Value Refereces: Durrett [Sectio 1.3] 4.5 Expected Value Deote by (Ω, F, P) a probability space. Defiitio 4.5.1 Let X : Ω R be a F\B-measurable radom variable. The expected value of X is defied by E(X) := XdP = X(ω)P(dω) (4.1) Ω The itegral is defied as i Lebesgue itegratio, wheever X dp <. Ω Theorem 4.5.2 (Existece of the itegral for oegative e.r.r.v.) Let (Ω, F, P) be a probability space. There is a uique fuctioal E : X E(X) [0, ] such that Ω E(1 A ) = P(A), A F (4.2) E(cX) = ce(x), c 0, X 0 (4.3) E(X + Y ) = E(X) + E(Y ), X, Y 0 (4.4) X Y E(X) E(Y ) (4.5) X X E(X ) E(X) (4.6) Proof Sketch: From these desired properties, we see immediately how to defie E(X). The procedure is well kow from Lebesgue itegratio. First exted E from idicators to simple r.v. s by liearity, the to positive r.v. s by cotiuity from below, ad fially check that everythig is cosistet. Step 1: Simple radom variables Check that if X = i=1 c i1 Ai is a simple radom variable, the E(X) = c i P(A i ) (4.7) i=1 works. Verify that E is well defied, etc.
Step 2: Noegative radom variables Now use (4.6) to exted E for geeral X 0. We kow that there exists a icreasig sequece X of simple r.v. with X X. Now see that E(X ) (by mootoicity of E). Defie Verify agai that E(X) is well defied. E(X) = lim E(X ) (4.8) Remark 4.5.3 Note that E(X) = + is possible eve if P(X < ) = 1. As a example look at G which is a geometric r.v., i.e. P(G = g) = 2 g, g = 1, 2, 3,... Note that P(G < ) = 1, but E(2 G ) = i=1 2g 2 g =. Step 3: Siged radom variables Write X as X = X + X, where X + := max(x, 0) ad X := mi(x, 0). Defie E(X) as follows E(X) = E(X + ) E(X ) (4.9) provided this expressio is ot. Such X are quasi-itegrable. X is itegrable if E( X ) <. 4.6 Itegratio ad Limit Theorem 4.6.1 (Fatou s Lemma) If X 0 the lim if EX E(lim if X ). Example 4.6.2 Defie X o [0, 1] as X = 1 (0,1/). lim E(X ) = lim 1 = 1 0 = E(0) = E ( ) lim X (4.10) Theorem 4.6.3 (Mootoe Covergece Theorem) If 0 X X the E(X ) E(X). Theorem 4.6.4 (Domiated Covergece Theorem) If X X a.s., X Y for all, ad E(Y ) <, the E(X ) E(X). Remark 4.6.5 Fatou s Lemma is usually applied whe lim X exists, so the lim if o the left is a lim. Remember example 4.6.2 to get the iequality the right way. Remark 4.6.6 The simplest boud Y i the domiated covergece theorem is a costat. (This works because we are i a fiite measure space!) 4-2
Lecture 5: Iequalities 5-1 Lecture 5 : Iequalities 5.7 Iequalities Let X, Y etc. be real r.v. s defied o (Ω, F, P). Theorem 5.7.1 (Jese s Iequality) Let ϕ be covex, E( X ) <, E( ϕ(x) ) <. The ϕ(e(x)) E(ϕ(X)) (5.11) Proof Sketch: As ϕ is covex, ϕ is the supremum of a coutable collectio of lies. ϕ(x) = sup L (x), L (EX) (1) = E(L (X)) (2) E(ϕ(X)) L (x) = a x + b Take sup o. (1) used liearity, (2) used mootoicity. Keep the followig example i mid to remember the directio of the iequality. Example 5.7.2 EX 2 (EX) 2. (5.12) I other words, if we defie Var(X) = E(X E(X)) 2 the we get Var(X) 0. Example 5.7.3 Let Ω = { 1, 1} ad S (x) = i=1 x i the we ca calculate ( 1 ( E(S (x)) = E 21 2 S 1(x) + 2 1 2 )) S 1(x) = 5 2 4 E(2S 1(x) ). By iductio we see that E(2 S ) = ( 5 4 ) ad ( 5 4 ) = E(2 S ) 2 E(S ) = 2 0 = 1. Other oteworthy facts ca be derived as corollaries of Jese s Iequality.
Lecture 5: Iequalities 5-2 Example 5.7.4 X p as p (5.13) E(X) E( X ) (5.14) Theorem 5.7.5 (Markov s Iequality) If X 0, a > 0, the P(X a) E(X)/a (5.15) Proof: Itegrate 1 X a X/a. The stated result follows by mootoicity ad liearity. Theorem 5.7.6 (Chebyshev s Iequality) Let ψ : R + R + be icreasig. The P( Y > b) E(ψ( Y ))/ψ(b) (5.16) Proof: P( Y > b) (1) = P(ψ( Y ) > ψ(b)) (2) E(ψ( Y ))/ψ(b) (1) used that ψ is icreasig, ad (2) used Markov s iequality. Example 5.7.7 Note importat examples ψ(x) = x p, exp(x), etc. ψ(x) = x 2 = P( Y > b) E(Y 2 )/b 2 (5.17) X = Y E(Y ) = P( Y E(Y ) > b) E ( (Y E(Y )) 2) /b 2 (5.18) Cosider Ω = { 1, 1} N ad S N (x) = N 1 x i. The with ψ(y) = 2 y we get P(S N (x) > 1 2 N) = P(2S N (x) > 2 N/2 ) E(2S N (x) ) 2 N/2 5N 4 N 2 N/2.9N. With a little more work we ca show that for every λ > 0 there exists α > 1 ad β < 1 such that usig Chebychev s iequality with ψ(y) = α y we get P(S N (x) > λn) = P(alpha S N (x) > alpha λn ) β N. O the first day of the quarter we did this same calculatio with ψ(y) = y 2. We calculated that E(SN 2 ) = N ad that P( S N > λn) N (λn) 2 = 1 λ 2 N.
Lecture 5: Iequalities 5-3 Does there exist a radom variable Y ad b > 0 such that Chebychev s iequality is a equality, P( Y b) = E(Y 2 ) b 2? Yes. We ca choose b = 1 ad P(Y = 1) = 1. More geerally we could set P(Y = 1) = 1 p ad P(Y = 1) = p. Does there exist Y such that lim sup b b 2 P( Y > b) E(Y 2 ) > 0? Fix Y ad ɛ > 0. Pick N such that E(Y 1 Y >N ) < ɛ ad b > N. The P( Y > b) = P( Y 1 Y >b > b) ad E ( Y 1 Y >b ) < E ( Y 1 Y >N ) < ɛ. By Chebychev s iequality Thus b 2 P( Y 1 Y >b > b) E ( Y 2 (1 Y >b ) 2) (5.19) E ( Y 2 1 Y >b ) (5.20) ɛ (5.21) b 2 P( Y > b) E(Y ) = b 2 P( Y > b) (5.22) ɛ (5.23)