MATH 418: Lectures on Conditional Expectation

Transcription

1 MATH 418: Lectures on Conditional Expectation Instructor: r. Ed Perkins, Notes taken by Adrian She Conditional expectation is one of the most useful tools of probability. The Radon-Nikodym theorem enables us to construct conditional expectations. efinition 1. A measure µ on (Ω, F) is σ-finite iff there is A n F satisfying A n Ω where µ(a n ) < for all n. efinition 2. Let µ, ν be measures on (Ω, F). Then ν is absolutely continuous with respect to µ (ν << µ) iff for all A F, µ(a) = 0 means ν(a) = 0. For instance, let (Ω, F, P ) be a probability space and let X : Ω [0, ) be a random variable. Let ν(a) = X dp for all A F. Then ν is a σ-finite measure which is absolutely A continuous (ν << P ). Proof. Let A n = {X n n}, which increases to Ω. Then ν(a n ) n dp = n < implies that ν is σ-finite. Now let A satisfy P (A) = 0. Then X1 A = 0 a.s. implies that X1 A dp = 0 dp = 0 = ν(a). Hence ν << P. The Radon-Nikodym theorem says that the converse of the above phenomenon is true. Theorem 4.1 (Radon-Nikodym). Let ν, µ be σ-finite measures on a measure space (Ω, F) such that ν << µ. Then there is a random variable X : Ω [0, ) such that, for all F, ν() = X dµ. dν Write X = dµ. If X is another random variable satisfying the previous condition, then X = X almost surely. Proof. Refer to urrett, Chapters A.4.5 and A.4.6 Radon-Nikodym gives us existence of conditional expectations. Write σ F if is a sub σ-field of F. Theorem 4.2. Suppose σ F and X is integrable on (Ω, F). Then there is a unique, up to P -null sets, random variable E(X )(ω) which is i) -measurable and ii) for all, we have: E(X ) dp = X dp Call E(X ) the conditional expectation of X given. Proof. Case 1: Firstly, suppose X is a non-negative random variable. efine ν on sub σ-algebra by ν() = X dp. We have ν << P where P is the measure on the original probability space restricted to. Hence, Radon Nikodym says that there is a random variable E(X ) > 0 such that for all sets, we have X dp = E(X ) dp. Case 2: In the general case, suppose X = X + X. Apply Case 1 to each of X +, X and define E(X ) = E(X + ) E(X ). Then the resulting random variable is -measurable 1

2 since each x y is a measurable function whenever x, y are, and furthermore ii) holds by the linearity of the integral. Uniqueness: Suppose Ẽ(X ) = E(X ) is a random variable satisfying conditions i/ii. We need to show that they are equal almost surely. Consider the set n = {ω : (Ẽ(X ) E(X )(ω) 1 n }, which is -measurable. Then, P ( n ) (Ẽ(X ) E(X ))n dp = 0 n since each of Ẽ(X ) and E(X ) integrate like X over -measurable sets. Hence, n {Ẽ(X ) > E(X )} means that P (Ẽ(X ) > E(X )) = lim P ( n) = 0. Hence, Ẽ(X ) E(X ) almost surely, and by symmetry, the reverse inequality holds a.s.. Hence, Ẽ(X ) = E(X ), almost surely. If Y is a random vector, define E(X Y ) = E(X σ(y )). Furthermore, if B is an event with P (B) > 0, then E(X B) = 1 P (B) X dp R. B The following are some examples of conditional expectation, with an intention of relating the abstract definition of conditional expectation to more concrete examples. Let Ω = B 1... B N where each B i F satisfy P (B i ) > 0. Let = σ({b 1,..., B N }). An element of the σ-field is a finite disjoint union of the {B i } Claim: E(X )(ω) = N i=1 E(X B i)1 Bi (ω). Proof. Let Z(ω) = N i=1 E(X B i)1 Bi (ω). Since B i, then Z is a -measurable function. It remains to verify property (ii) of conditional expectation. To begin, take one of the B j s. Then Z dp = E(X B j )P (B j ) = E(X B j )dp = P (B j) X dp = X dp B j B j P (B j ) B j B j by definition of conditioning of an event of positive probability. Now recall that the elements of are finite disjoint unions of the B j. Let = m k=1 B i k. Then by linearity and the previous observation: m m Z dp = Z dp = X dp = B ik B ik k=1 k=1 X dp By uniqueness of conditional expectations, then E(X ) = Z almost surely. Suppose X is a -measurable function. Then here, we know everything, and then E(X )(ω) = X(ω) almost surely. Property i is verified by assumption, and ii follows trivially. Suppose random variables and X are independent (which occurs if sigma fields σ(), σ(x) are.) Here we know nothing. Claim: E(X ) = E(X) almost surely. 2

3 Proof. For i, constants are -measurable. Now suppose σ(). Then by independence: X dp = 1 X dp = 1 dp X dp = P ()E(X) = E(X) dp Hence, E(X ) = E(X) satisfies the two defining properties of conditional expectation. Similarly, suppose = {, Ω}, that is is the trivial σ-field. Then again, E(X ) = E(X) since we have no information. Conditional expectation has many of the properties of expectation. Proposition 4.3. Let X 1, X 2 be integrable random variables and σ E σ F be sub σ-fields. Then a For all a 1, a 2 R: E(a 1 X 1 + a 2 X 2 ) = a 1 E(X 1 ) + a 2 E(X 2 ) almost surely. b If X 1 X 2 almost surely, then E(X 1 ) E(X 2 ) almost surely. c E(E(X 1 E) ) = E(X 1 ) almost surely. Proof. 1. Suppose Z = a 1 E(X 1 ) + a 2 E(X 2 ). Then Z is -measurable by linearity. Suppose. By linearity of the integral and since E(X 1 ), E(X 2 ) are conditional expectations, then: Z dp = a 1 E(X 1 ) dp +a 2 E(X 2 ) dp = a 1 means that Z = E(a 1 X 1 + a 2 X 2 ) almost surely. X 1 dp +a 2 X 2 dp = 2. Since X 2 X 1 0 almost surely, then by a, it suffices to show that E(X 2 X 1 ) 0 almost surely. Let n = {ω : E(X 2 X 1 ) 1 n } which is in. Then E(X 2 X 1 ) dp = X 2 X 1 dp 0 n n by assumption, means that P ( n ) = 0 for all n. Hence P ( n n) = 0 and hence E(X 2 X 1 ) 0 almost surely. 3. By definition Z = E(X 1 ). Furthermore, since sets which are -measurable are also E- measurable, then by the definition of conditional expectation and the previous observation: E(E(X E) ) dp = E(X E) dp = X dp = Z dp (a 1 X 1 +a 2 X 2 ) dp Proposition 4.4. Assume X, XZ are integrable random variables with Z -measurable and σ F. Then: a E(ZX ) = ZE(X ) b Ω ZX dp = ZE(X ) dp Ω 3

4 Proof. a b is immediate since Ω is always in a sigma-field. To prove a, we apply the usual argument when developing Lebesgue integrals of functions. Firstly, suppose Z = 1 where,. Then since : 1 X dp = X dp = E(X ) dp = 1 E(X ) dp By linearity, a holds when Z is simple and a holds for non-negative Z by the monotone convergence theorem. For general Z, a holds once we write Z = Z + Z. Note 4.5. If X 1 X 2 almost surely, then E(X 1 ) E(X 2 ) almost surely. Hence if X 1 = almost surely, then E(X 1 ) = E(X 2 ) almost surely. The convergence theorems for random variables apply for conditional expectations. Theorem 4.6. Let σ F. Then: a (MCT) Let X n be non-negative random variables increasing to an r.v. X with E(X) <. Then E(X n ) E(X ) almost surely. b (Fatou s Lemma) Let X n 0 be integrable, and lim inf X n be integrable. Then E(lim inf X n ) lim inf E(X n ) almost surely. c (CT) Assume X n X almost surely, where X n Y and E(Y ) < for some random variable Y. Then E(X n ) E(X ) almost surely. Proof. a We have, almost surely, that E(X ) E(X n+1 ) E(X n ) for all n. Let be the set of ω for which this property holds (P ( ) = 1), and U(ω) = lim n E(X n )(ω)1 (ω). Each E(X n )1 (ω) is also a conditional expectation since E(X n )1 (ω) = E(X n ) almost surely. Next, E(X n ) U pointwise by definition, means that for and by the monotone convergence theorem: E(X n ) dp = X n dp U dp = X dp Hence, U = E(X ) almost surely, implies that E(X n ) E(X ) almost surely. b The proofs of Fatou s Lemma and the ominated Convergence Theorem are analogous to those of the classical ones. Now, we will examine what it means to condition on a random variable. Recall E(X Y )(ω) = E(X σ(y ))(ω) where X, Y are random variables. Recall that σ(y ) = {Y 1 (B) : B S) if Y takes values in (S, S). Proposition 4.7. Let Y : (Ω, F) (S, S) be a random vector. Then, a random variable Z is σ(y )-measurable iff Z = φ(y ) for some measurable function φ : (S, S) (R, B(R)). Proof. ( ) is trivial since Y is σ(y )-measurable. For ( ), let σ(y ). Then = Y 1 (B) for some B S. Hence, 1 = 1 B (Y ). We will construct φ using this observation. Firstly, suppose Z be σ(y )-measurable and simple. That is, let Z = N i=1 α i1 i. By observation, Z = N i=1 α i1 Bi (Y ) = φ(y ), where φ(y) = N i=1 α i1 Bi (y). 4

5 Now let Z 0. Take {Z n } as a sequence of discrete skeletons converging to Z, where each Z n are σ(y )-measurable. By the previous case, there is a φ n such that Z n = φ n (Y ) for each n. Redefine, Zn = max k n Z k = max k n φ k (Y ) = φ n (Y ). Then φ n increase pointwise to a measurable map φ satisfying Z = φ(y ). Now, φ by take infinite values on a measure zero set so we get rid of those by redefining, φ(x) = φ1 R which also makes Z = φ(y ). For the general case, let Z R. Seperate into positive and negative parts by Z = Z + Z and by the previous case, Z = φ + (Y ) φ (Y ). Setting φ(x) = φ + (x) φ (x) makes Z = φ(y ). Note 4.7. If φ, φ : (S, S) (R, B(R)) are two such maps satisfying φ(y ) = Z, φ(y ) = Z, then φ = φ P Y -almost surely. That is, P Y (φ φ) = P (φ(y ) φ(y )) = 0. efinition 3. Let Y : (Ω, F) (S, S) and X be an integrable random variable. Then h(y) = E(X Y = y) is the unique, up to P Y -null sets map such that h : (S, S) (R, B(R)) satisfies h(y ) = E(X Y ). Call h(y), the conditional expectation of X given {Y = y}. When Y is a random variable with a probability density function function, we call h(y) the conditional density. For instance, if X, Y are real valued, then we claim that: for almost every y. This was proved in 419. E(X Y = y) = lim ɛ 0 E(X1 Y [y,y+ɛ) ) P (Y [y, y + ɛ)) 5