Solutions Homework 6

Transcription

1 1 Solutions Homework 6 October 26, 2015 Solution to Exercise 1.5.9: Part (a) is easy: we know E[X Y ] = k if X = k, a.s. The function φ(y) k is Borel measurable from the range space of Y to, and E[X Y ] = φ(y ), a.s. For part (b), if X 1 X 2, a.s., then φ 1 (Y ) = E[X 1 Y ] E[X 2 Y ] = φ 2 (Y ), a.s., so we claim that φ 1 (y) = E[X 1 Y = y] E[X 2 Y = y] = φ 2 (y), P Y -a.s. Letting A = {y : φ 1 (y) > φ 2 (y)}, we have P Y (A) = P (Y 1 (A)) = P ({ω : Y (ω) A}) = P ({ω : φ 1 (Y (ω)) > φ 2 (Y (ω))}) = 0. For part (c), the obvious conjecture is that E[a 1 X 1 + a 2 X 2 Y = y] = a 1 E[X 1 Y = y] + a 2 E[X 2 Y = y], P Y a.s. We have from the theorem that E[a 1 X 1 + a 2 X 2 Y ] = a 1 E[X 1 Y ] + a 2 E[X 2 Y ], P a.s. Now we know that E[a 1 X 1 + a 2 X 2 Y ], E[X 1 Y ], and E[X 2 Y ] can each be expressed as a function of Y, say E[a 1 X 1 + a 2 X 2 Y ] = h(y ) E[X 1 Y ] = h 1 (Y ) E[X 2 Y ] = h 2 (Y ). See the discussion beginning in the middle of p. 70. Our equation above then says h(y ) = a 1 h 1 (Y ) + a 1 h 1 (Y ), P a.s. Also, by definition, So, can t we conclude that E[a 1 X 1 + a 2 X 2 Y = y] = h(y) E[X 1 Y = y] = h 1 (y) E[X 2 Y = y] = h 2 (y). h(y) = a 1 h 1 (y) + a 1 h 1 (y), P Y a.s.?

2 2 Let A = {y : h(y) a 1 h 1 (y) + a 1 h 1 (y)}. This is a subset of the range space of Y. It is measurable in that space since it is the inverse image of the Borel set \ {0} under the measurable map h (a 1 h 1 + a 2 h 2 ). We want to show that P Y (A) = 0. Now P Y (A) = P ( Y 1 (A) ) = P ({ω : h(y (ω)) a 1 h 1 (Y (ω)) + a 1 h 1 (Y (ω))}). We already observed this latter event has probability 0 when we noted that h(y ) = a 1 h 1 (Y ) + a 1 h 1 (Y ), P -a.s. So we are done. The previous paragraph illustrates one way of proving a result for the conditional expectation of the type E[X Y = y]: prove the corresponding result for for conditional expectation of the type E[X Y ] and simply translate it over. This will work in most cases, so one doesn t have to do separate proofs. The P -null sets in Ω where an equality fails will automatically become a P Y -null set on the range of Y by the same sort of argument as above. However, most students seem to want to derive a result using the defining properties of E[X Y = y]. So, for example, we observe that a 1 E[X 1 Y = y] + a 2 E[X 2 Y = y] is a Borel measurable function of y (whose domain is the range space of Y ), and if A is a measurable set in the range space of Y, then Y 1 (A) (a 1 X 1 + a 2 X 2 ) dp = = Y 1 (A) A (a 1 E[X 1 Y ] + a 2 E[X 2 Y ]) dp (a 1 E[X 1 Y = y] + a 2 E[X 2 Y = y]) dp Y (y), which shows that a 1 E[X 1 Y = y] + a 2 E[X 2 Y = y] has satisifies the integral property to be E[a 1 X 1 + a 2 X 2 Y = y]. Moving on to part (d), it is very tempting to write E[E[X Y = y]] but this doesn t make sense. For a r.v. Z, E[Z] = Ω ZdP is an integral over the underlying probability space, but the domain of the function E[X Y = y] is the range of Y, not Ω. Of course, the range of Y has the probability measure P Y, so it makes sense to write Λ E[X Y = y] dp Y (y) = E[X],

3 where Λ is the range of Y. Since E[X Y = y] y=y is E[X Y ], we have by the law of the unconscious statistician that Λ E[X Y = y] dp Y (y) = Y 1 (Λ) E[X Y ] dp = E[E[X Y ]] = E[X] where the last equality follows by part (d) of Theorem 1.5.7(d). To deal with part (e) of the theorem, we need to translate E[X {, Ω}] into some kind of statement about E[X Y = y], we need to think of it as E[X Y ] where σ(y ) = {, Ω}. But this happens if and only if Y is a constant r.v. (Check it out!). Hence, we claim 3 If Y is a constant r.v., E[X Y = y] = E[X], P Y -a.s.. Clearly if Y = c where c Λ is fixed, then for h(y ) = E[X Y ] = E[X], we must have h(c) = E[X], and h(y) can be defined arbitrarily for y c. But this means h(y) = E[X], P Y -a.s., since P Y = δ c. Theorem 1.5.7(f) tells us that if σ(x) σ(y ), then E[X Y ] = X, a.s. Now, it wouldn t make sense to claim E[X Y = y] is equal to X, since they are functions with different domains. However, if σ(x) σ(y ), then we know from Theorem that X = φ(y ) for some function φ whose domain is Λ, the range of Y. Thus, it would make sense to claim that If X = φ(y )for some measurable φ, then E[X Y = y] = φ(y), P Y a.s. The proof is immediate from φ(y ) = X = E[X Y ], a.s. Part (g) is a little trickier. Suppose Y 1 is some other random element (with range space (Λ 1, G 1 ), say), and σ(y 1 ) σ(y ). We know (at least if the range of Y 1 is (, B) that then Y 1 = ψ(y ), for some ψ, by Theorem So let us just assume Y 1 = ψ(y ), ψ : (Λ, G) (Λ 1, G 1 ). Now it makes no sense to write E[E[X Y 1 = y 1 ] Y = y] since the domain of E[X Y 1 = y 1 ] is Λ 1, and not Ω. (Recall that when we write

4 4 E[Z Y = y], Z must be a r.v., i.e. a mapping from Ω to.) Also, how are we to match up the given values y 1 and y? If we are given Y = y and Y 1 = ψ(y ), then it must be that Y 1 = ψ(y). So let s try the following: If Y 1 = ψ(y ), then E[E[X Y 1 ] Y = y] = E[X Y 1 = ψ(y)], P Y1 a.s. Of course, we haven t fully interpreted E[E[X Y 1 ] Y = y] into the requisite kind of conditional expectation, but there is no way to do so. Also, what does the r.h.s. mean? We are taking the function h 1 given by h 1 (y 1 ) = E[X Y 1 = y 1 ] and composing it with ψ, i.e. the r.h.s. is h ψ evaluated at y, and h ψ is a map from Λ to, so the domains and ranges of the two sides match. Again, the proof is trivial: E[X Y 1 ] = h(y 1 ) = h(ψ(y )) = (h ψ)(y ), so we apply our result above on part (f) of the theorem. Now for the other side of the Law of Successive conditioning, we need to deal with E[E[X Y ] Y 1 = y 1 ] when Y 1 = ψ(y ). We could just write If Y 1 = ψ(y ), then E[E[X Y ] Y 1 = y 1 ] = E[X Y 1 = y 1 ], P Y1 a.s. Certainly everything makes sense: the l.h.s. and r.h.s. of the equation are the same kind of objects (both sides are functions with argument y 1 varying over the domain Λ 1 and the ranges are ). And we know that E[E[X Y ] Y 1 ] = E[X Y 1 ] a.s. in this case, which proves the result. This one is easy. Finally, part (h) is fairly straightforward: If X 1 is σ(y )-measurable, then X 1 = ψ(y ) for some measurable ψ, and we claim that E[ψ(Y )X 2 Y = y] = ψ(y)e[x 2 Y = y]. We see that φ(y) = ψ(y)e[x 2 Y = y] is a Borel measurable function on the range of Y (since it is the product of two such functions), and φ(y ) = ψ(y )E[X 2 Y ] = E[ψ(Y )X 2 Y ], where the last equality follows from the result given in the theorem. The translation and proof of Theorem is straightforward. For part (a), if 0 X n X, then we claim E[X n Y = y] E[X Y = y], P Y -a.s. We know that φ n (Y ) = E[X n Y ] E[X Y ] = φ(y ), a.s. Simply translate the null set to the range space of Y as in part (b) of the previous theorem. The dominated convergence theorem is similar.

5 Solution to Exercise : Let Z Z, then MSPE(Z) is given by E[(X Z) 2 ] = E[(X E[X Y ] + E[X Y ] Z) 2 ] adding and subtracting E[X Y ] = E[(X E[X Y ]) 2 ] + 2E[(X E[X Y ])(E[X Y ] Z)] + E[(E[X Y ] Z) 2 ] algebra and linearity of expectation = E[(X E[X Y ]) 2 ] + 2E[E[(X E[X Y ])(E[X Y ] Z) Y ]] + E[(E[X Y ] Z) 2 ] total expectation = E[(X E[X Y ]) 2 ] + 2E[(E[X Y ] Z)E[(X E[X Y ]) Y ]] + E[(E[X Y ] Z) 2 ] factorization result (Theorem 1.5.7(h)) since both E[X Y ] and Z are σ(y )-measurable = E[(X E[X Y ]) 2 ] + 2E[(E[X Y ] Z)(E[X Y ] E[X Y ])] + E[(E[X Y ] Z) 2 ] by linearity of E[ G] and Theorem 1.5.7(f) applied to E[X G] = E[(X E[X Y ]) 2 ] + E[(E[X Y ] Z) 2 ] = MSPE(E[X Y ]) + E[(E[X Y ] Z) 2 ]. Thus, MSPE(Z) MSPE(E[X Y ]) = E[(E[X Y ] Z) 2 ] 0, so the result follows. The desired uniqueness result is evident as well: E[(E[X Y ] Z) 2 ] > 0 unles (E[X Y ] Z) 2 = 0, a.s., which is equivalent to Z = E[X Y ], a.s. REMARK. Note the similarities in the calculations for the solution to this exercise and Exercise 1.5.7(b). Solution to Exercise : Let p(b, x) = δ x (B) = I B (x), for x and B B. Clearly p(, x) is a Borel probability measure for each x, so condition (ii) in the definition of conditional distributions holds (Definition 1.5.2). Turning to condition (i), of course P [X 5

6 6 B X] = E[I B (X) X] by the definition of conditional probability, and by Theorem 1.5.7(f), E[I B (X) X] = I B (X), a.s., so E[I B (X) X = x] = I B (x), P X -a.s. But, as noted above, this is just p(b, x). This is intuitively the right answer since given X = x, we should have a conditional probability totally concentrated on x, which is what δ x does. Solution to Exercise : We are looking for a conditional distribution of a two dimensional random vector, so our conditional distribution has to live on two dimensional space. Note that we are given that the first component X 1 = x 1, so its marginal conditional distribution should be δ x1. See the result in Exercise Now, the marginal conditional distribution for X 2 should be the obvious answer: P X2 X 1 ( x 1 ) = Law[X 2 X 1 = x 1 ]. Note that any r.v. is independent of a degenerate r.v., so there is only one way to make a joint distribution with these marginals: p(, x 1 ) = δ x1 Law[X 2 X 1 = x 1 ]. Let s check that this satisfies the requisite properties as spelled out in Remark 1.5.7(a): (1) For all x 1, p(, x 1 ) is a Borel p.m. on ( 2, B 2 ) since it is the product of two Borel p.m. s on. (2) We want to check that for all B B 2, p(b, x 1 ) is a measurable function of x 1. Note that p(b, x 1 ) = I B (ξ 1, ξ 2 ) d [ δ x1 P X2 X 1 ( x 1 ) ] (ξ 1, ξ 2 ) 2 [ ] = I B (ξ 1, ξ 2 ) dδ x1 (ξ 1 ) dp X2 X 1 ( x 1 )(ξ 2 ) = I B (x 1, ξ 2 ) dp X2 X 1 ( x 1 )(ξ 2 ) = P X2 X 1 (B 1 (x 1 ) x 1 ) = P [X 2 B 1 (x 1 ) X 1 = x 1 ], where B 1 (x 1 ) = {x 2 : (x 1, x 2 ) B}.

7 Note that for each x 1, this set is measurable this is implicitly one of the conclusions of Fubini s theorem, that we can fix the value of one variable and then the function is measurable in the other variable. Now, the last expression in our little calculation above, namely P [X 2 B 1 (x 1 ) X 1 = x 1 ], is a Borel function of x 1 by definition of (this kind of) conditional probability (expectation). (3) Now, we want to show that for all Borel sets A, B 2, P [X 1 A & (X 1, X 2 ) B] = I A (x 1 )p(b, x 1 )dp X1 (x 1 ). If we write out p(b, x 1 ) as an integral and carry out the calculation as in the previous step, then we obtain I A (x 1 )p(b, x 1 )dp X1 (x 1 ) [ ] = I A (x 1 ) I B (x 1, x 2 ) dp X2 X 1 ( x 1 )(x 2 ) dp X1 (x 1 ) = I A (x 1 )I B (x 1, x 2 ) dp X2 X 1 ( x 1 )(x 2 )dp X1 (x 1 ) = E [I A (X 1 )I B (X 1, X 2 ) X 1 = x 1 ] dp X1 (x 1 ) (by equation (1.71) in Theorem 1.5.6) = E [E [I A (X 1 )I B (X 1, X 2 ) X 1 ]] = E [I A (X 1 )I B (X 1, X 2 )] = P [X 1 A & (X 1, X 2 ) B], which is the desired result. 7