2. Conditional Expectation (9/15/12; cf. Ross)

Transcription

1 2. Conditional Expectation (9/15/12; cf. Ross) Intro / Definition Examples Conditional Expectation Computing Probabilities by Conditioning 1

2 Intro / Definition Recall conditional probability: Pr(A B) Pr(A B)/Pr(B) if Pr(B) > 0. Suppose that X and Y are jointly discrete RV s. Then if Pr(Y y) > 0, Pr(X x Y y) Pr(X x Y y) Pr(Y y) f(x, y) f Y (y) Pr(X x Y 2) defines the probabilities on X given that Y 2. 2

3 Definition: If f Y (y) > 0, then f X Y (x y) f(x,y) f Y (y) conditional pmf/pdf of X given Y y. is the Remark: Usually just write f(x y) instead of f X Y (x y). Remark: Of course, f Y X (y x) f(y x) f(x,y) f X (x). 3

4 Old Discrete Example: f(x, y) Pr(X x, Y y). X 1 X 2 X 3 X 4 f Y (y) Y Y Y f X (x) Find f(x 2). 4

5 Then f(x 2) f(x, 2) f Y (2) f(x, 2) if x 1 0 if x if x if x 4 0 otherwise 5

6 Old Cts Example: f(x, y) 21 4 x2 y, if x 2 y 1 f X (x) 21 8 x2 (1 x 4 ), if 1 x 1 f Y (y) 7 2 y5/2, if 0 y 1 Find f(y X 1/2). 6

7 f(y 1 2 ) f(1 2, y) f X ( 1 2 ) y 14 ( ), if 1 4 y y, if 1 4 y 1 7

8 More generally, f(y x) f(x, y) f X (x) 21 4 x 2 y 21 8 x 2 (1 x 4 ), if x2 y 1 2y 1 x 4 if x2 y 1. Note: 2/(1 x 4 ) is a constant with respect to y, and we can check to see that f(y x) is a legit condl pdf: 1 x 2 2y 1 x 4 dy 1. 8

9 Typical Problem: Given f X (x) and f(y x), find f Y (y). Steps: (1) f(x, y) f X (x)f(y x) (2) f Y (y) R f(x, y) dx. Example: f X (x) 2x, 0 < x < 1. Given X x, suppose that Y x U(0, x). f Y (y). Now find 9

10 Solution: Y x U(0, x) f(y x) 1/x, 0 < y < x. So Thus, f(x, y) f X (x)f(y x) 2x 1 x, if 0 < x < 1 and 0 < y < x 2, if 0 < y < x < 1. f Y (y) R f(x, y) dx 1 y 2 dx 2(1 y), 0 < y < 1. 10

11 Conditional Expectation Usual definition of expectation: E[Y ] y yf(y) discrete R yf(y) dy continuous f(y x) is the conditional pdf/pmf of Y given X x. Definition: The conditional expectation of Y given X x is E[Y X x] y yf(y x) discrete R yf(y x) dy continuous 11

12 Note that E[Y X x] is a function of x. Example: Suppose that Then f(y X 2) 0.2 if y if y if y 3 0 otherwise E[Y X 2] y yf(y 2) 1(.2)+2(.3)+3(.5)

13 Old Cts Example: Recall that f(x, y) 21 4 x2 y, if x 2 y 1. f(y x) 2y 1 x 4 if x2 y 1. Thus, E[Y x] R yf(y x) dy 2 1 x 4 1 x 2 y2 dy x6 1 x 4. 13

14 Theorem (double expectations): E[E(Y X)] E[Y ]. Remarks: Yikes, what the heck is this!? The exp value (averaged over all X s) of the conditional exp value (of Y X) is the plain old exp value (of Y ). Think of the outside exp value as the exp value of h(x) E(Y X). Then the Law of the Unconscious Statistician miraculously gives us E[Y ]. 14

15 Proof (cts case): By the Unconscious Statistician, E[E(Y X)] R E(Y x)f X(x) dx R ( R yf(y x) dy ) f X (x) dx R R yf(y x)f X(x) dx dy R y R f(x, y) dx dy R yf Y (y) dy E[Y ]. 15

16 Old Example: Suppose f(x, y) 21 4 x2 y, if x 2 y 1. Find E[Y ] two ways. By previous examples, we know that f X (x) 21 8 x2 (1 x 4 ), if 1 x 1 f Y (y) 7 2 y5/2, if 0 y 1 E[Y x] x6 1 x 4. 16

17 Solution #1 (old, boring way): E[Y ] R yf Y (y) dy y7/2 dy 7 9. Solution #2 (new, exciting way): E[Y ] E[E(Y X)] R E(Y x)f X(x) dx 1 1 ( x6 )( ) 21 1 x 4 8 x2 (1 x 4 ) dx

18 Notice that both answers are the same (good)! Believe it or not, sometimes it s easier to calculate E[Y ] indirectly by using our double expectation trick. 18

19 Example: of the Geom(p). An alternative way to calculate the mean Let N Geom(p), e.g., N could be the number of coin flips before H appears. Let X 1 if first flip is H 0 otherwise. We ll apply a standard conditioning argument (in the discrete case) to compute E[N]. 19

20 E[N] E[E(N X)] E(N x)f X(x) x E(N X 0)Pr(X 0) + E(N X 1)Pr(X 1) (1 + E[N])(1 p) + 1(p). Solving, we get E[N] 1/p. 20

21 Theorem (expectation of a random number of RV s): Suppose that X 1, X 2,... are independent RV s, all with the same mean. Also suppose that N is a nonnegative, integer-valued RV, that s independent of the X i s. Then E N i1 X i E[N]E[X 1 ]. 21

22 Proof: By double expectation, we have N N E X i E E X i N i1 n1 n1 n1 n1 E E E i1 N i1 n i1 n i1 E[X 1 ] X i N n Pr(N n) X i N n Pr(N n) X i Pr(N n) ne[x 1 ]Pr(N n) n1 npr(n n). 22

23 Example: Suppose the number of times we roll a die is N Pois(10). If X i denotes the value of the ith toss, then the expected number of rolls is E N i1 X i E[N]E[X 1 ] 10(3.5) 35. Theorem: Under the same conditions as before, Var N i1 X i E[N]Var(X 1 ) + (E[X 1 ]) 2 Var(N). Proof: See, for instance, Ross. 23

24 Computing Probabilities by Conditioning Let A be some event, and define the RV Y as: Then Y 1 if A occurs 0 otherwise. E[Y ] y yf Y (y) Pr(Y 1) Pr(A). 24

25 Similarly, for any RV X, we have E[Y X x] yf Y (y x) y Pr(Y 1 X x) Pr(A X x). 25

26 Further, since E[Y ] E[E(Y X)], we have Pr(A) E[Y ] E[E(Y X)] R E[Y x]df X(x) R Pr(A X x)df X(x). 26

27 Example/Theorem: If X and Y are independent continuous RV s, then Pr(Y < X) R F Y (x)f X (x) dx, where F Y ( ) is the c.d.f. of Y and f X ( ) is the p.d.f. of X. 27

28 Proof: (Actually, there are many proofs.) Let the event A {Y < X}. Then Pr(Y < X) R Pr(Y < X X x)f X(x) dx R Pr(Y < x X x)f X(x) dx R Pr(Y < x)f X(x) dx (since X, Y are indep). 28

29 Example: If X Exp(µ) and Y Exp(λ) are independent RV s. Then Pr(Y < X) R F Y (x)f X (x) dx 0 (1 e λx )µe µx dx λ λ + µ. 29

30 Example/Theorem: If X and Y are independent continuous RV s, then Pr(X + Y < a) R F Y (a x)f X (x) dx, where F Y ( ) is the c.d.f. of Y and f X ( ) is the p.d.f. of X. The quantity X + Y is called a convolution. 30

31 Proof: Pr(X + Y < a) R Pr(X + Y < a X x)f X(x) dx R Pr(Y < a x X x)f X(x) dx R Pr(Y < a x)f X(x) dx (since X, Y are indep). 31

32 Example: Suppose X, Y iid Exp(λ). Note that F Y (a x) 1 e λ(a x) if a x 0 and x 0 (i.e., 0 x a) 0 if otherwise Pr(X + Y < a) R F Y (a x)f X (x) dx a 0 (1 e λ(a x) )λe λx dx 1 e λa λae λa, if a 0. d da Pr(X + Y < a) λ2 ae λa, a 0. This implies that X + Y Gamma(2, λ). 32