2. Discrete Random Variables Part II: Expecta:on. ECE 302 Spring 2012 Purdue University, School of ECE Prof. Ilya Pollak

Transcription

1 2. Discrete Random Variables Part II: Expecta:on ECE 302 Spring 2012 Purdue University, School of ECE Prof.

2 Expected value of X: Defini:on

3 Expected value of X: Defini:on E[X] is also called the mean of X

4 Example: mean of a Bernoulli random variable

5 Example: mean of a discrete uniform random variable (Note: E[X] is not necessarily one of the values that X can assume with a non-zero probability!)

6 Expected value: Center of gravity of PMF Imagine that a box with weight p X (x) is placed at each x. Center of gravity c is the point at which the sum of the torques is zero: c x

7 Expected value and empirical mean Many independent Bernoulli trials with p=0.2 Bernoulli random variables X 1, X 2, E[X n ]=0.2 Empirical average from many independent experiments close to the expected value (law of large numbers)

8 What E[X] is NOT It s not necessarily the most likely value of X It s not even always the case that P(X=E[X])>0 It s not guaranteed to equal to the empirical average

9 Two ways to evaluate E[g(X)] This is true because g(x)p X (x) = g(x) p X (x) x y {x g(x)= y} = y p X (x) y {x g(x)= y} y P(g( X )= y)= p Y (y) = yp Y (y) = E[Y ] y

10 Cau:on: In general, E[g(X)] g(e[x]) Example: average speed vs average :me. Suppose you need to drive 60 miles. A very bad storm will hit your area with probability 1/2. If the storm hits, you will drive 30 miles/hour during the en:re trip; If the storm does not hit, you will do 60 miles/hour during the en:re trip. What s the expected value of your speed? (1/2) 30 + (1/2) 60 = 45 mph Is the expected value of your travel :me 60/45 = 1 hr 20 min? NO: T = 60/V, therefore E[T] = (1/2) 60/30 + (1/2) 60/60 = 1 hr 30 min Because you have equal chances to spend 1 hour or 2 hours driving. Interpreta:on: in N independent repe::ons of the trip You would expect to spend a total of 1.5N hours driving Your average speed per trip (not per unit :me traveled!) would be about 45mph

11 What is the meaning of E[b] if b is a non- random number?

12 Linearity of expecta:on

13 Variance and Standard Devia:on of X: var(x) = E Defini:ons [( X E[X] ) 2 ]

14 Calcula:ng var(x) var(x) = E [( X E[X] ) 2 ] = (x E[X]) x = ( x 2 2xE[X] + (E[X]) 2 )p X (x) x 2 px (x) = x 2 p X (x) 2 xe[x]p X (x) + (E[X]) 2 p X (x) x x x x x x = x 2 p X (x) 2E[X] xp X (x) + (E[X]) 2 p X (x) E[ X 2 ] E[ X ] 1 = E[X 2 ] (E[X]) 2 (E[ X ]) 2 Sometimes this is easier to compute than E [( X E[X] ) 2 ]

15 Example 2.4: Variance of a Bernoulli random variable

16 Another Bernoulli random variable Toss a coin with P(H) = p, and let Y = a if H b if T p if k = a Then p Y (k) = 1 p if k = b Note : Y b a b = 1 if H = X 0 if T Therefore, E[Y 2 ] = E[X 2 (a b) 2 + 2X(a b)b + b 2 ] How to compute the expectation and variance of this random variable? Y = X(a b) + b E[Y] = E[X](a b) + b = p(a b) + b = E[X 2 ](a b) 2 + 2E[X](a b)b + b 2 = p(a b) 2 + 2p(a b)b + b 2

17 Another Bernoulli random variable Toss a coin with P(H) = p, and let Y = a if H b if T p if k = a Then p Y (k) = 1 p if k = b Note : Y b a b = 1 if H = X 0 if T Therefore, E[Y 2 ] = p(a b) 2 + 2p(a b)b + b 2 var(y ) = E[Y 2 ] (E[Y ]) 2 = (p p 2 )(a b) 2 How to compute the expectation and variance of this random variable? Y = X(a b) + b E[Y] = E[X](a b) + b = p(a b) + b

18 Standard devia:on as measurement error Run a series of independent, iden:cal experiments, e.g., Bernoulli trials. Empirically es:mate the probability of an event, say, the success in a Bernoulli trial, as the number of successes divided by the number of experiments. Standard devia:on characterizes by how much we would expect our es:mate to deviate from the actual probability of success, over many experiments. For example, if p=0.2, then E[X] = 0.2, var(x)=0.16, and standard devia:on of X is 0.4.

19 Root mean- square devia:on of the es:mate of p from 0.2, as a func:on of the number of trials

20 Standard devia:on as risk Suppose you have two investment opportuni:es: Opportunity 1: invest $1000, have equal chances of a total wipeout or of $2000 profit Opportunity 2: invest $1000, earn $500 profit guaranteed. Expected profit for 1 is 0.5( $1000) + 0.5($2000) = $500. Expected profit for 2 is $500. But clearly the two opportuni:es are very different: you risk a lot under the first one, whereas the second one is riskless. Standard devia:on characterizes the risk: Opportunity 1: st.dev. = (0.5( ) ( ) 2 ) 1/2 = 1500 Opportunity 2: st.dev. = (1( ) 2 ) 1/2 = 0 Standard devia:on characterizes the spread of possible profits around the expected profit.

21 Problem 2.20: Expecta:on and variance of a geometric random variable As an ad campaign, a chocolate factory places golden :ckets in some of its candy bars, with the promise that a golden :cket is worth a trip through the chocolate factory, and all the chocolate you can eat for life. If the probability of finding a gold :cket is p, find the mean and the variance of the number of candy bars you need to eat to find a :cket.

22 Mean of a geometric random variable Let C = # candy bars un:l 1 st success Model C as geometric with parameter p: p C (k) = E[C] = (1 p)k 1 p, k =1,2, 0, otherwise k=1 k(1 p) k 1 p Useful fact : k(1 p) k 1 = d dp {(1 } p)k

23 Mean of a geometric random variable Let C = # candy bars un:l 1 st success Model C as geometric with parameter p: p C (k) = E[C] = (1 p)k 1 p, k =1,2, 0, otherwise k=1 k(1 p) k 1 p Useful fact : k(1 p) k 1 = d dp {(1 } p)k

24 Mean of a geometric random variable Let C = # candy bars un:l 1 st success Model C as geometric with parameter p: p C (k) = E[C] = (1 p)k 1 p, k =1,2, 0, otherwise k=1 k(1 p) k 1 p Useful fact : k(1 p) k 1 = d dp d Therefore, E[C] = p dp k =1 {(1 } p)k {(1 } p)k = p d dp k =1 = p d 1 p dp p = p 1 p 2 = 1 p (1 p) k

25 How many candy bars un:l first success? If there are five golden :ckets per 1,000,000 bars, then p=1/200,000. [Note: we assume an infinite number of bars so that C is truly geometric.] Then E[C] = 200,000. On average, have to buy 200,000 chocolate bars un:l first success.

26 E[C 2 ] [ ] = k 2 (1 p) k 1 p E C 2 k=1 Another useful fact : k 2 (1 p) k 1 = d 2 { } + d dp dp 2 (1 p) k +1 {(1 p)k } [Because the right - hand side is (k +1)k(1 p) k 1 k(1 p) k 1.] E C 2 =p d 2 (1 p) k +1 + p d (1 p) k dp 2 k =1 dp k =1 (1 p) 2 / p = p d 2 1 dp 2 p 2 + p 1 p = p d dp 1 p p = 2p p 1 3 p = 2 p 1 2 p 1/ p

27 Variance of a geometric random variable var(c) = E [ C 2 ] E[C] ( ) 2 = 2 p 1 2 p 1 p 2 = 1 p 2 1 p If p =1/200,000, the standard deviation is 200, , ,000 Thus, your actual number candy bars until first success may be quite far from the mean!