Expectations and moments

Transcription

1 slidenumbers,noslidenumbers debugmode,normalmode slidenumbers debugmode Expectations and moments Helle Bunzel

2 Expectations and moments Helle Bunzel

3 Expectations of Random Variables Denition The expected value of a discrete random variable exists and is dened by E [x] = x x f (x), if and only if x jxj f (x) <? Discrete case: This is the weighted average of the values taken by x.? Clearly x x f (x) < is needed. Why is x jxj f (x) < needed?? Consider L = n= ( ) n+ n =? It can be shown that L = log(2)? Note that n= ( ) n+ n! () Expectations and moments, page 3 of 60

4 Expectations of Random Variables? Now, 2 L = L = ? Then, adding () and (2), we get 3 L = ( + 0) L = ? Same terms as in () but converges to different value (2) Expectations and moments, page 4 of 60

5 Expectations of Random Variables? Conclusion: Absolute convergence is required for uniqueness of the expected value. Expectations and moments, page 5 of 60

6 Example? A life insurance company offers a 50-year-old male $000 face value, one year term life insurance for a premium of $4.? Standard mortality tables indicate that the probability that a male in this age group will die within a year is ? What is the expected gain of the insurance company?? Dene a random variable X. What is the relevant sample space?? What are the corresponding probabilities? E [X] = ( 986) = 8? Note that 8 is not in the sample space.? Throwing a dice: What is the expected value? Expectations and moments, page 6 of 60

7 Expectations of Random Variables Denition 2 The expected value of a continuous random variable exists and is dened by E [x] = R x x f (x) dx, if and only if R x jxj f (x) dx <? This, too, can be thought of as a weighted sum.? Consider f (x) taking positive values on [a, b] and 0 elsewhere.? Let x 0 = a < x < x 2 <... < x n = b? Let x i = x i x i.? Consider the sum i= n X0 i f X0 i? Picture: xi, where X 0 i 2 [x i, x i ]. a 0 X x X 4 X x 2 2 X x 3 3 b Expectations and moments, page 7 of 60

8 Expectations of Random Variables? But lim n!, mesh!0 n Xi X 0 f i 0 i= Example Z x i =? A manufacturer mail out surveys each quarter. x x f (x) dx? The proportion of surveys returned each quarter is the outcome of a r.v. with density f (x) = 3x 2 I [0,] (x)? What is the expected proportion of returned survey any given quarter? Z Z E (x) = x f (x) dx = x3x 2 I [0,] (x) dx x Z 3 = 3x 3 dx = 4 x4 = Expectations and moments, page 8 of 60

9 Expectation of a function of a r.v.? Now, let g (x) be some function of x. Then x g (x) f (x) E [g (x)] = R g (x) f (x) dx? Intuition?? Different outcomes, but weighted by the same weights. x Expectations and moments, page 9 of 60

10 Examples? Continuous: Consider the following prot function: Π (X) = 5X 2X X is a perishable commodity delivered for processing. f (X) = +2x 0 [0,0] (x) Expected value of prot: E (Π) = Z = 3 0 = 3 0 (5x 2x) + 2x 0 [0,0] (x) dx = 3 x + 2x 2 dx = = = 3.77 Z 0 Z 0 2 x x x + 2x 0 dx Expectations and moments, page 0 of 60

11 Examples? Discrete Values Mean Y ? Consider /Y Values /4 /6 /8 /0 /2 Mean /Y = (/8)? Note!!!!! E [g (x)] 6= g (E (x)) Expectations and moments, page of 60

12 Examples? Also note that existence of E [g (x)] is not implied by existence of E [x] : Values 0 Mean Y ? Consider /Y. Absolute convergence? Expectations and moments, page 2 of 60

13 Jensen's inequality Theorem 3 Let X be a r.v. and let g be a continuos function. Then a) E [g (X)] g (E (X)) if g is convex and E [g (X)] > g (E (X)) if g is strictly convex and X is not degenerate b) E [g (X)] g (E (X)) if g is concave and E [g (X)] < g (E (X)) if g is strictly concave and X is not degenerate. Proof of a)? Since g is convex, the tangents to g lie below the function. Expectations and moments, page 3 of 60

14 Jensen's inequality? Let l (x) = a + bx be the equation for the tangent to the point (E (X), g (E (X))).? Then g (x) l (x) = a + bx 8x and g (E (X)) = a + be (X) and E (g (X)) = g (x) f (x) (a + bx) f (x) = a + be (X) = g (E (X)) and E (g (X)) = Z g (x) f (x) Z (a + bx) f (x) = a + be (X) = g (E (X)) Expectations and moments, page 4 of 60

15 Expectations of multivariate r.v.s Denition 4 Let X = (X, X 2,..., X n ) be a multivariate random variable. Then Z Z E [X] = (E [X ], E [X 2 ],..., E [X n ]) = x f X (x),..., x f Xn (x) Theorem 5 Let X = (X, X 2,..., X n ) be a multivariate random variable with density function f (x, x 2,..., x n ). Then the expectation of Y = g (x, x 2,..., x n ) is E [Y] = g (x, x 2,..., x n ) f (x, x 2,..., x n ) if X is discrete E [Y] = Z Z if X is continuous g (x, x 2,..., x n ) f (x, x 2,..., x n ) dx dx 2...dx n Expectations and moments, page 5 of 60

16 Properties of expectations. Let X be a r.v. with density f (x) and A an event for X. Then E ( A (x)) = P X (A) 2. If c is a constant the E (c) = c. 3. If c is a constant the E (cx) = ce (X). h i 4. E i= k g i (X, X 2,..., X n ) = i= k E [g i (X, X 2,..., X n )] 5. If (X, X 2,..., X n ) are independent random variables then E [ n i= X i] = n i= E [X i] Expectations and moments, page 6 of 60

17 Proof of 5. E Continuous case " n i= X i # = = = Z Z Z = E (X ) Z Z Z Z =... n = E (X i ) i= n i= n i= n x i! x i! f (x, x 2,..., x n ) dx dx 2...dx n n i= f Xi (x i )! dx dx 2...dx n Z n x i! x f X (x ) dx i=2 f Xi (x i ) i=2!! Z n n x i f Xi (x i ) dx 2...dx n i=2 i=2! dx 2...dx n Expectations and moments, page 7 of 60

18 Moments of a random variable? Two types of moments: Moments about the origin Moments about the mean. (Central moments) Denition 6 Let X be a r.v. with density f (x). Then the r 0 th moment about the origin, denoted by µ r 0 is dened as? Note that: µ 0 0 = µ 0 = E (X) µ 0 r = E (Xr ) Expectations and moments, page 8 of 60

19 Moments of a random variable Denition 7 Let X be a r.v. with density f (x) and mean µ. Then the r 0 th central moment, denoted by µ r is dened as µ r = E (X µ) r? Note that: µ 0 = µ = 0? Second central moment: Variance h V [X] = E (X µ) 2i = ( R x (x x (x µ) 2 f (x) µ)2 f (x) dx? Frequently used symbol: σ 2, standard deviation σ. Expectations and moments, page 9 of 60

20 Properties of variance? Variance of ax + b : V [ax + b] = a 2 V [X]? A useful formula: V [X] = E h X 2i µ 2? V [a] = 0.? A constant is a special case of a random variable. Variance 0 and P (X = a) =. Discrete or continuous? Expectations and moments, page 20 of 60

21 Moments of a random variable? A couple of useful inequalities: Theorem 8 Markov's Inequality: Let X be a r.v. with density function f (x), and let g be a nonnegative-valued function of X. Then P (g (X) a) E(g(X)) a for any value a > 0.? If E (g (X)) exists, we can always put an upper bound on the probability that g (X) a. Theorem 9 Chebyshev's Inequality: P (jx µj kσ) k 2 for k > 0.? The probability that x falls more than k standard deviations away from the mean of X.? No densities assumed!! Expectations and moments, page 2 of 60

22 Proof of Markov's Inequality? We will prove the continuous case. E (g (X)) = = Z Z g (x) f (x) dx fx:g(x)<ag g (x) f (x) dx + Z fx:g(x)ag g (x) f (x) dx? Note that R fx:g(x)<ag g (x) f (x) dx 0 How do we know that? Expectations and moments, page 22 of 60

23 Proof of Markov's Inequality E (g (X)) = Z Z Z = a fx:g(x)<ag fx:g(x)ag fx:g(x)ag Z fx:g(x)ag g (x) f (x) dx + g (x) f (x) dx a f (x) dx f (x) dx = ap (g (x) a), P (g (X) a) Z E (g (X)) a fx:g(x)ag g (x) f (x) dx Expectations and moments, page 23 of 60

24 Proof of Chebyshev's Inequality? Recall Markov's Inequality: P (g (X) a) E (g (X)) a? Let g (x) = (x µ) 2 and a = k 2 σ 2. Then, by Markov's inequality, (X P (X µ) 2 k 2 σ 2 E µ) 2 k 2 σ 2, P (jx µj kσ) σ2 k 2 σ 2 = k 2 Expectations and moments, page 24 of 60

25 Moments of a random variable? Note that we can re-write Chebychev's inequality: P (jx µj kσ) k 2, P (jx µj kσ) P (jx µj < kσ)? Now, let k = σ c. c : Some small constant.? As σ 2! 0 P (jx µj < c) k 2, k 2 σ 2 c 2 P (jx µj < c) P (jx µj < c) = P (µ c < X < µ + c) = Expectations and moments, page 25 of 60

26 Moments of a random variable? Intuition?? Example: µ = 0, c = 0. Then σ = 5 ) P ( 0 < x < 0) σ = 2 ) P ( 0 < x < 0) σ = ) P ( 0 < x < 0) σ 2 c 2 = 0.75 σ 2 c 2 = 0.96 σ 2 c 2 = 0.99 Expectations and moments, page 26 of 60