Expected Value and Variance

Transcription

1 Expected Value and Variance This handout, reorganizes the material in. of the text. It also adds to it slightly (Theorem on p.) and presents two results from chapter of the text, because they fit in here and will simplify the presentation of some of the material in chapters and. In addition, the handout contains a motivating discussion that leads up to the definition of expected value. Expected Value.. Motivating Discussion and Definition. Suppose that I play the following game with anyone who is interested in playing: I will charge a player an up-front fee F (to be determined). The player then flips a fair coin three times, and I pay him/her $.00 for each head that (s)he gets and $2.00 for each tail that (s)he gets. Below, I show the random variable Y that logs my payout as a function defined on a sample space as well as the the distribution of Y. Sample Space: Distribution: Sample point s: p(s) : Y(s) : HHH T HH HT H HHT HT T T HT T T H T T T 6 Value y: 6 P(Y = y): The question I want to consider is this: what would be an equitable value for F, the up-front fee? If I overcharge, I will be swindling people, but if I undercharge, I will eventually lose all my savings. In order to be fair, I should take in from each player the amount that, on average, I must pay out to each player: F should be the average payout per game. () The number referred to in box () is the theoretical average of the random variable Y ; it is called the expected value of Y and is denoted EY or µ Y. In order to discover the correct formula for EY, let us see what an µ is the Greek lowercase letter mu; I am certain that it was chosen because m is the first letter in the word mean.

2 estimate of it looks like. Suppose we estimate the average payout per game by repeating the game a huge number of times say 0 9 times, to be definite and then calculating the arithmetic average of the payouts: arithmetic average = y + y y , (2) where the numbers (y, y 2,...) are the amounts of the individual payouts. We now need to simplify (2). replaced by (): The first step is to group like terms together in the numerator, so that (2) is (++ +) + (++ +) + (++ +) + ( ) arithmetic average = 0 9 () (++ +) (++ +) (++ +) ( ) = ( ) ( ) ( ) ( 0 9 ) # of s # of s # of s # of 6 s arithmetic average = () Now, if you look carefully at (), you can discern the theoretical quantity that the arithmetic average is estimating. Since 0 9 is such a big number, we can be sure that ( ) # of s 0 9 P (Y = ), ( ) # of s 0 9 P (Y = ), ( ) # of s 0 9 P (Y = ), ( ) # of 6 s 0 9 and P (Y = 6). Making these approximate substitutions into the right side of () gives us the approximate equation arithmetic average P (Y = ) + P (Y = ) + P (Y = ) + 6P (Y = 6). () We have identified the theoretical average value of this particular random variable: it is the right side of (). The corresponding formula works for any discrete random variable. 2 Definition Let Y be a discrete random variable with distribution given by The expected value of Y (EY or µ Y ) is given by P (Y = y i ) = p i, (i ). EY = µ Y := y P (Y = y ) + y 2 P (Y = y 2 ) + y P (Y = y ) + = i y i P (Y = y i ). (6).2 Basic Properties of Expected Value. There are quite a few of these; together, they make expected value a very user-friendly quantity. The first of these, which is not in the text, simplifies the proofs of some of the others; it shows that E Y, which is defined from the distribution of Y, can also be viewed as a sum over the sample space on which Y is defined. 2 When the random variable has infinitely many values, the sum in (6) can diverge. It is strongly suggested that at this point you compute the expected value of the random variable on p. both from the definition and from formula (7). 2

3 Theorem E Y = Y (s)p(s). (7) Proof. For each i, let E i be the event Y = y i so that P (Y = y i ) = P (E i ). Observe that the events {E, E 2,...} partition S. Starting from the definition, we get: E Y = i y i P (Y = y i ) = y i P (E i ) i (by the definition of P (E i ) ) = ( ) y i p(s) i s E i (because y i = Y (s) for s E i ) = ( ) Y (s) p(s) i s E i (distribute ) = Y (s)p(s) i s E i (because the events {E, E 2,...} partition S ) = Y (s)p(s). Theorem 2 Let X, Y and be random variables defined on the same sample space, and let c and d be constants. a: If X is constant that is, if P (X = x 0 ) = for some x 0 then E X = x 0. b: E cy + d = ce Y + d (or, equivalently: µ cy +d = cµ Y + d.) c: E Y + = E Y + E (or, equivalently: µ Y + = µ Y + µ Y2.) Proof of a. E X = x 0 P (X = x 0 ) = x 0 = x 0. Proof of b. E cy + d = i (cy i + d)p (Y = y i ) = i ( cyi P (Y = y i ) + dp (Y = y i ) ) = i cy i P (Y = y i ) + i dp (Y = y i )) = c y i P (Y = y i ) + d P (Y = y i )) i i = ce Y + d () = ce Y + d. Proof of c. This can be proved more easily using (7) than it can by using the definition of expected value. E Y + = ) (Y + (s)p(s) (definition of Y + ) = ( ) Y (s) + (s) p(s) (distribute ) = Y (s)p(s) + (s)p(s)

4 (group into two sums ) = Y (s)p(s) + (s)p(s) (equation (7), used twice ) = E Y + E. Exercise Show that E Y µ Y = 0.. Independence and Expected Values. The concept of independent random variables is built upon the idea of independent events. Definition 2 Let X and Y be two random variables defined on the same sample space. X and Y are independent iff for every value x i in the distribution of X and every y j in the distribution of Y, the events X = x i and Y = y j are independent events; that is, ) P (X = x i Y = yj = P (X = x i )P (Y = y j ). Many desirable things happen when random variables are independent; in this section, I will focus on one particular one. Theorem If X and Y are independent random variables, then E XY = E X E Y (if X and Y are independent) () Proof. The simplest proof I can find uses both the definition and Theorem. For each value x j of X, let F j = X = x j, and for each value y i of Y, let E i = Y = y i. Note that the events {(F j E i )} partition S. E X E Y = x j P (X = x j ) y i P (Y = y i ) j i = x j P (F j ) y i P (E i ) j i (multiply out ) = x j y i P (F j )P (E i ) j i (because X and Y are independent ) = x j y i P (F j E i ) j i (definition of P (F j E i ) ) = x j y i p(s) j i s F j E i (multiply out ) = x j y i p(s) j i s F j E i (correct for s (F j E i ) ) = X(s)Y (s)p(s) j i s F j E i (because the events{(f j E i )} partition S ) = X(s)Y (s)p(s) (formula (7) ) = E XY. Because they are independent, they are necessarily defined on the same sample space.

5 2 Variance and Standard Deviation. 2. Motivating Discussion and Definition. Just as the expected value µ Y of a random variable Y is analogous to the mean y of a sample (y, y 2,..., y n ), there is a number called the variance of Y that is analogous to the variance s 2 of the sample y, y 2,..., y n. The considerations that lead to the definition of s 2 apply in this case as well:. The goal is to find to measure the spread of the distribution of Y ; that is, to find a number that is small for a tight, compact distribution and large for a widely spread-out distribution. 2. The expected (signed) distance of a random value of Y from the mean µ Y does not work, because E Y µ Y = 0, always (Exercise ).. The expected (unsigned) distance of Y from µ Y, namely E Y µ Y, does work, but it is not particularly easy to work with.. The expected squared distance of Y from µ Y the variance also works, just as #2 does; but unlike #2, the variance has a number of nice mathematical properties, and (probably because of them) it is by far the most commonly used measure of dispersion.. The variance has one undesirable property, namely that its units are the square of the units of Y ; this can be fixed by taking the square root of the variance, to get the so-called standard deviation of Y. Definition Let Y be a discrete random variable.. The variance of Y, denoted V(Y) or σy, 2 is given by := E (Y µ Y ) 2. σ 2 Y 2. The standard deviation of Y, denoted σ Y, is the square root of the variance of Y : σ Y := σ 2 Y = E (Y µ Y ) Basic Properties of the Variance and Standard Deviation. While there are parallels between the properties of variance and those of expected value, there are also differences. Notice in particular that part d of Theorem requires independence, whereas part c of Theorem 2 (p.) does not. Theorem Let X, Y and be random variables defined on the same sample space, and let c and d be constants. a: σ 2 Y = E Y 2 µ 2 Y. b: If X is constant that is, if P (X = x 0 ) = for some x 0 then σ 2 X = 0.6 c: V (cy + d) = c 2 V (Y ) (or, equivalently: σ 2 cy +d = c 2 σ 2 Y.) d: If Y and are independent, V (Y + ) = V (Y )+V ( ) (or, equivalently: σ 2 Y + = σ 2 Y + σ 2.) with the exception of the (n ) in the denominator in the definition of s 2. The reason for that will become clear when you reach Chapter. 6 The converse of b is also true (Exercise ).

6 Proof of a. (by Theorem 2 part c ) = E (by Theorem 2 parts a and b ) = E (because E Y = µ Y ) = E = E σ 2 Y = E (Y µ Y ) 2 = E 2Y µ Y + µ 2 Y + E 2µ Y E 2µ 2 Y + µ 2 Y µ 2 Y. 2Y µ Y + E µ 2 Y Y + µ 2 Y Proof of b. Since P (X = x 0 ) =, it is also true that P (X 2 = x 2 0) =, so that by part a of Theorem 2, E X 2 = x 2 0. Then, by part a of Theorem, Proof of c. σx 2 = E X 2 (µ X ) 2 = x 2 0 (µ X ) 2 = x 2 0 (x 0 ) 2 = 0. V (cy + d) = ( ) 2 E cy + d µ cy +d (by Theorem 2b ) = ( ) 2 E cy + d (cµ Y + d) ( ) 2 (arithmetic ) = E c(y µ Y ) (arithmetic ) = E c 2 (Y µ Y ) 2 (by Theorem 2b ) = c 2 E (Y µ Y ) 2 = c 2 V (Y ). Proof of d. V (Y + ) = ) 2 E (Y + µ Y+Y2 (by Theorem 2c ) = ( ) 2 E Y + (µ Y + µ Y2 ) ( ) 2 (arithmetic ) = E (Y µ Y ) + ( µ Y2 ) (arithmetic ) = E ((Y µ Y ) 2 + 2(Y µ Y )( µ Y2 ) + ( µ Y2 ) 2 ( (by Theorem 2 b,c ) = E (Y µ Y ) 2 +2E (Y µ Y )( µ Y2 ) +E ( µ Y2 ) 2 ( (by Theorem (we have independence) ) = E (Y µ Y ) 2 +2E Y µ Y E µ Y2 +E ( µ Y2 ) 2 ( (by Exercise on p. ) = E (Y µ Y ) 2 +2(0)(0)+E ( µ Y2 ) 2 = V (Y ) + V ( ). Exercise 2 For each part of Theorem, there is a corresponding fact about standard deviations. Write these facts down. Exercise Show that if σ 2 Y = 0, then Y is a constant. 6