Variance of Discrete Random Variables Class 5, Jeremy Orloff and Jonathan Bloom

Size: px

Start display at page:

Download "Variance of Discrete Random Variables Class 5, Jeremy Orloff and Jonathan Bloom"

Alannah Watts
6 years ago
Views:

1 Variace of Discrete Radom Variables Class 5, Jeremy Orloff ad Joatha Bloom 1 Learig Goals 1. Be able to compute the variace ad stadard deviatio of a radom variable.. Uderstad that stadard deviatio is a measure of scale or spread. 3. Be able to compute variace usig the properties of scalig ad liearity. Spread The expected value (mea) of a radom variable is a measure of locatio or cetral tedecy. If you had to summarize a radom variable with a sigle umber, the mea would be a good choice. Still, the mea leaves out a good deal of iformatio. For example, the radom variables X ad Y below both have mea 0, but their probability mass is spread out about the mea quite differetly. values X values Y -3 3 pmf p(x) 1/10 /10 4/10 /10 1/10 pmf p(y) 1/ 1/ It s probably a little easier to see the differet spreads i plots of the probability mass fuctios. We use bars istead of dots to give a better sese of the mass. 4/10 p(x) pmf for X pmf for Y 1/ p(y) /10 1/ x pmf s for two differet distributios both with mea 0 I the ext sectio, we will lear how to quatify this spread. y 3 Variace ad stadard deviatio Takig the mea as the ceter of a radom variable s probability distributio, the variace is a measure of how much the probability mass is spread out aroud this ceter. We ll start with the formal defiitio of variace ad the upack its meaig. Defiitio: If X is a radom variable with mea E(X) = µ, the the variace of X is defied by Var(X) = E((X µ) ). 1

2 18.05 class 5, Variace of Discrete Radom Variables, Sprig 014 The stadard deviatio σ of X is defied by σ = Var(X). If the relevat radom variable is clear from cotext, the the variace ad stadard deviatio are ofte deoted by σ ad σ ( sigma ), just as the mea is µ ( mu ). What does this mea? First, let s rewrite the defiitio explicitly as a sum. If X takes values x 1, x,..., x with probability mass fuctio p(x i ) the Var(X) = E((X µ) ) = p(x i )(x i µ). I words, the formula for Var(X) says to take a weighted average of the squared distace to the mea. By squarig, we make sure we are averagig oly o-egative values, so that the spread to the right of the mea wo t cacel that to the left. By usig expectatio, we are weightig high probability values more tha low probability values. (See Example below.) Note o uits: 1. σ has the same uits as X.. Var(X) has the same uits as the square of X. So if X is i meters, the Var(X) is i meters squared. Because σ ad X have the same uits, the stadard deviatio is a atural measure of spread. Let s work some examples to make the otio of variace clear. Example 1. Compute the mea, variace ad stadard deviatio of the radom variable X with the followig table of values ad probabilities. value x pmf p(x) 1/4 1/4 1/ aswer: First we compute E(X) = 7/. The we exted the table to iclude (X 7/). value x p(x) 1/4 1/4 1/ (x 7/) 5/4 1/4 9/4 Now the computatio of the variace is similar to that of expectatio: i= Var(X) = + + = Takig the square root we have the stadard deviatio σ = 11/4. Example. For each radom variable X, Y, Z, ad W plot the pmf ad compute the mea ad variace. (i) value x pmf p(x) 1/5 1/5 1/5 1/5 1/5 (ii) value y pmf p(y) 1/10 /10 4/10 /10 1/10

3 18.05 class 5, Variace of Discrete Radom Variables, Sprig (iii) value z pmf p(z) 5/ /10 (iv) value w pmf p(w) aswer: Each radom variable has the same mea 3, but the probability is spread out differetly. I the plots below, we order the pmf s from largest to smallest variace: Z, X, Y, W..5 p(z) pmf for Z p(x) pmf for X 1/ z x 1 p(w).5 p(y) pmf for Y pmf for W y W Next we ll verify our visual ituitio by computig the variace of each of the variables. All of them have mea µ = 3. Sice the variace is defied as a expected value, we ca compute it usig the tables. (i) value x pmf p(x) 1/5 1/5 1/5 1/5 1/5 (X µ) Var(X) = E((X µ) ) = = (ii) value y p(y) 1/10 /10 4/10 /10 1/10 (Y µ) Var(Y ) = E((Y µ) ) = = (iii) value z pmf p(z) 5/ /10 (Z µ) Var(Z) = E((Z µ) ) = =

4 18.05 class 5, Variace of Discrete Radom Variables, Sprig (iv) value w pmf p(w) (W µ) Var(W ) = 0. Note that W does t vary, so it has variace 0! 3.1 The variace of a Beroulli(p) radom variable. Beroulli radom variables are fudametal, so we should kow their variace. If X Beroulli(p) the Var(X) = p(1 p). Proof: We kow that E(X) = p. We compute Var(X) usig a table. values X 0 1 pmf p(x) 1 p p (X µ) (0 p) (1 p) Var(X) = (1 p)p + p(1 p) = (1 p)p(1 p + p) = (1 p)p. As with all thigs Beroulli, you should remember this formula. Thik: For what value of p does Beroulli(p) have the highest variace? Try to aswer this by plottig the PMF for various p. 3. A word about idepedece So far we have bee usig the otio of idepedet radom variable without ever carefully defiig it. For example, a biomial distributio is the sum of idepedet Beroulli trials. This may (should?) have bothered you. Of course, we have a ituitive sese of what idepedece meas for experimetal trials. We also have the probabilistic sese that radom variables X ad Y are idepedet if kowig the value of X gives you o iformatio about the value of Y. I a few classes we will work with cotiuous radom variables ad joit probability fuctios. After that we will be ready for a full defiitio of idepedece. For ow we ca use the followig defiitio, which is exactly what you expect ad is valid for discrete radom variables. Defiitio: The discrete radom variables X ad Y are idepedet if P (X = a, Y = b) = P (X = a)p (Y = b) for ay values a, b. That is, the probabilities multiply. 3.3 Properties of variace The three most useful properties for computig variace are: 1. If X ad Y are idepedet the Var(X + Y ) = Var(X) + Var(Y ).

5 18.05 class 5, Variace of Discrete Radom Variables, Sprig For costats a ad b, Var(aX + b) = a Var(X). 3. Var(X) = E(X ) E(X). For Property 1, ote carefully the requiremet that X ad Y are idepedet. We will retur to the proof of Property 1 i a later class. Property 3 gives a formula for Var(X) that is ofte easier to use i had calculatios. The computer is happy to use the defiitio! We ll prove Properties ad 3 after some examples. Example 3. Suppose X ad Y are idepedet ad Var(X) = 3 ad Var(Y ) = 5. Fid: (i) Var(X + Y ), (ii) Var(3X + 4), (iii) Var(X + X), (iv) Var(X + 3Y ). aswer: To compute these variaces we make use of Properties 1 ad. (i) Sice X ad Y are idepedet, Var(X + Y ) = Var(X) + Var(Y ) = 8. (ii) Usig Property, Var(3X + 4) = 9 Var(X) = 7. (iii) Do t be fooled! Property 1 fails sice X is certaily ot idepedet of itself. We ca use Property : Var(X + X) = Var(X) = 4 Var(X) = 1. (Note: if we mistakely used Property 1, we would the wrog aswer of 6.) (iv) We use both Properties 1 ad. Var(X + 3Y ) = Var(X) + Var(3Y ) = = 48. Example 4. Use Property 3 to compute the variace of X Beroulli(p). aswer: From the table we have E(X ) = p. So Property 3 gives This agrees with our earlier calculatio. X 0 1 p(x) 1 p p X 0 1 Var(X) = E(X ) E(X) = p p = p(1 p). Example 5. Redo Example 1 usig Property 3. aswer: From the table X p(x) 1/4 1/4 1/ X 1 9 we have E(X) = 7/ ad E(X ) = = = So Var(X) = 15 (7/) = 11/4 as before i Example Variace of biomial(,p) Suppose X biomial(, p). Sice X is the sum of idepedet Beroulli(p) variables ad each Beroulli variable has variace p(1 p) we have X biomial(, p) Var(X) = p(1 p).

6 18.05 class 5, Variace of Discrete Radom Variables, Sprig Proof of properties ad 3 Proof of Property : This follows from the properties of E(X) ad some algebra. Let µ = E(X). The E(aX + b) = aµ + b ad Var(aX+b) = E((aX+b (aµ+b)) ) = E((aX aµ) ) = E(a (X µ) ) = a E((X µ) ) = a Var(X). Proof of Property 3: We use the properties of E(X) ad a bit of algebra. Remember that µ is a costat ad that E(X) = µ. E((X µ) ) = E(X µx + µ ) = E(X ) µe(x) + µ = E(X ) µ + µ = E(X ) µ = E(X ) E(X). QED 4 Tables of Distributios ad Properties Distributio rage X pmf p(x) mea E(X) variace Var(X) Beroulli(p) 0, 1 p(0) = 1 p, p(1) = p p p(1 p) Biomial(, p) 0, 1,..., p(k) = p k (1 p) k k p p(1 p) Uiform() 1,,..., p(k) = Geometric(p) 0, 1,,... p(k) = p(1 p) k 1 p p 1 p p Let X be a discrete radom variable with rage x 1, x,... ad pmf p(x j ). Expected Value: Variace: Syoyms: mea, average Notatio: E(X), µ Var(X), σ Defiitio: E(X) = p(x j )x j E((X µ) ) = p(x j )(x j µ) j Scale ad shift: E(aX + b) = ae(x) + b Var(aX + b) = a Var(X) Liearity: (for ay X, Y ) E(X + Y ) = E(X) + E(Y ) (for X, Y idepedet) Var(X + Y ) = Var(X) + Var(Y ) Fuctios of X: _ E(h(X)) = p(x j ) h(x j ) Alterative formula: Var(X) = E(X ) E(X) = E(X ) µ j

7 MIT OpeCourseWare Itroductio to Probability ad Statistics Sprig 014 For iformatio about citig these materials or our Terms of Use, visit:

n outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,

n outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n, CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe