Lecture 6: E(X ), Var(X ), & Cov(X, Y ) Sta230/Mth230 Colin Rundel February 5, 2014 We have been using them for a while now in a variety of forms but it is good to explicitly define what we mean Random Variable A real-valued function on the sample space Ω Example: If Ω is the 36 element space resulting from rolling two fair six-sided dies (r and g), then the following are all random variables X (r, g) = r Y (r, g) = r g Z(r, g) = r + g Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 1 / 24, cont. - Vocabulary Random variables are in essence a fancy way of describing an event. Previous example: Ω = {(r, g) : 1 r, g 6} Y (r, g) = r g What is the event for P(Y = 1) in terms of ω Ω? Range of a random variable Set of all possible values Distribution of a random variable Specification of P(X A) for every set A. If X has a countably large range then we can define f (x) = P(X = x) as P(X A) = x A f (x) Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 2 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 3 / 24
Expected Value The expected value of a random variable is defined as follows Discrete Random Variable: E[X ] = all x Continous Random Variable: E[X ] = all x xp(x = x) xp(x = x)dx This is a natural generalization of what we do when deciding if a casino game is fair. Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 4 / 24 Properties of Expected Value Constants - E(c) = c if c is constant Indicators - E(I A ) = P(A) where I A is an indicator function { all x g(x) P(X = x) if discrete Functions - E[g(X )] = g(x) P(X = x) dx if continuous Constant Factors - E(cX ) = ce(x) Addition - E(X + Y ) = E(X ) + E(Y ) x Multiplication - E(XY ) = E(X )E(Y ) if X and Y are independent. Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 5 / 24 Variance Another common property of random variables we are interested in is the Variance which measures the squared deviation from the mean. Properties of Variance What is Var(aX + b) when a and b are constants? Var(X ) = E [ (X E(X )) 2] = E(X µ) 2 One common simplification: Var(X ) = E(X µ) 2 = E(X 2 2µX + µ 2 ) = E(X 2 ) 2µE(X ) + µ 2 = E(X 2 ) µ 2 Standard Deviation: SD(X ) = Var(X ) Which gives us: Var(aX ) = a 2 Var(X ) Var(X + c) = Var(X ) Var(c) = 0 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 6 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 7 / 24
Properties of Variance, cont. Covariance What about when X and Y are not independent? E(XY ) E(X )E(Y ) E(XY ) µ x µ y 0 What about Var(X + Y )? This quantity is known as Covariance, and is roughly speaking a generalization of variance to two variables Cov(X, Y ) = E[(X E(X ))(Y E(Y ))] = E[(X µ x )(Y µ y )] = E[XY + µ x µ y X µ y Y µ x ]) = E(XY ) µ x µ y Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 8 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 9 / 24 Properties of Covariance Properties of Variance, cont. Cov(X, Y ) = E[(X µ x )(Y µ y )] = E(XY ) µ x µ y A general formula for the variance of the linear combination of two random variables: Cov(X, Y ) = 0 if X and Y are independent Cov(X, c) = 0 Cov(X, X ) = Var(X ) Cov(aX, by ) = ab Cov(X, Y ) Cov(X + a, Y + b) = Cov(X, Y ) From which we can see that Var(X + Y ) = Var(X ) + Var(Y ) + Cov(X, Y ) Var(X Y ) = Var(X ) + Var(Y ) Cov(X, Y ) Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 10 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 11 / 24
Properties of Variance, cont. Bernoulli Random Variable For a completely general formula for the variances of a linear combination of n random variables: ( ) Var c i X i = i=1 = i=1 j=1 Cov(c i X i, c j X j ) ci 2 Var(X i ) + i=1 i=1 c i c j Cov(X i, X j ) j=1 i j Let X Bern(p), what is E(X ) and Var(X )? Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 12 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 13 / 24 Binomial Random Variable Let X Binom(n, p), what is E(X ) and Var(X )? We can redefine X = n i=1 Y i where Y 1,, Y n Bern(p), and since we are sampling with replacement all Y i and Y j are independent. Hypergeometric Random Variable - E(X ) Lets consider a simple case where we have an urn with m black marbles and N m white marbles. Let B i be an indicator variable for the ith marble being black. { 1 if ith draw is black B i = 0 otherwise In the case where N = 2 and m = 1 what is P(B i ) = 1 for all i? Ω = {BW, WB} P(B 1 ) = 1/2, P(B 2 ) = 1/2 P(W 1 ) = 1/2, P(W 2 ) = 1/2 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 14 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 15 / 24
Hypergeometric Random Variable - E(X ) - cont. Hypergeometric Random Variable - E(X ) - cont. What about when N = 3 and m = 1? Ω = {BW 1 W 2, BW 2 W 1, W 1 BW 2, W 2 BW 1, W 1 W 2 B, W 2 W 1 B} P(B 1 ) = 1/3, P(B 2 ) = 1/3, P(B 3 ) = 1/3 P(W 1 ) = 2/3, P(W 2 ) = 2/3, P(W 3 ) = 2/3 Let X Hypergeo(N, m, n) then X = B 1 + B 2 + + B n Proposition P(B i = 1) = m/n for all i Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 16 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 17 / 24 Hypergeometric Random Variable - E(X ) - 2nd way Hypergeometric Random Variable - Var(X ) Let X Hypergeo(N, m, n), what is E(X )? Let X Hypergeo(N, m, n), what is Var(X )? Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 18 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 19 / 24
Hypergeometric Random Variable - Variance, cont. Hypergeometric Random Variable - Variance, cont. Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 20 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 21 / 24 Poisson Random Variable - E(X ) Poisson Random Variable - Var(X ) Let X Poisson(λ), what is E(X )? Let X Poisson(λ), waht is Var(X )? Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 22 / 24 Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 23 / 24
St. Petersburg Lottery We start with $1 on the table and a coin. At each step: Toss the coin; if it shows Heads, take the money. If it shows Tails, I double the money on the table. Let X be the amount you win, what is E(X )? Sta230/Mth230 (Colin Rundel) Lecture 6 February 5, 2014 24 / 24