Distributions of linear combinations

Transcription

1 Distributions of linear combinations CE 311S

2 MORE THAN TWO RANDOM VARIABLES

3 The same concepts used for two random variables can be applied to three or more random variables, but they are harder to visualize (triple integrals, triple sums, etc.) One common thing to do with multiple random variables is to calculate linear combinations of them. More than two random variables

4 DISTRIBUTIONS OF LINEAR COMBINATIONS

5 A linear combination of the random variables X 1,..., X n has the form a 1 X 1 + a 2 X a n X n That is, we multiply each random variable by a constant coefficient, and add them up. Examples: X 1 + X 2 ; X 1 X 2 ; 5X X 2 + 3X 3 Distributions of Linear Combinations

6 To calculate the total of n random variables, we have a linear combination with a 1 = a 2 = = a n = 1 To calculate the difference between 2 random variables, we have a linear combination with a 1 = 1 and a 2 = 1 I want to calculate the toll revenue on SH-130 today. If X 1 is the number of cars and X 2 the number of semi trucks, the revenue is a 1 X 1 + a 2 X 2 where a 1 and a 2 are the toll charged to each car and truck. Distributions of Linear Combinations

7 We have the following formulas: For any random variables X 1,..., X n E[a 1 X 1 + a 2 X 2 + a n X n ] = a 1 E[X 1 ] + a 2 E[X 2 ] + + a n E[X n ] V [a 1 X 1 + a 2 X 2 + a n X n ] = n n a i a j Cov(X i, X j ) i=1 j=1 If X 1,..., X n are independent, the formula for variance simplifies to V [a 1 X 1 + a 2 X 2 + a n X n ] = a 2 1V [X 1 ] + a 2 2V [X 2 ] + + a 2 nv [X n ] Distributions of Linear Combinations

8 Example The toll on SH-130 is $1.50 for cars and $4.50 for trucks. The mean and variance of the number of cars is 15,000 and 250,000; and the mean and variance of the number of trucks is 5,000 and 10,000. The number of cars and trucks is correlated, with covariance 50,000. What is the mean and standard deviation of the daily toll revenue? The toll is 1.50X X 2 where X 1 and X 2 are the number of cars and trucks. E[1.50X X 2 ] = 1.50E[X 1 ] E[X 2 ] = dollars V [1.50X X 2 ] = Cov(X 1, X 1 ) + (1.50)(4.50)Cov(X 1, X 2 ) +(4.50)(1.50)Cov(X 2, X 1 ) Cov(X 2, X 2 ) = 2.25V [X 1 ] Cov(X 1, X 2 ) V [X 2 ] = so the standard deviation is = 1200 dollars Distributions of Linear Combinations

9 Example I run a business where my daily revenue has a mean of 1500 and a standard deviation of 400, while my daily costs have a mean of 1000 and a standard deviation of 300. What is the mean and standard deviation of my daily profit, assuming my daily revenue and costs are independent? Π = R X so E[Π] = E[R] E[X ] = 500 V [R X ] = V [R] + V [X ] = = so σ R X = 500 This is one reason why we use variance even though standard deviation is easier to interpret. Variances add nicely, standard deviations do not. Distributions of Linear Combinations

10 Example I run a business where my daily revenue has a mean of 1500 and a standard deviation of 400, while my daily costs have a mean of 1000 and a standard deviation of 300. What is the mean and standard deviation of my daily profit, assuming my daily revenue and costs have a correlation coefficient of +0.5? Π = R X so E[Π] = E[R] E[X ] = 500 V [R X ] = V [R] + V [X ] 2Cov(R, X ) = (0.5)(300)(400) = so σ R X = 360 If revenue and costs were negatively correlated, would my daily profit have a higher or lower standard deviation? Distributions of Linear Combinations

11 Let s try to derive these formulas with n = 2 to keep the numbers manageable: E [a 1 X 1 + a 2 X 2 ] = (a 1 x 1 + a 2 x 2 )p(x 1, x 2 ) x 1 x 2 = a 1 x 1 p(x 1, x 2 ) + a 2 x 2 p(x 1, x 2 ) x 1 x 2 x 1 x 2 = a 1 x 1 p(x 1, x 2 ) + a 2 x 2 p(x 1, x 2 ) x 2 x 2 x 1 x 1 = a 1 E[X 1 ] + a 2 E[X 2 ] Notice that we did not have to assume independence to derive this formula. Distributions of Linear Combinations

12 What about the variance? V [a 1 X 1 + a 2 X 2 ] = E[(a 1 X 1 + a 2 X 2 µ a1 X 1 +a 2 X 2 ) 2 ] = E[(a 1 (X 1 µ 1 ) + a 2 (X 2 µ 2 )) 2 ] = E[a1(X 2 1 µ 1 ) 2 + a 1 a 2 (X 1 µ 1 )(X 2 µ 2 )+ a 2 a 1 (X 2 µ 2 )(X 1 µ 1 ) + a 2 a 2 (X 2 µ 2 )(X 2 µ 2 )] = a 1 a 1 E[(X 1 µ 1 )(X 1 µ 1 )] + a 1 a 2 E[(X 1 µ 1 )(X 2 µ 2 )]+ a 2 a 1 E[(X 2 µ 1 )(X 1 0µ 1 )] + a 2 a 2 E[(X 2 µ 1 )(X 2 µ 1 )] = 2 2 a i a j Cov(X 1, X 2 ) i=1 j=1 Distributions of Linear Combinations

13 If X 1 and X 2 are independent, their covariance is zero, so the formula simplifies to a 2 1Cov(X 1, X 1 ) + a 2 2Cov(X 2, X 2 ) or simply a 2 1 V [X 1] + a 2 2 V [X 2] Distributions of Linear Combinations

14 WHAT ARE STATISTICS?

15 Remember the measures of location and variability from Chapter 1? What was the purpose of these?

16 We wanted to use a single number to describe the data set in some way. (This is the definition of a statistic. In mathematical terms: Consider a sample of n elements, and let X i describe the variable of the i-th member of the sample. A statistic is a random variable Y which is determined from the random variables X 1,..., X n Examples: Sample mean: Y = n i=1 X i/n Maximum value: Y = max n i=1 {X i} Total: Y = n i=1 X i

17 The important thing to notice is that the statistics are random variables themselves. Let s say I roll a die five times, and take the average of the values. 6, 6, 3, 1, , 1, 1, 5, , 6, 3, 2, , 3, 2, 4, , 6, 6, 1, Each sample could have a different mean, so the sample means form a random variable (taking the values 3.4, 2.8, 4.0, 3.4, 3.6, and so on). Can we say anything meaningful about its distribution?

18 Yes, we can. In fact, we will shortly see that if n is large, the sample mean has a normal distribution no matter what the distribution of the X i is. Furthermore, its mean is simply the mean of the individual random variables, and its variance is the variance of the individual random variables, divided by n.

19 To begin, we need to make some assumptions about the X i The random variables X 1,..., X n are a random sample if they are independent and identically distributed. (This is often abbreviated as the iid property.)

20 Assume that X 1,..., X n are a random sample, and let X represent the sample mean: n i=1 X = X i n What is E[X ]? In the dice example above, this is asking what the average is of the average of five dice rolls. This is conceptually different from asking what the average is of each roll of the die, although we might think the answers should be the same.

21 Let s try it with n = 2 to keep the numbers manageable: [ n i=1 E[X ] = E X ] i = E[(X 1 + X 2 )/2] n = x 1 + x 2 p(x 1, x 2 ) 2 x 1 x 2 by independence = x 1 + x 2 p X1 (x 1 )p X2 (x 2 ) 2 x 1 x 2 = x 1 x 2 x 1 2 p X 1 (x 1 )p X2 (x 2 ) + x 2 2 p X 1 (x 1 )p X2 (x 2 ) x 1 x 2

22 = x 1 x 1 2 p X 1 (x 1 ) x 2 p X2 (x 2 ) + x 2 x 2 2 p X 2 (x 2 ) x 1 p X1 (x 1 ) rearranging the sums (Why?) = x 1 x 1 2 p X 1 (x 1 ) x 2 x 2 2 p X 2 (x 2 ) = E[X 1] 2 + E[X 2] 2

23 However, since X 1 and X 2 are identically distributed, they have the same expected value (3.5) which we can write as E[X ]: E[X 1 ] 2 + E[X 2] 2 = E[X ] 2 + E[X ] 2 = E[X ] = 3.5 So, all this is just to show the intuitive result E[X ] = E[X ], that is, that the expected value of the sample mean is just the expected value of each observation.

24 What about the variance? V [X ] = E[(X E[X ]) 2 ] because E[X ] = E[X ]. Writing µ = E[X ] to save space, we have = V [X ] = E = 1 4 [ (X1 + X 2 2 ) ] 2 µ = E = 1 4 E [ {(X 1 µ) + (X 2 µ)} 2] [ (X1 ) ] + X 2 2µ 2 = 1 4 E [ (X 1 µ) 2 + (X 2 µ) 2 + 2(X 1 µ)(x 2 µ) ] ( E[(X1 µ) 2 ] + E[(X 2 µ) 2 ] + 2E[(X 1 µ)(x 2 µ)] ) 2 = V [X ]/2

25 The same results hold even for larger n: If X 1,... X n are a random sample (with each X i having mean µ and variance σ 2, then E[X ] = µ and V [X ] = σ2 n

26 The central limit theorem goes one step further and specifies what type of distribution the sample mean has: Let X 1,..., X n be a random sample. Then if n is sufficiently large, X has approximately a normal distribution, with mean and standard deviation given on the previous slide. This is true no matter what distribution the X i are taken from. As a practical rule of thumb, if n > 30 it is safe to use the Central Limit Theorem.

27 Example I flip a coin 49 times, and calculate the proportion of flips which were heads. What is the probability that this proportion is between 0.49 and 0.51?

28 Visualizing the central limit theorem This is the PMF for a random variable:

29 Visualizing the central limit theorem This is the PMF of the average of two independent draws of the same random variable:

30 Visualizing the central limit theorem This is the PMF of the average of three independent draws of the same random variable:

31 Visualizing the central limit theorem This is the PMF of the average of thirty independent draws of the same random variable:

32 Visualizing the central limit theorem This is the PDF for a random variable:

33 Visualizing the central limit theorem This is the PDF of the average of two independent draws of the same random variable:

34 Visualizing the central limit theorem This is the PDF of the average of three independent draws of the same random variable:

35 Visualizing the central limit theorem This is the PDF of the average of thirty independent draws of the same random variable: