Final Review: Problem Solving Strategies for Stat 430

Transcription

1 Final Review: Problem Solving Strategies for Stat 430 Hyunseung Kang December 14, 011 This document covers the material from the last 1/3 of the class. It s not comprehensive nor is it complete (because it s 4:00AM and I have to go to bed...), but I hope it s helpful. Disclaimer: I apologize for any mistakes/typos I make in advance. Dec. 14, 011: Typos fixed(variance of sum of indicators and question a from section.), added one more problem to question in section 1., and added question 3 to section. 1 Expectation, Variance, and Covariance 1.1 Fundamentals 1. For any continuous random variable, X R, the expectation of a function of X, g(x), is E(g(X)) = g(x)f X (x)dx This also works when we have more than one random variables. For example, if we have two random variables, (X, Y ), then the expectation of a function of (X, Y ), g(x, Y ) R is E(g(X, Y )) = g(x, Y )g(x, y)dxdy. For any random variables that take on values X = {1, 0}, E(X) = P (X = 1). These variables are called indicator random variables 3. For any random variable X and a, b R, E(aX + b) = ae(x) + b 4. For any random variable X, V ar(x) = E(X ) E(X) 5. For any random variables X and Y, covariance is defined as Cov(X, Y ) = E(XY ) E(X)E(Y ) 6. For any random variables X and Y, correlation is defined as Corr(X, Y ) = 7. For any random variable U, V, X, Y along with a, b, c, d R Cov(X,Y ) V ar(x)v ar(y ) Cov(aU + bv, cx + dy ) = accov(u, X) + adcov(u, Y ) + bccov(v, X) + bdcov(v, Y ) 8. For any random variables X and Y, V ar(x) = E(V ar(x Y )) + V ar(e(x Y )) 9. For any sequence of random variables, X i, i = 1,..., n, E( X i) = E(X i) 10. For any sequence of independent random variables, X i, i = 1,..., n, V ar( X i) = V ar(x i) 1

2 11. For any sequence of indicator random variables, I i, i = 1,..., n V ar( I i ) = P (I i = 1) + i,j P (I i = 1, I j = 1) ( P (I i = 1)) where there are n(n 1) terms in the last summation. 1. Practice Problems 1. Suppose you flip a fair coin N times where N P ois(1). Further suppose that the result of the coin flips are independent to N. (a) Calculate the expected number of heads. Proof. Realize that N X i denote the number of heads for this process. N N E( X i ) = E(E( X i ) N) = E(N 1 ) = 1 (b) What is the probability that you will see at least one head? Proof. Given N = n, the probability of seeing at least one head, { N X i 1}, follows a binomial distribution with parameters (n, 1 ). Specifically, P ( N X i 1 N = n) = 1 ( ) 1 n N P ( X i 1) = P ( X i 1 N = n)p (N = n) = n=0 n=0 ( 1 ( ) 1 n ) 1 n! e 1 = 1 e 1/. Suppose N flirty boys and M flirty girls decide to play spin the bottle. For those who are not aware, spin the bottle is basically a game where individuals sit around in a circle. At the beginning, an individual is randomly chosen to spin a bottle that s located in the middle. When the bottle stops spinning, the bottle will point to another individual and the spinner and the individual which the bottle points to get to spend time together. The game continues with the individual who was chosen by the bottle. Assume that the bottle chooses each individual with equal probability and that the spins are independent. (a) After the first spin, what is the probability that the bottle did not choose the same gender? Proof. If a gir spin first, P (not same gender girl) = M N+M. Thus, N N+M. If a boy spin first, P (not same gender boy) = P (not same gender) = P (not same gender girl)p (girl) + P (not same gender boy)p (boy) ( ) ( ) ( ) ( ) N M M N = + = NM N + M N + M N + M N + M (N + M)

3 (b) After K rounds, what is the expected number of times that the bottle did not choose the same gender? What is its variance? Proof. Let I i = 1 when the bottle did not choose the same gender on the ith round and I i = 0 otherwise. From (a), we know P (I i = 1) = NM. Thus, (N+M) K E( I i ) = K P (I i = 1) = KNM (N + M) The variance can be computed by using the formula we have above on indicator random variables. K V ar( I i ) = KNM (N + M) + i j = KNM (K)(K 1) (N + M) ( ) NM NM KNM (N + M) (N + M) (N + M) ( ) NM (N + M) ( KNM (N + M) ) 3. Suppose you and your friend decide to throw coins. You throw a coin that is slightly biased towards heads (i.e. p > 1 ) while your friend throws a coin that is fair (i.e. p = 1 ). Assume both of you are throwing this coin independently of each other and that each toss is independent. (a) What is the expected number of times you and your friends throw your respective coins until both of you match (e.g. on the last toss, you and your friend both obtained heads or tails)? Proof. You and your friend obtain the same result on theith flip with probability p 1 + (1 p) 1 = 1. Let N be the first time you and your friend have the same result from the flip. Note that N Geo( 1 ) and thus, E(N) =. Notice that this DOES NOT depend on p, which is somewhat counter-intuitive! (b) What is the expected number of times you and your friends throw your respective coins until both of you match twice in a row? Proof. Let N be the consecutive two matches and N 1 be the first time til the first match between you and your friend. Then E(N ) = E(E(N N 1 )) = E((N 1 + 1)( 1 ) + (N E(N )) 1 ) = E(N 1) E(N ) 1 Using the answer from (a), we can solve for E(N ) and obtain E(N ) = 6 Bivariate Normals Unfortunately, because linear algebra wasn t a prerequisite for this class, there are lots of highschool algebra associated with defining multivariate normals. 3

4 .1 Fundamentals 1. Continuous random variables (X, Y ) follow a standard bivariate normal (SBN) with correlation ρ if and only if Y can be written as where X and Z are iid standard normals. Y = ρx + 1 ρ Z. X and Y are independent iff they are uncorrelated (aka ρ = 0) 3. If (X, Y ) follow SBN, (Y, X) follow SBN. Here, X can be written as X = ρy + 1 ρ Z where Y and Z are iid standard normals 4. If (X, Y ) follow SBN, then the marginal distributions of X and Y are standard normal 5. If (X, Y ) follow SBN, then the conditional distribution of Y given X is Y X = x N(ρx, 1 ρ ) 6. Continuous variables (X, Y ) follow a bivariate normal (BN), written as ( ) (( ) ( )) X µx σ N, X ρσ X σ Y Y µ Y ρσ X σ Y iff the standardized variables, X = X µ X σ X σ Y and Y = Y µ Y σ Y follow a SBN with correlation ρ. 7. If (X, Y ) follow BN and (X, Y ) are its SBN counterparts, corr(x, Y ) = corr(x, Y ) = ρ. Practice Problems When you are faced with bivariate normal type of problems, NEVER try to use double integrals or what-have-you to obtain the desired answer. It s too hard and you ll mess up along the way. Instead, use the properties above to get all of your desired quantities. 1. Let X, Y iid N(0, 1). Calculate P ( Y X < 1) Proof. P ( Y X < 1) = P (Y < X, X > 0) + P (Y > X, X < 0) = P (Y < X X > 0)P (X > 0) + P (Y > X X < 0)P (X < 0) = 1 [P (Y < 0) + P (0 < Y < X X > 0)] + 1 [P (Y > 0) + P (X < Y < 0 X < 0)] = 1 [ ] + 1 [1 + 1 = 1 [3 4 ] + 1 [3 4 ] = ] To go from P (Y < X X > 0) to P (Y < 0) + P (0 < Y < X), realize that {Y < X X > 0} is only true if either Y is always negative (aka Y < 0)or Y is positive, but less than X (aka 0 < Y < X). Next, to go from P (0 < Y < X X > 0) to 1 1, notice that the probability that Y is positive is 1 (and thus, we satisfied 0 < Y condition).next, to satisfy the {Y < X} condition, note that the probabiltiy that Y is less than X is 1 since they are iid standard 4

5 normals; more precisely, {Y < X} = {Y X < 0} and since Y X N(0, ), P (Y X < 0) = 1. Combining these together gives us 1 1. You can repeat the analysis for the X < 0 case.. Suppose we have two sequences of random variables (Y 1,.., Y n ) and (X 1,..., X n ). We believe there is a linear relationship between each Y i and X i and postulate the following model. Y i = X i β + Z i Here, X i iid N(0, 1) and Zi iid N(0, 1). Assume Zi is independent of X i. (a) What is the distribution of Y i? Proof. By using the property of bivariate normals, we know that Y i N(0, β + 1) (b) What is the distribution of Y i given X i = x? Proof. Again, by the property of bivariate normals, Y i N(X i β, 1). (c) What is Cov(Y i, X i )? Proof. Cov(Y i, X i ) = Cov(X i β + Z i, X i ) = βcov(x i, X i ) = β (d) Suppose we want to find the β that minimizes the following criterion n (Y i X i β) What is the expectation of this minmizing β? Proof. From calculus, we know that the we can find the minimum of this function by taking the derivative d n (Y i X i β) dβ = 1 1 n (Y (Y i X i β)( X i ) = 0 i X i β) Y i X i = β Y ix i X i = β We can check the second deriative to make sure that this is a minimizer. But, you really don t have to since square root functions are convex and the sum inside the square root is also convex. Thus = β is the point where (Y i X i β) is minimzed. Y ix i X i X i 5

6 Now, to find the expectation, let s first compute (using the law of total expectation where we condition on all the X i s) E( X i Y i X i X 1,..., X n ) = X i X i E(Y i X i ) = X i X i X i β = β X i n X i Then, E( Y ix i n X 1,..., X n ) = β X i n X i X i = β Thus, E( X i Y i X i ) = E(E( X i Y i X i X 1,..., X n )) = E(β) = β 3. Suppose you have (X, Y ) follow a bivariate normal distribution with µ x = µ y = 0, σ x = σ y = σ and Corr(X, Y ) = ρ (a) Find a, b R where V ar(ax + by ) a + b is maximized. For this problem, assume ρ = 1 Proof. First, we compute the variance expression by computing each expectation components. In particular, E(aX + by ) = 0 for any a, b and E((aX + by ) ) = E(a X + b Y + abxy ) = σ (a + b ) abσ. Therefore, V ar(ax + by ) = σ (a 1 b) We just have to find values a, b where (a 1 b). With elementary calculus (or by simple a +b polynomial tricks), we realize that for any values of a, b where a = b, the function is maxmized at 3. 3 Chains and First-Step Analysis Generally speaking, these type of problems require you to look only one step ahead so that you can get some sort of a recursive relation out. I think the practice problems do an excellent job demonstratnig the techniques involved in problems like these (actually, to be honest, I m tired and I need to go to bed...). But, section 1..3.b of this sheet provides on example. 4 Must-Know Univariate Distributions There are TONS of univariate distributions you should know. I suggest you check out Wikipedia for details. However, I ll highlight a couple. 1. X is a geometric random variable iff P (X = x) = (1 p) x 1 p, x = 1,..., where p is the probability of success for a sequence of iid trials (e.g. coin toss). E(X) = 1 1 p p and V ar(x) = p 6

7 . X is a hypergeometric random variable iff P (X = x) = (m x)( N m n x ) with mean ( N n) nm N. This random variable describes the process where you pick n balls from an urn containing N balls and X denote the number of white balls you pick from a set of m inside the urn. 7