Math 180B, Winter Notes on covariance and the bivariate normal distribution

Transcription

1 Math 180B Winter 015 Notes on covariance and the bivariate normal distribution 1 Covariance If and are random variables with finite variances then their covariance is the quantity 11 Cov := E[ µ ] where µ = E[] and = E[ ] The covariance is a measure of the extent to which and are linearly related Because µ = ν + µ the covariance can also be expressed as 1 Cov = E[ ] E[] E[ ] = E[ ] µ Observe that if and are independent then E[ ] = µ Therefore 13 = Cov = 0 The converse implication fails as a general statement by an example discussed in class [ uniformly distributed on 1 1 = ] But see Corollar 3 below Variance Part of the importance of covariance is the way in which it completes the addition formula for the variance of a sum of random variable: 1 Var[ + ] = Var[] + Var[ ] + Cov More generally because we evidently have Covα β = αβcov for reals α and β it is also true that 3 Var[α + β ] = α Var[] + β Var[ ] + αβcov 3 Correlation The correlation of two random variables and is their standardized covariance: 31 Cov ρ = ρ = corr : = Var[]Var[ ] [ ] µ = E

2 4 Example Suppose = α + β where α and β are real constants Then = αµ + β Var[ ] = α Var[] and Cov = αvar[] Consequently 41 corr = { 1 if α > 0; 0 if α = 0; 1 if α < 0 when is a non-random straight-line function of 5 Theorem [Cauchy-Schwarz Inequality] 51 Cov and 5 corr 1 Assuming that > 0 equality holds in either of 51 and 5 only if P[ µ / = signcorr / ] = 1 Proof The inequalities 51 and 5 are equivalent so it is enough to demonstrate 5 which in view of the third equality in 31 can be written as 53 Cov ˆ Ŷ 1 where ˆ = µ / and Ŷ = / To see 53 we consider the function [ gt := E ˆ tŷ ] t R Being the expectation of a square gt 0 for all real t quadratic: gt = E[ ˆ ] te[ ˆŶ ] + t E[Ŷ ] = 1 tcov ˆ Ŷ + t On the other hand g is a The only way a quadratic function can take only non-negative values is if its discriminant is non-positive Thus we must have [ Cov ˆ Ŷ ] That is 4[Cov ˆ Ŷ ] 4 0 or what is the same 54 [Cov ˆ Ŷ ] 1

3 which is equivalent to 53 Suppose now that corr = 1 In this case we have gt = 1 t so g1 = 0 But g1 = E [ ˆ ] Ŷ and the only way this the expectation of a non-negative random variable can be 0 is if that random variable is itself 0 with probability 1 This shows that if corr = 1 then P[ ˆ = Ŷ ] = 1 as required by the final sentence of the Theorem Similar considerations apply when corr = 1 6 Bivariate Normal Distribution A pair of random variables and is said to have the bivariate normal distribution provided 61 α + β has the univariate normal distribution for each pair α β of real numbers This state of affairs will be indicated by the notation or by 6 µ if I wish to indicate the means µ the variances and the covariance of and Observe that if 6 holds then 1 that is has a univariate normal distribution take α = 1 and β = 0 and also 1 In what follows I will write 63 ρ = for the correlation of and The column vector µ appearing in 6 is the mean vector for the pair t while the matrix = ρ ρ is its variance-covariance matrix The standard bivariate normal distribution is the special case in which the means are both 0 and the variances are both 1 7 Representation The following construction of a standard bivariate normal pair in terms of iid univariate normals is useful for various calculations Let and Z be

4 independent standard normal random variables Then for real constants α and β the random variables α and β are also independent and so their sum α + β is also normally distributed as discussed in class It follows from the discussion in 6 that and Z have a bivariate normal distribution More precisely Z the standard bivariate normal distribution Now fix ρ [ 1 1] and consider the random variable defined by 7 = ρ + 1 ρ Z It is easy to check that E[ ] = 0 Var[ ] = 1 and Cov = ρ Also if α and β are any two real numbers then α + β = α + βρ + β 1 ρ Z is normally distributed because and Z have a bivariate normal distribution as remarked above It follows that and themselves have a bivariate normal distribution More precisely ρ ρ 1 8 Corollary 1 With and as in 73 the conditional distribution of given that takes the value x is normal with mean ρx and variance 1 ρ In symbols 81 = x 1 ρx 1 ρ Proof Look at 7: If I tell you that equals x this has no effect on the independent random variable Z which still has the standard normal distribution Thus under the condition = x the random variable is Z scaled by a factor of 1 ρ which changes its variance to 1 ρ and then translated by ρx which changes its mean to ρx Neither the scaling nor the translation alter the fact that the random variable is normally distributed 9 Corollary Suppose that µ 91 ρ ρ

5 Then the conditional distribution of given that takes the value x is normal: 9 = x + ρ x µ 1 ρ Proof Just apply Corollary 1 to the standardized random variables ˆ = µ / and Ŷ = / 10 Corollary 3 Suppose that µ ρ ρ Then and are independent if and only if ρ = 0 Proof We need only consider the if part of this assertion the only if part holds for any two random variables bivariate normal or not If ρ = 0 then according to 9 the conditional density of given the value of namely f y x does not depend on x and so is a function of y alone call it gy This means that the joint density fx y of and factors: 101 fx y = f x f y x = f x gy Now integrate out x: 10 f y = Using 10 in 101 we find that fx y dx = which proves to the asserted independence fx y = f x f y f x gy dx = gy 11 References The material discussed above can be found in section 73 and 78 of A First Course in Probability by Sheldon Ross you may own this text if you took Math 180A from Professor Schweinsberg and also in sections 64 and 64 of PROBABILIT by Jim Pitman The latter text is available in pdf form for UCSD students at