and Statistics 3513 Fall 008 Mike Anderson Abstract and correlation are measures of association; how strongly one random variable is related to another. Page 1 of 8
1. is a measure of association, how much one random variable changes with respect to another random variable. It is defined as Cov(X, Y ) E [(X µ x )(Y µ y )] We often see covariance in the sums and differences of random variables Similarly and in general V [X + Y ] E [ (X + Y µ x µ y ) ] E [ (X µ x ) ] + E [ (Y µ y ) ] + E [(X µ x )(Y µ y )] V [X] + V [Y ] + Cov(X, Y ) V [X Y ] V [X] + V [Y ] Cov(X, Y ) [ ] V X i V [X i ] + Cov(X i, X j ) i i i<j Page of 8
..1. 008 SAT Scores The College Boards publish a lot of data on SAT scores every year, but some obvious statistics are missing. For example, on their website are the means and standard deviations for the various subject scores (math, critical reading, and writing) as well as the composite scores (CR+M, CR+M+W), but nowhere are there measure of association, say between math and critical reading scores. Fortunately, there is enough data to calculate a covariance: subject µ σ M 515 116 CR 50 11 CR+M 11 1017.. Refresher: A Normal Probability σcr+m σcr + σm + Cov(CR, M) Cov(CR, M) 1 ( σ CR+M σcr σm ) Current admission standards at UTSA are such that a student with combined SAT score, CR+M, of 100 or better, is eligible for admission, regardless of high school class standing. What proportion of students who took the 008 SAT exam are eligible for admission to UTSA? Page 3 of 8
3. Variance- Matrix When dealing with multiple random variables, it s convenient to represent variance and covariance as a single mathematical object, the variance-covariance matrix. If we look at the SAT data and let X m represent the math score and X r represent the reading score, then we have E [X] ( E [Xm ] E [X r ] ) ( 50 515 ) ( V [X] 3.1. Using the Matrix: Linear Combinations V [X m ] Cov(X m, X r ) Cov(X m, X r ) V [X r ] ) ( ) 116 960.5 960.5 113 The variance-covariance matrix simplifies variance calculations for linear combinations of random variables. If A is a linear transformation on the random vector X then E [AX] AE [X] V [AX] AV [X] A T The matrix A can be a single linear combination (a row vector), or it can be a set of linear combinations (a matrix). When A is a matrix, the result above will include the covariances between each pair of transformed variables. Consider the sum and difference of two random variables [ ] [ ] ( ) 1 1 X1 X1 + X Y AX 1 1 X X 1 X The variance-covariance matrix for Y is Σ Y AΣ X A T [ 1 1 1 1 ] [ σ 1 σ 1 σ 1 σ ] [ 1 1 1 1 ] Page 4 of 8
3.. The Sum and Difference in SAT Scores Consider the sum and difference of the Math and Critical Reading SAT scores. From our previous result we see that [ σ Σ Y 1 + σ 1 + σ σ1 σ σ1 σ σ1 σ 1 + σ X m + X r N(1017, 11 ) X m X r N(13, σ ) ] Questions: What is σ? What is the probability that a person s math and reading scores differ by more than 00 points? Page 5 of 8
4. 4.1. Definition is just covariance rescaled to the interval (-1, +1): 4.. Examples ρ XY Cov(X, Y ) V [X] V [Y ] From the previous example about SAT scores, we found V [CR] 11 V [M] 116 Cov(CR, M) 960.5 so the correlation is ρ CR,M 960.5 11 116 Page 6 of 8
5. Principal Components (OPTIONAL) Take another look at the variance-covariance matrix for the sum and difference of the math and reading scores: [ ] [ ] σ Σ ± 1 + σ 1 + σ σ1 σ 44, 51 91 σ1 σ σ1 σ 1 + σ 91 7, 479 The covariance is quite small compared to either of the two variances. Might it be possible to find two linear combinations a weighted sum and difference that have zero covariance? 5.1. Rotation Matrices Yes we can. The key is to use an orthonormal transformation, or rotation matrix: [ ] cos θ sin θ R θ sin θ cos θ This is a length-preserving transformation in -D. To get a better idea of what R θ does, answer these Questions: What is R θ? On graph paper, plot the points (column vectors) X and R π/4 X: X [ 1 1 1 1 1 1 1 1 ] Page 7 of 8
5.. Rotating to Zero Now apply the rotation to our known variance-covariance matrix: [ ] [ ] [ R θ ΣR T cos θ sin θ σ θ 1 σ 1 cos θ sin θ sin θ cos θ σ 1 σ sin θ cos θ ] Then find θ such that the covariance term is zero. These identities might be useful sin θ sin θ cos θ cos θ cos θ sin θ Page 8 of 8