Topic 9: Canonical Correlation

Size: px

Start display at page:

Download "Topic 9: Canonical Correlation"

Georgia Jones
5 years ago
Views:

1 Topic 9: Canonical Correlation Ying Li Stockholm University October 22, /19

2 Basic Concepts Objectives In canonical correlation analysis, we examine the linear relationship between a set of X variables and a set of more then one Y variables. 2/19

3 Basic Concepts Definition The canonical correlation technique is to find several linear combinations of X variables and the same number of linear combination of Y variables in such as these linear combination best express the correlation between the two sets. 3/19

4 Basic Concepts Definition The canonical correlation technique is to find several linear combinations of X variables and the same number of linear combination of Y variables in such as these linear combination best express the correlation between the two sets. The linear combinations are called the canonical variables. The correlation between the corresponding pairs of canonical variables are called canonical correlation. 3/19

5 Analytical Approach: Suppose we desire to examine the relationship between a set of variables x 1, x 2,, x p and another set y 1, y 2,, y q. And the sample means for all x and y variables are zero. The first step in canonical correlation is to form two linear combination: W 1 = a 11 x 1 + a 12 x a 1p x p V 1 = b 11 y 1 + b 12 y b 1q y q, such that corr(w 1, V 1 ) = C 1 is maximum. 4/19

6 Analytical Approach: Then the second step is to identify another set of canonical variables W 2 = a 21 x 1 + a 22 x a 2p x p V 2 = b 21 y 1 + b 22 y b 2q y q, such that corr(w 2, V 2 ) = C 2 is maximum and corr(w 1, W 2 ) = 0, corr(v 1, V 2 ) = 0. 5/19

7 Analytical Approach: Then the second step is to identify another set of canonical variables W 2 = a 21 x 1 + a 22 x a 2p x p V 2 = b 21 y 1 + b 22 y b 2q y q, such that corr(w 2, V 2 ) = C 2 is maximum and corr(w 1, W 2 ) = 0, corr(v 1, V 2 ) = 0. This procedure continues. In total, no more than min(p, q) canonical variable sets can be identified. 5/19

8 Data A depress study on 294 observations. n = 294 Dep. variables: y 1 = CESD: an index of depression,0-60, high score indicates likelihood of depression y 2 = health: rating score, 1-4, high score indicates poor health Indep. variables: x 1 = sex: 0 male, 1 female x 2 = age: age in years x 3 = educat: 1-7, high score indicates high education x 4 = income: thousands of dollars per year. 6/19

9 Data Figure: Summary statistics 7/19

10 Data Figure: Correlation matrix 8/19

11 Interpret the result Canonical correlation Figure: Canonical correlation 9/19

12 Interpret the result Test of hypothesis H 0 : C 1 = C 2 = = C k = 0 Two tests: Bartlett s chi-square test An approximate F test 10/19

13 Interpret the result Test of hypothesis H 0 : C 1 = C 2 = = C k = 0 Two tests: Bartlett s chi-square test An approximate F test A large chi-square or a large F are indication that not all the correlations are equal to 0. At least the largest canonical correlation is not 0. 10/19

14 Interpret the result Test of hypothesis H 0 : C 1 = C 2 = = C k = 0 Two tests: Bartlett s chi-square test An approximate F test A large chi-square or a large F are indication that not all the correlations are equal to 0. At least the largest canonical correlation is not 0. It quite possible the remaining k 1 may be not stat. sign.. 10/19

15 Interpret the result Test of hypothesis H 0 : C 1 = C 2 = = C k = 0 Two tests: Bartlett s chi-square test An approximate F test A large chi-square or a large F are indication that not all the correlations are equal to 0. At least the largest canonical correlation is not 0. It quite possible the remaining k 1 may be not stat. sign.. H 0 : C 2 = = C k = 0 10/19

16 Interpret the result Test of hypothesis Example 11/19

17 Interpret the result Interpretation of the Canonical Variables Standardized coefficients Coefficients a 11 = 0.051(sex) a 12 = 0.048(age) a 13 = 0.29(educat) a 14 = 0.005(income) b 11 = 0.055(CESD) b 12 = 1.17(health) Standardized coefficients a 11 = 0.025(sex) a 12 = 0.871(age) a 13 = 0.383(educat) a 14 = 0.082(income) b 11 = 0.490(CESD) b 12 = 0.982(health) 12/19

18 Interpret the result Interpretation of the Canonical Variables 13/19

19 Interpret the result Interpretation of the Canonical Variables Loadings Canonical loadings(canonical structural coefficients) loadings : corr(x i, w j ), corr(y i, v j ) when the set of variables are uncorrelated, loading= std. coefficients. when the set of variables are correlated, loading and std. coefficients can be quite different. It s simpler to try to interpret the loadings rather than coefficients. 14/19

20 Interpret the result Interpretation of the Canonical Variables Figure: Correlation matrix 15/19

21 Interpret the result Interpretation of the Canonical Variables 16/19

22 Interpret the result Redundancy Analysis Redundancy measure(rm) is to determine how much of the variance accounted for in one set of variables by other set of variables. Average amount variance in Y variables that is accounted by V i : q i=1 AV (Y V i ) = loadings2 y i q RM vi w i = AV (Y V i ) C 2 i eg: AV (Y V 1 ) = ( 0.282) r 2 = /19

23 Relations Most of dependence methods are special cases of canonical correlation. only one response: multiple regression only one dummy variable as response:two-group discriminant several dummy variables as responses: multi-group discriminats only one response and dummy variables as indep: ANOVA several responses and dummy variables as indep: MANOVA 18/19

24 Relations Concluding remarks If the sample size is large enough, it is advisable to split it, run a canonical analysis on both halves, and compare results to see if they are similar. Tests of hypothesis regarding canonical correlation assume that joint distribution is multivariate normal. This assumption should be checked if such tests are to be reported. Canonical correlation analysis is one of the less commonly used multivariate techniques. Its limited use may be due, in part, to the difficulty often encountered in trying to interpret the results. 19/19

MS-E2112 Multivariate Statistical Analysis (5cr) Lecture 8: Canonical Correlation Analysis

MS-E2112 Multivariate Statistical (5cr) Lecture 8: Contents Canonical correlation analysis involves partition of variables into two vectors x and y. The aim is to find linear combinations α T x and β