Correspondence analysis and related methods

Transcription

1 Michael Greenacre Universitat Pompeu Fabra Correspondence analysis and related methods Middle East Technical University Fall semester A A A A A5 B B B B B5 C C C C C5 D D D D D5 E E E E E5 F F F F F5 G G G G G5 H H H H,5. (.6%). (8.%) mmr.upf@gmail.com Notice address for course ADDITIONAL READING Gabriel, K.R. (). The biplot graphic display of matrices with application to principal component analysis. Biometrika 58, 5 6. Bradu, D. and Gabriel, K.R. (8). The biplot as a diagnostic tool for models of two-way tables. Technometrics, 68. Gower, J. and Hand, D. (6). Biplots. Chapman & Hall, London. Greenacre, M. and Blasius, J. (6). Multiple Correspondence Analysis and Related Methods. Chapman & Hall /CRC Press. Greenacre, M. (). Correspondence Analysis in Practice, nd edition. Chapman & Hall/ CRC Press. (Spanish edition available for free download from Nenadić, O. and Greenacre, M. (). Correspondence analysis in R, with two- and three-dimensional graphics: The ca package. Journal of Statistical Software. Download from Hastie, T., Tibshirani, R. and Friedman, J. (). The Elements of Statistical Learning (nd edition). Springer, New York. This book may be freely downloaded at Greenacre, M. (). Biplots in Practice. BBVA Foundation, Madrid.

2 Website of book Biplots in Practice published by the BBVA Foundation (Madrid) in September and available online for free download. Includes R code for all the analyses in the book, as well as data sets, videos, a searchable glossary in English and Spanish, and chapter summaries in Spanish. IMPORTANT: Homework during the Bayram holiday: Read and study Chapters - of this book Biplot: the fundamental concept. Generalized scatterplot y x x x x 5 x x scatterplot biplot

3 Some multivariate data Let s start with some simple trivariate data... Continuous variables X Purchasing power/capita (euros) X GDP/capita (index) X inflation rate (%) Country X X X Be Belgium 5..5 De Denmark..6 Ge Germany Gr Greece 88.. Sp Spain 6.6. Fr France Ir Ireland It Italy Lu Luxembourg Ne Netherlands.. Po Portugal UK United Kingdom Education Some primary Count variables C Glance reader C Fairly thorough reader C Very thorough reader Primary completed Some secondary Secondary completed Some tertiary E E E E E5 C 5 8 C C 6 6 Visualizing trivariate continuous data 5.5 Be Sp Gr Lu Continuous variables X Purchasing power/capita (euros) X GDP/capita (index) X inflation rate (%) Inflation Rate.5.5 Po It De Fr Ir Ge Ne UK Country X X X Be Belgium 5..5 De Denmark..6 Ge Germany Gr Greece 88.. Sp Spain 6.6. Fr France Ir Ireland It Italy Lu Luxembourg Ne Netherlands.. Po Portugal UK United Kingdom GDP per Capita (indexed) Purchasing Power per Capita Lu 5 5 Ne Ir Be Ge De UK Sp It Fr Gr Po Purchasing Power per Capita

4 Visualizing trivariate continuous data Continuous variables X Purchasing power/capita (euros) X GDP/capita (index) X inflation rate (%) Country X X X Be Belgium 5..5 De Denmark..6 Ge Germany Gr Greece 88.. Sp Spain 6.6. Fr France Ir Ireland It Italy Lu Luxembourg Ne Netherlands.. Po Portugal UK United Kingdom Visualizing trivariate continuous data Continuous variables X Purchasing power/capita (euros) X GDP/capita (index) X inflation rate (%) Country X X X Be Belgium 5..5 De Denmark..6 Ge Germany Gr Greece 88.. Sp Spain 6.6. Fr France Ir Ireland It Italy Lu Luxembourg Ne Netherlands.. Po Portugal UK United Kingdom

5 Visualizing trivariate continuous data Continuous variables X Purchasing power/capita (euros) X GDP/capita (index) X inflation rate (%) GDP PP Country X X X Be Belgium 5..5 De Denmark..6 Ge Germany Gr Greece 88.. Sp Spain 6.6. Fr France Ir Ireland It Italy Lu Luxembourg Ne Netherlands.. Po Portugal UK United Kingdom INF Visualizing trivariate continuous data Continuous variables X Purchasing power/capita (euros) X GDP/capita (index) X inflation rate (%) Country X X X Be Belgium 5..5 De Denmark..6 Ge Germany Gr Greece 88.. Sp Spain 6.6. Fr France Ir Ireland It Italy Lu Luxembourg Ne Netherlands.. Po Portugal UK United Kingdom cor X X X X... X... X... GDP INF PP This is almost a biplot!

6 Visualizing trivariate count data Count variables C Glance reader C Fairly thorough reader C Very thorough reader row profiles Education Primary incomplete E C 5 C C E C C C.6.5. Primary completed E E..55. Secondary incomplete E 8 E...5 Secondary completed E E... Some tertiary E5 6 6 E5...6 Visualizing trivariate count data Count variables C Glance reader C Fairly thorough reader C Very thorough reader row profiles Education C C C Some primary E.6.5. Primary completed E..55. Some secondary E...5 Secondary completed E... Some tertiary E5...6 C C C

7 Visualizing trivariate count data Count variables C Glance reader C Fairly thorough reader C Very thorough reader row profiles Education C C C Some primary E.6.5. Primary completed E..55. Some secondary E...5 This is almost a correspondence analysis! Secondary completed Some tertiary E E And almost a correspondence analysis biplot if vectors drawn to the corners! C C C The biplot: The result of a simple matrix decomposition = 6 6 target matrix = left matrix right matrix S = AB T (Note: we say that the target matrix is of rank )

8 The biplot: A graphical display of a matrix decomposition y y y y 8 6 x 5 x = x x 6 6 x 5 y x x x y target matrix = left matrix right matrix y x y = x = + = 8 T y x 5 (scalar product between two vectors) Cases usually points. Variables usually vectors Geometry of scalar products x x y origin xcos y x T y = x y cos( ) = x cos( ) y = projection of x onto y length of y

9 The biplot: A graphical display of a matrix decomposition y y y y 8 6 x 5 x = x x 6 6 x 5 y x x x y target matrix = left matrix right matrix = x = + = 8 T y (scalar product between two vectors) y x y x 5 Projection of x onto y =.5 (angle between them is 6.5 ) Length of y = = = 8. /.6.6 = Calibration unit is /.6 =.6 (/length of biplot vector) Calibrated biplot y x x 5 x y 5 y x y N.B. It s important to draw a biplot with aspect ratio equal to ( asp= option in R) x = 6 6

10 Regression biplots: Data set bioenv SITE SPECIES COUNTS ENVIRONMENTAL VARS. NO. a b c d e Pollution Pollution Depth Depth Temp. Temp. Sedmt Sedmt s s S s s C s s C s s S s5 s C s6 s G s s S s8 s C s s C s s S s s C s s S s s S s s C s5 s C s6 s G s s C s8 s G s s S s s S s s C s s S s s G s s C s5 s G s6 s S s s G s8 s G s s S s s G d y (pollution) Simple linear regression species d versus pollution (y) species d dˆ.. 85y d =..85y (R =.) y (pollution)

11 d y x Multiple linear regression dˆ = y+ y.8x x x d Regression model is a (hyper)plane R =. y d * Multiple linear regression, variables standardized y* x* d* dˆ * =.6.6y* y * +.x* x* d* R =. Explanatory variables x and y and response variable d standardized y*

12 d y x Another geometry of regression & prediction Standardized Pollution (Y* ) dˆ *.6y *.x* Standardized Depth (X* ) variance explained (R ):.% d 5 y* x* (pollution) y* - Regression biplot e x* a c 6 b d 6 (depth) variance explained: a: 5.% b:.% c:.8% d:.% e:.5% overall:.5% significance: y x a: ** b: ** c: * d: ** * e: * * **=p<. *=p<.5

13 Regression is a matrix decomposition For d, regression model was: d* ˆ* =.6y* y * +.x* For all five variables, the five regression models can be written as: aˆ * bˆ* cˆ* dˆ * eˆ* y * x* Ŝ Target matrix: the predicted values from the regression models = UB T. Left matrix: the explanatory variables (fixed!) [ a* b* c* d* e* ] S = UB T + E B T = (U T U) U T S. Right matrix: the regression coefficients... Classic notation: y = X + e ^ =(X T X) X T y ^y = X ^ What happens for three predictors? Each regression model can be represented as a point vector in three-dimensional space. Reconstruct the data from projections of cases onto variable directions, but only as well as measured by R ; in this example the increase in explained variance from two-dimensional to threedimensional (adding temperature as an explanatory variable) is from.5% to.%, hence temperature is explaining very little extra variance. There will be a particular orientation of the vectors that gives maximum variance explained in the two-dimensional projection... Dimension reduction coming...

14 Generalized linear model biplots e.g., logistic regression link function logit(p d ) = ( y) x x e residuals with probability distribution SITE SPECIES COUNTS NO. a b c d e s s 6 s 8 s 5 s5 5 transform to s6 6 5 presence/absence s 6 s8 (/) data s 6 s 5 6 s s 5 s 6 : : : : : : : : : : : pd log =.. y*. x* p d y* (pollution) s s s s s s s8 s s s s5 e s s s.8 s5 s s s s8 p d =.8 s.5 s8 s s s6 s6 s6 s s s5 s d b c a Logistic regression biplot pd logit(p d ) = log p d =.. y*. x* error deviances: a:.6 b:.56 c:. d:.8 e: x* (depth) Similarly for Poisson regression and any generalized linear model