FINM 33180/STAT32940: MULTIVARIATE DATA ANALYSIS VIA MATRIX DECOMPOSITIONS

Transcription

1 FINM 33180/STAT32940: MULTIVARIATE DATA ANALYSIS VIA MATRIX DECOMPOSITIONS Overview, Motivation, Background Thanks: Rasmus Bro (Copenhagen), Mei Wang (Chicago)

2 Administrative Matters Course webpage: Two course assistants: Klakow Akepanidtaworn and Jeremy Bejarano Grades: 60% problem sets, 40% exam (roughly) Textbooks: none required, list of references on course webpage (will be expanded as we go along), see also K V Mardia, J T Kent, J M Bibby, Multivariate Analysis, Academic Press, 1980 C D Meyer, Matrix Analysis and Applied Linear Algebra, SIAM, 2001 Office hours: Wed, 1:00 2:30pm, Jones 122C Course assistants office hours: Tue, 3:00 4:30pm (Jeremy), Fri, 4:00 5:30pm (Klakow), Math-Stat Library (or Room 302 if there is an event in the library), Math-Stat Building (Stevanovich Center)

3 Content of this course What is often called Multivariate Analysis or Multivariate Data Analysis in Statistics Also known as Unsupervised Learning in Machine Learning A collection of very useful tools: Principal component analysis Factor analysis Canonical correlation analysis Discriminant analysis Multidimensional scaling Partial least squares Manifold learning Topological data analysis Etc Allows us to analyze all kinds of data: demographic, financial, medical, scientific, social, etc Exploratory data analysis: what you do if you do not already know what to look for Preprocess raw data: outputs of these become inputs to supervised learning algorithms Discover latent variables that generate the observed/measured data (dimension reduction)

4 Where it fits in

5 Objective of this course Understand the principles behind these tools: mathematical statistical computational Know general principles so that you may apply to any data, regardless of origin Word for our finance students Financial applications? You know more than me Here you learn general tools that may not have been used in finance Acquire necessary background knowledge in math/stat on which on these tools are built on Develop confidence and a sense of comfort to use these tools without a second thought

6 Using a single variable is inadequate Caucasian African Asian Korea overlaps with Caucasian Buttock-Knee Length - 300mm North America Latin America (Indian) Latin America (European) Northern Europe Central Europe Eastern Europe South-Eastern Europe France Iberian Peninsula North Africa West Africa South-eastern Africa Near East North India South India North Asia South China South-east Asia Australia Korea/Japan

7 Using two variables starts getting interesting 150 Shoulder Breadth (biacromial) - 300mm Caucasian African Asian Buttock-Knee Length - 300mm North America Latin America (Indian) Latin America (European) Northern Europe Central Europe Eastern Europe South-Eastern Europe France Iberian Peninsula North Africa West Africa South-eastern Africa Near East North India South India North Asia South China South-east Asia Australia Korea/Japan

8 Plot the two variables against each other Co-variation = new information that is not available in the individual variables 380, South-east Africa, North Africa Buttock-Knee Length North India, West Africa Korea/Japan Near East North Asia South China Northern Europe Central Europe North America, Australia France Latin America (European) Eastern Europe South-Eastern Europe Iberian Peninsula 330 South India South-east Asia Latin America (Indian) Shoulder Breadth Lee S,Bro R, Regional Differences in World Human Body Dimensions Theoretical Issues in Ergonomics Science, 2008

9 More generally: Multivariate data analysis 7 variables Austral Austria Barbado Belgium Brit Gu Bulgari Canada Chile Costa R Cyprus Czechos Denmark El Salv Finland France West Ge Yugosla 49 objects x1000 Gunst & Mason (1980) Regression analysis and its applications: A data-oriented approach, NY, Marcel Dekker, p 358

10 But which variable to plot against which variable? 7 variables Austral Austria Barbado Belgium Brit Gu Bulgari Canada Chile Costa R Cyprus Czechos Denmark El Salv Finland France West Ge Yugosla 49 objects Simple questions Which European country is most similar to Japan? Which countries are the most bizarre? x1000

11 Answer: Use Principal Components Outliers are easily spotted Singapore Hong Kong T2 27% Malta Netherl Taiwan Japan Belgium Austral 0 Barbado Puerto West Un Czechos Denmark Switzer King Canada Austria Finland France Cyprus Bulgari Italy Hungary Ireland Iceland Luxembo Norway Sweden Ge New Zea Trinida Spain USSR Mauriti Jamaica Portuga Panama Romania Costa R Poland El Salv Malaya Nicarag Brit Mexico Yugosla Gu Guatema Chile USA India 0 T1 47%

12 Principal Components Analysis (PCA) Biplot: Why outliers are outliers pop per km 2 pop agri land Singapo Hong Ko What variables are correlated? Why is the outlier strange? What variables (eg expensive) can be spared? T2/P2 27% Malta Netherl Taiwan Japan Belgium Austral 0 Barbado Puerto West Un Czechos Denmark Switzer King Canada Austria Finland France Cyprus Bulgari Italy Hungary Ireland Iceland Luxembo Norway Sweden Ge New Zea Trinida Spain USSR Mauriti Jamaica Portuga Panama Romania # per physician Costa R Poland El Salv Malaya Nicarag Brit Mexico Yugosla Gu Guatema Chile # students GNP USA % literate India infant death 0 T1/P1 47%

13 Principal Components Analysis (PCA) Versatile exploratory tool Not two (Rich/poor), but three basic phenomena: Wealth, Island, health Barbado Taiwan Mauriti Trinida Japan Netherl Belgium Puerto West Ge Un King T2 27% 0 India Jamaica Italy Czechos Luxembo Switzer Hungary Austria Norway Denmark Poland Bulgari France El Salv Malaya Portuga Romania Spain Cyprus Ireland Finland Sweden Yugosla Costa R Iceland USSR New Chile Panama Austral Zea Guatema Nicarag Canada Brit Mexico Gu USA 0

14 Principal Components Analysis (PCA) See other patterns Score plot - Singapore and Hong Kong removed Rich Eastern Europe Poor 3 Score Score 1

15 Multivariate data multivariate data analysis deals with measurements of outcomes that consists of a set of variables for each observation denote x ij = measurement of the jth variable on the ith item same thing, different names measurement = observation = outcome = response = feature item = object = subject data displayed as an array, rows index individual observations, columns index variables standard in statistics (not necessarily elsewhere): n objects, p variables 1 / 1

16 Multivariate data variable 1 variable 2 variable j variable p item 1 x 11 x 12 x 1k x 1p item 2 x 21 x 22 x 2k x 2p item i x i1 x i2 x ij x ip item n x n1 x n2 x nj x np what is this table? multivariate analysis: regard this table as a matrix tools we mentioned: perform all kinds of operations to the matrix to deduce useful information 2 / 1

17 Matrices as a matrix X = x 11 x 12 x 1j x 1p x 21 x 22 x 2j x 2p x i1 x i2 x ij x ip x n1 x n2 x nj xnp = x T 1 x T 2 x T j x T n 1st (multivariate) observation 2nd (multivariate) observation jth (multivariate) observation nth (multivariate) observation ith row vector x T i = (x i1,, x ip ) comprises p components of the ith observation convention: all vectors are column vectors row vectors denoted by x T, transpose of x (statisticians use x ) x j = x j1 x jp = (x j1,, x jp ) T lower case x indicates an observed value or a realization of a random variable x ij represents ith observation of a random variable X j, usually denoted by a capital letter 3 / 1

18 Examples measurements stored as collection of feature vectors x 1,, x n R p corresponding to each of the objects biological data: gene expression levels, metabolomic profile image data: mri tractographs, movie clips text data: blogs, s, tweets webpages represent data conveniently as a matrix X R n p gene-microarray matrices of gene expression levels frame-pixel matrices of grey scale values term-document matrices of term frequencies-inverse document frequencies 4 / 1

19 Gene-microarray matrix Figure: 1,940 genes by 61 samples of different pathological stages Rows represent genes and columns represent samples, color-coded by tumor grade: blue, green, and red correspond to grades I, II, and III X Ma el al, Gene expression profiles of human breast cancer progression, Proc Nat Acad Sci, 100 (2003), no 10, pp / 1

20 Term-document matrix topics from MEDLINE: term-document matrix: 6 / 1

21 Therefore need to know how to handle matrices basic quantities: norms, inner products, determinant, rank fundamental subspaces: row space, column space, null space, range space common decompositions: LU, QR, EVD, SVD we will start from here 7 / 1