Multivariate Analysis - Overview

Transcription

1 Multivariate Analysis - Overview In general: - Analysis of Multivariate data, i.e. each observation has two or more variables as predictor variables, Analyses Analysis of Treatment Means (Single (multivariate) sample, twosamples,etc.) Study interrelationships correlations and predictions (regression) Other specific methods (discriminant analysis, principal components, clustering) Limitations: Many parameters estimated large sample sizes Whenever testing hypotheses, assumption of normality (almost overall) We ll focus on the multivariate methods and applications with somewhat limited mathematical emphasis (without proofs) Textbook: Applied Multivariate Analysis - Fifth Edition, Richard Johnson and Dean Wichern

2 Sweat data One sample testing means Table 5.1 Perspiration from 20 healthy women analyzed Three components: X 1 =sweat rate X 2 =sodium content X 3 =potasium content Null hypothesis: simultaneousely or or µ 1 = 4 µ 2 = 50 µ 3 = 10 [ 4 ] µ =[ 50] [10] µ = [4, 50,10]

3 Sweat data One sample testing means Table 5.1 Sweat rate Sodium Potasium

4 Sweat data One sample testing means Table 5.1 µ 0 = [4, 50,10] One-Sample T: Sweatrate, Sodium, Potasium Variable N Mean StDev SE Mean 90% CI Sweatrate ( , ) Sodium ( , ) Potasium ( , ) One-Sample T: Sweatrate, Sodium, Potasium Variable N Mean StDev SE Mean 95% CI Sweatrate ( , ) Sodium ( , ) Potasium ( , )

5 Sweat data One sample testing means Table 5.1 µ 0 = [4, 50,10] S-matrix Sweat rate Sweat rate Sodium Potasium Sodium Means Miu0 Xbar-Miu0 Potasium S-matrix(-1) Sweat rate Sodium Potasium Sweat rate Sodium Potasium n*(xbar-miu0) S(-1) (Xbar-Miu0) =20*.487=9.74 F=(n-p)/[(n-1)*p]*T2 =2.905 F(3,17,.9)=2.44 F(3,17,.9)=3.20

6 Turtle Carapaces- Two Samples Testing means Table 6.7 Jolicoeur and Mosimann studied relationship of size and shape for painted turtled. Measures on the caprapaces of 24 male and 24 female turtles Three components: X 1 =Length X 2 =Width X 3 =Height Null hypothesis: µ 1 = µ 2 (vectors of size 3)

7 Turtle Carapaces- Two Samples Testing means Table 6.7 Length Width Height Gender Length Width Height Gender female male female male female male female male female male female male female male female male female male female male female male female male female male female male female male female male female male female male female male female male female male female male

8 Turtle Carapaces- Two Samples Testing means Table 6.7 ANOVA: Length, Width, Height versus Gender Gender fixed 2 female, male Analysis of Variance for Length Source DF SS MS F P Gender Error Total S = R-Sq = 26.78% R-Sq(adj) = 25.23% Analysis of Variance for Width Source DF SS MS F P Gender Error Total S = R-Sq = 28.47% R-Sq(adj) = 26.95% Analysis of Variance for Height Source DF SS MS F P Gender Error Total S = R-Sq = 42.31% R-Sq(adj) = 41.08%

9 Turtle Carapaces- Two Samples Testing means Table 6.7 Means Gender N Length Width Height female male SSCP Matrix for Gender (B) MANOVA for Gender Length Width Height s = 1 m = 0.5 n = 21.5 Length Test DF Width Criterion Statistic F Num Denom P Height Wilks' Determinant =-426 Lawley-Hotelling SSCP Matrix for Error (W) Pillai's Length Width Height Length Width Height Determinant = 6.067E+08 SSCP Matrix for Error (B+W) Length Width Height Length Width Height Determinant = 1.481E+09 Determinant (W)/Determinant (B+W)=.6067/1.481=.410

10 Notations for MANOVA- Exact F-distributions for Wilk s Lambda ~ ) ( ) ( : 2 ) ~ ) ( ) ( : 2 ) ~ ) ( ) ( : 1 ) ~ ) ( ) ( : 1 ) 1), way}min( One - { ), min( way) in One - E ( of way) -1in One - ( Hypothesis (H) of Denote in MANOVA ) 1,2( ),2( , 1 1, 1 p v p p p v v q q v p v p p p v v q q v F F q d F F p c F F q b F F p a g p In q p s N-g df v g df q + + Λ Λ Λ Λ + + Λ Λ Λ Λ = = = = = = = = = = = =

11 Turtle Carapaces- Two Samples Testing means Table 6.7 Hotelling T 2 Means Gender N Length Width Height female male Diff SSCP Matrix for Error (W) Length Width Height Length Width Height S pooled = SSCP Matrix for Error (W)/(n1+n2-2) Hotelling T 2 =(xbar1-xbar2) [S pooled (1/n1+1/n2)] -1 (xbar1-xbar2)=65.66 F= Hotelling T 2 *(n1+n2-p-1)/[(n1+n2-2)p]= 65.66*[44/(3*46)]=65.66*0.319=20.94 df=p,n1+n2-p-1=3,44 Wilks'

12 Amitriptyline Data Multivariate Regression Analysis - Table7.6 Amitriptyline drug for depression. Several side effects: irregular heartbeat, abnormal BP, etc. Data on 17 patients admited after amitriptyline overdose Two dependent variables and 5 predictor variables: Y 1 =Total TCAD plasma level (TOT) Y 2 =Amount of amitriptyline in TCAD plasma level (AMI) z 1 =Gender 1=female, 0=male z 2 =Amount of amitriptyline taken at time of overdose(amt) z 3 =PR wave measurements (PR) z 4 =Diastolic blood pressure (DIAP) z 5 =QRS wave measurements (QRS) Analysis: Model to predict Y 1 and Y 2 from the predictor variables Multivariate Linear Regression Models

13 Amitriptyline Data Multivariate Regression Analysis - Table7.6 Y1-Tot - TCAD z2-amt Antidepress z4-diap (Diastolic BP) Y2-Ami z1- Gender z3-pr z5-qrs

14 Amitriptyline Data Multivariate Regression Analysis - Table7.6 Multivariate Linear Regression Models Regression Analysis: Y1 versus Z1, Z2, Z3, Z4, Z5 The regression equation is Y1 = Z Z Z Z Z5 Predictor Coef SE Coef T P Constant Z Z Z Z Z S = R-Sq = 88.7% R-Sq(adj) = 83.6% Analysis of Variance Source DF SS MS F P Regression Residual Error Total

15 Amitriptyline Data Multivariate Regression Analysis - Table7.6 Multivariate Linear Regression Models Regression Analysis: Y2 versus Z1, Z2, Z3, Z4, Z5 The regression equation is Y2 = Z Z Z Z Z5 Predictor Coef SE Coef T P Constant Z Z Z Z Z Analysis of Variance Source DF SS MS F P Regression Residual Error Total

16 Radiotheraphy-Principal Components Table1.5 Data of the 98 average ratings over the course of the treatment for 98 patients undergoing radiotherapy. Six components: X 1 =number of symptoms (as nausea, sore throat) X 2 =amount of activity (on a 1-5 scale) X 3 =amount of sleep (on a 1-5 scale) X 4 =amount of food consumed (on a 1-3 scale) X 5 =appetite (on a 1-5 scale) X 6 =skin reaction (on a 0-3 scale) Analysis: Finding a single (or several) measures linear combinations of the 6 components, to represent patients response to therapy. Principal components

17 Radiotheraphy-Principal Components Table1.5 Symptoms Activity Sleep Eat Appetite Skin Reaction

18 Radiotheraphy-Principal Components Table1.5 X 1 =number of symptoms (as nausea, sore throat) X 2 =amount of activity (on a 1-5 scale) X 3 =amount of sleep (on a 1-5 scale) X 4 =amount of food consumed (on a 1-3 scale) X 5 =appetite (on a 1-5 scale) X 6 =skin reaction (on a 0-3 scale) Principal components and Factor Analysis Principal Component Analysis: Symptoms, Activity, Sleep, Eat, Appetite, SkinRea Eigenanalysis of the Correlation Matrix Eigenvalue Proportion Cumulative Variable PC1 PC2 PC3 PC4 PC5 PC6 Symptoms Activity Sleep Eat Appetite SkinReaction

19 Radiotheraphy- Factor Analysis Table1.5 Principal Component Factor Analysis of the Correlation Matrix Unrotated Factor Loadings and Communalities Variable Factor1 Factor2 Factor3 Factor4 Communality Symptoms Activity Sleep Eat Appetite SkinReaction Variance % Var Rotated Factor Loadings and Communalities - Varimax Rotation Variable Factor1 Factor2 Factor3 Factor4 Communality Symptoms Activity Sleep Eat Appetite SkinReaction Variance % Var

20 Hemophilia Data- Discriminant Analysis- Table 11.8 To construct a procedure for detecting potential hemophilia A carriers, blood samples assayed and measurements made on two variables: X 1 =log 10 (AHF activity) where AHF=antihemophilic factor X 2 = log 10 (AHF-like antigen) Measurements taken on two groups of women: A group of n 1 =24 women who do not carry the hemophilic gene Normal group A group n 2 =22 women from known hemophilia A carriers (daughters of hemophiliacs, mothers with more than one hemophiliac son, mothers with one hemophiliac son and other hemophilic relatives) Obligatory carriers New cases to be classified Classification and Discrimination Discriminant Analysis

21 Hemophilia Data- Discriminant Analysis- Table 11.8 Noncarriers Obligatory Carriers New cases Requiring Classific log(ahf Groupactivity) log(ahf antigen) Group log(ahf activity) log(ahf antigen) log(ahf Group activity) log(ahf antigen)

22 Hemophilia Data- Discriminant Analysis- Table 11.8 Discriminant Analysis: GroupDisc versus logactivity, logantigen Linear Method for Response: GroupDisc Predictors: logactivity, logantigen Group 1 2 Count Summary of classification True Group Put into Group Total N N correct Proportion N = 75 N Correct = 64 Proportion Correct = Squared Distance Between Groups

23 Hemophilia Data- Discriminant Analysis- Table 11.8 Discriminant Analysis: GroupDisc versus logactivity, logantigen Linear Discriminant Function for Groups 1 2 Constant logactivity logantigen Summary of Misclassified Observations True Pred Squared Observation Group Group Group Distance Probability 5** ** Prediction for Test Observations Squared Observation Pred Group From Group Distance Probability

24 Hemophilia Data- Discriminant Analysis- Table 11.8 Discriminant Analysis: -Minitab Distance and discriminant functions Squared distance: The squared distance (also called the Mahalanobis distance) of observation x to the center (mean) of group i is given by the general form: d i2 (x) = (x - m i )' S p -1 (x - m i ) where: x = p-column vector with the values of this observation m i = column vector of length p containing the means of the predictors calculated from the data in group i; S p = pooled covariance matrix, used in linear discriminant analysis; The linear discriminant function =m i ' S p -1 x - 0.5m i 'S p -1 m i + ln p where: x = column vector of length p containing the values of the predictors for this observation (note, this column vector is stored as one row) mi = column vector of length p containing the means of the predictors calculated from the data in group i S p = pooled covariance matrix ln p = natural log of the prior probability For a given x, the group with the smallest squared distance has the largest linear discriminant function.

25 Hemophilia Data- Discriminant Analysis- Table 11.8 Discriminant Analysis: -Minitab Distance and discriminant functions Posterior probability- The posterior probability for group i given the data and is calculated by: p i f i (x)/σp i f i (x) where: p i = prior probability of group i f i (x) = the joint density for the data in group i (with the population parameters replaced by sample estimates) The largest posterior probability is equivalent to the largest value of ln [p i f i (x)], where (under normality): ln [p i f i (x)] = -0.5 [d i2 (x) - 2 lnp i ] - constant value where: d i2 (x) = -2 [m i ' S p -1 x - 0.5m i 'S p -1 m i + lnp i ] + x' S p -1 x

26 Bone and Skull - White leghorn fowls Canonical Correlation- Ex10.4 To assess correlation between two sets of variables Head Measurements (X (1) ) X (1) 1 X (1) 2 =Skull length =Skull breadth Leg Measurements (X (1) ) X (2) 1 X (2) 2 =Femur length =Tibia length Create new variables which represent the most of the correlations between the two sets of variables Cannonical Correlation

27 Bone and Skull - White leghorn fowls Canonical Correlation- Ex10.4 Skull Lenghts Skull Breadth Skull Breadth Tibia Length Skull Lenghts Skull Breadth Skull Breadth Tibia Length

28 Universities Cluster Analysis-Table 12-9 Data on certain universities for certain variables used to compare or rank major universities. The variables: X 1 =Average SAT for new freshmen X 2 = Percent new freshmen in top 10% of high school class X 3 =Percent of applicants accepted X 4 =Student faculty ratio X 5 =Estimated annual expenses X 6 =Graduation rate (%) Analysis: Clustering observations (universities) based on linear combinations of variables (or specific variables). Definition of distances Cluster Analysis, Distance Methods and Ordination

29 Universities Cluster Analysis-Table 12-9 # SAT Top10 Accept SFRatio Expenses 1 Harvard Princeton Yale Stanford MIT Duke CalTech Dartmouth Brown JohnsHopkin Uchicago UPenn Cornell Northwestern Columbia NotreDame UVir Georgetown CarnegieMello Umichigan UCBerkeley Uwisconsin PennState Purdue TexasA&M

30 Universities Cluster Analysis-Table 12-9 Cluster Analysis of Observations: SAT, Top10, Accept, SFRatio, Expenses, Grad Euclidean Distance, Single Linkage Amalgamation Steps Number Number of obs. of Similarity Distance Clusters New in new Step clusters level level joined cluster cluster Notre Dame UVirginia UPenn Northwestern Stanford Dartmouth MIT Duke (Stanford, Dartmouth) (MIT, Duke)

31 Universities Cluster Analysis-Table 12-9 Dendrogram with Single Linkage and Euclidean Distance Harvard Yale Stanford Dartmouth MIT Duke Princet on Brown Upenn Nort hwest ern Cornell Not redame Uvir Columbia Georget own Uchicago UCBerkeley JohnsHopkins CarnegieMellon Umichigan Uwisconsin TexasA&M PennState Purdue CalT ech Similarity Observations

32 Universities Cluster Analysis-Table 12-9 D e n d r o g r a m w i th S i n g l e L i n k a g e a n d C o r r e l a ti o n C o e f f i c i e n t D i s ta n c e Similarity SAT Top10 Expenses Grad V a r ia b le s Accept SFRatio Cluster Analysis of Variables: SAT, Top10, Accept, SFRatio, Expenses, Grad Correlation Coefficient Distance, Single Linkage- Amalgamation Steps Number Number of obs. of Similarity Distance Clusters New in new Step clusters level level joined cluster cluster Correlations: SAT, Top10, Accept, SFRatio, Expenses, Grad SAT Top10 Accept SFRatio Expenses Top Accept SFRatio Expenses Grad

33 Universities Cluster Analysis-Table 12-9 Cluster Analysis of Observations: SAT, Top10, Accept, SFRatio, Expenses, Grad Euclidean Distance, Single Linkage Amalgamation Steps K-means Cluster Analysis: SAT, Top10, Accept, SFRatio, Expenses, Grad Number of clusters: 2 - Cluster Centroids Variable Cluster1 Cluster2 centroid SAT Top Accept SFRatio Expenses Grad Number of clusters: 3 - Cluster Centroids Grand Variable Cluster1 Cluster2 Cluster3 centroid SAT Top Accept SFRatio Expenses Grad