I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
|
|
- Derick Nash
- 6 years ago
- Views:
Transcription
1 Comparisons of Several Multivariate Populations Edps/Soc 584 and Psych 594 Applied Multivariate Statistics Carolyn J Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN MANOVA Slide 1 of 66
2 Outline Multivariate SAS IML and 1 way ANOVA Classic Treatment As a general linear model 1 way MANOVA The Model: Generalization of ANOVA to multivariate Example 1: Massed vs distributed practice Multivariate General Linear Model and Example 2: Increased survival Following up to a significant result Multivariate contrasts Simultaneous confidence intervals Discriminant function Summary of PCA, MANOVA, DA SAS IML and PROC Reading: Johnson & Wichern pages MANOVA Slide 2 of 66
3 Generalizing 1 way ANOVA to Multivariate Data Generalizing 1 way ANOVA to Multivariate Data Basic Assumptions Multivariate SAS IML and and Generalizing multivariate T 2 to more than two populations Suppose that we have random samples from g populations and measures on p variables: Population 1: Population 2: Population g: where each x lj is a (p 1) vector x 11, x 12,,x 1n1 x 21, x 22,,x 2n2 x g1, x g2,,x gng Examples: Are 5 standardized tests scores the same for high school students who attend different high school programs (ie, general, vo/tech, academic) Are survival times measured in two ways different between those treated with supplemental vitamin C the over six types of cancer? MANOVA Slide 3 of 66
4 Basic Assumptions These are needed for Statistical Inference Generalizing 1 way ANOVA to Multivariate Data Basic Assumptions 1 X l1, X l2,,x lnl is a random sample of size n l from a population with means µ l for l = 1,,g (ie, observations within populations are independent and representative of their populations) Multivariate 2 Random samples from different populations are independent 3 All populations have the same covariance matrix, Σ 4 X lj N(µ l,σ); that is, each population is multivariate normal If a population is not multivariate normal, then for large n l central limit theorem may kick-in SAS IML and MANOVA Slide 4 of 66
5 Review The univariate case where p = 1 Review The Model for an Observation The Sums of Squares Sums of Squared Decomposition & Geometry ANOVA Summary Table Multivariate SAS IML and Assumptions: X lj N(µ l, σ 2 ) iid for j = 1,,n l and l = 1,,g Hypotheses: H o : µ 1 = µ 2 = = µ g versus H a : not H o We usually express µ l as the sum of a grand mean and deviations from the grand mean µ l }{{} l th pop mean = µ }{{} grand mean = µ + τ l + µ l µ }{{} l th pop treatment effect If µ 1 = µ 2 = = µ g, then an equivalent way to write the null hypothesis is H o : τ 1 = τ 2 = = τ g = 0 MANOVA Slide 5 of 66
6 The Model for an Observation X lj = µ + τ l + ɛ lj Review The Model for an Observation The Sums of Squares Sums of Squared Decomposition & Geometry ANOVA Summary Table where ɛ lj N(0, σ 2 ) and independent ɛ lj is random error We typically impose the condition g l=1 τ l = 0 as an identification constraint The decomposition of an observation is X lj }{{} observation = X }{{} overall sample mean + ( X l X) }{{} estimated treatment effect + (X lj = X l ) }{{} residual error Multivariate SAS IML and X is the estimator of µ ˆτ l = ( X l X) is the estimator of τ l (X lj X l ) is the estimator of ɛ lj MANOVA Slide 6 of 66
7 The Sums of Squares The sum of squared observations SS obs = SS total = g l=1 n l j=1 X 2 lj Review The Model for an Observation The Sums of Squares Sums of Squared Decomposition & Geometry ANOVA Summary Table Multivariate We also take the three components of X lj and form sums of squares ( g n l g ) SS mean = X 2 = n l X 2 SS treatment = SS residual = l=1 g l=1 g l=1 j=1 n l j=1 n l j=1 ˆτ 2 l = ˆɛ 2 lj = l=1 g n l ( X l X) 2 = l=1 j=1 g n l (X lj X l ) 2 l=1 j=1 g n l ( X l X) 2 l=1 SAS IML and MANOVA Slide 7 of 66
8 Sums of Squared Decomposition & Geometry SS obs = SS mean + SS tr + SS res Review The Model for an Observation The Sums of Squares Sums of Squared Decomposition & Geometry ANOVA Summary Table Multivariate or SS corrected = SS obs SS mean = SS tr + SS res This work because the components (sums of squares) are orthogonal Geometry: Consider the n = ( g l=1 n l) dimensional observation space where each observation defines a dimension We break this space into three orthogonal sub-spaces corresponding to each component The dimensionality of the sub-space corresponds to the degrees of freedom for the corresponding SS (see text for more details) SAS IML and MANOVA Slide 8 of 66
9 ANOVA Summary Table Let n + = g l=1 n l, the total sample size Source of Variation Sum of Squares df Treatment SS tr = ( g l=1 n l) X l 2 g 1 Review The Model for an Observation The Sums of Squares Sums of Squared Decomposition & Geometry ANOVA Summary Table Multivariate SAS IML and Residual Total (SS obs SS mean ) (corrected for mean) = g l=1 SS res = g nl l=1 j=1 (X lj X l ) 2 n + g nl j=1 (X lj X) 2 n + 1 Test statistic for H o : µ 1 = = µ g (or H o : τ 1 = = τ g ) and its sampling distribution are F = SS tr/(g 1) SS res /(n + g) F (g 1),(n + g) Reject H o for large values of SS tr /SS res large values of 1 + SS tr /SS res small values of (1 + SS tr /SS res ) 1 = SS res SS tr +SS res MANOVA Slide 9 of 66
10 One-Way ANOVA as a One-Way ANOVA as a Least Squares Estimates of Testing Using C Multivariate X 11 X 1n1 X 21 X 2n2 X g1 X gng = µ τ 1 τ 1 τ 2 τ 2 τ g 1 τ g 1 + ɛ 11 ɛ 1n1 ɛ 21 ɛ 2n2 ɛ g1 ɛ gng SAS IML and X n+ 1 }{{} Dependent = A n+ g }{{} Design Matrix β g 1 }{{} Parameters + ɛ n+ 1 }{{} Residuals MANOVA Slide 10 of 66
11 Least Squares Estimates of One-Way ANOVA as a Least Squares Estimates of Testing Using C Multivariate SAS IML and How we get parameter estimates depends on how the design matrix is set up There are multiple ways of setting up the design matrix We ll use the rank g matrix A on the previous slide ˆβ = (A A) 1 A X ˆx = Aˆβ = { x l } n+ 1 ˆɛ = X ˆX = X A(A A) 1 A X = (I A(A A) 1 A )X Our hypothesis test of equal population means, H o : µ 1 = µ 2 = = µ g τ 1 = τ 2 = = τ g = 0 can be expressed as H o : Cβ = 0 where C is a contrast matrix MANOVA Slide 11 of 66
12 One-Way ANOVA as a Least Squares Estimates of Testing Using C Multivariate SAS IML and Testing Using C For example, C (g 1) g = So τ 1 τ 2 τ 2 τ 3 H o : Cβ = 0 = τ g 2 τ g 1 Our F test (given before) tests H o : Cβ = 0 From framework, you can introduce continuous (numerical) variables ANOVA and multiple regression are essentially the same We can generalize the to the multivariate SAS PROC will make more sense MANOVA Slide 12 of 66
13 MANOVA model for comparing g population mean vectors parallels univariate ANOVA: Observation Vectors Sums-of-Squares and Cross-Products (SSCP) Sum of Squares A Closer Look at Within Groups SSCP X lj observation vector p 1 Random = µ overall mean vector p 1 + Fixed τ l l th treatment effect vector p 1 + ɛ lj residual for l th group, j th case p 1 Random Multivariate where ɛ lj N p (0,Σ) and all independent for j = 1,,n l cases per group, and l = 1,,g groups For Identification, g l=1 n lτ l = 0 SAS IML and MANOVA Slide 13 of 66
14 Observation Vectors Each component of X lj satisfies the 1-way ANOVA model, but now the model includes covariances among the components Observation Vectors Sums-of-Squares and Cross-Products (SSCP) Sum of Squares A Closer Look at Within Groups SSCP Multivariate These covariances are assumed to be equal across populations A vector of observations can be decomposed as X lj Observation = X overall sample mean + ( X l X) estimated treatment effect = ˆµ + ˆτ l + ˆɛ lj We also have a decomposition of sums-of-squares and crossproducts, or SSCP for short + (X lj X l ) residual SAS IML and MANOVA Slide 14 of 66
15 Sums-of-Squares and Cross-Products (SSCP) First we ll find the total corrected squares and cross-products (x lj x)(x lj x) = [(x lj x l ) + ( x l x) ] [(x lj x l ) + ( x l x) ] Observation Vectors Sums-of-Squares and Cross-Products (SSCP) Sum of Squares A Closer Look at Within Groups SSCP Multivariate = (x lj x l )(x lj x l ) + ( x l x)( x l x) }{{} squares & cross-products (x lj x l )( x l x) + ( x l x)(x lj x l ) }{{} cross-products and now sum all of this over cases and groups Since addition is distributive, we ll do this in pieces and look just at cross-product first SAS IML and MANOVA Slide 15 of 66
16 Observation Vectors Sums-of-Squares and Cross-Products (SSCP) Sum of Squares A Closer Look at Within Groups SSCP Multivariate Sum of Squares n l j=1 (x lj x l )( x l x) = = n l (x lj x l ) ( x l x) j=1 n l j=1 x lj n l x l ( x l x) = n l ( x l x l )( x l x) = 0 }{{} 0 Now summing the rest over j and l we get g l=1 n l j=1 (x lj x)(x lj x) = g l=1 n l ( x l x)( x l x) + Total (corrected) SSCP = Treatment + Residual g l=1 n l j=1 (x lj x l )(x lj x l ) = Between Groups + Within Groups = Hypothesis + Error SAS IML and MANOVA Slide 16 of 66
17 A Closer Look at Within Groups SSCP Observation Vectors Sums-of-Squares and Cross-Products (SSCP) Sum of Squares A Closer Look at Within Groups SSCP Multivariate W = E = = g n l (x lj x l )(x lj x l ) l=1 n 1 j=1 + j=1 (x 1j x 1 )(x 1j x 1 ) + n g n 2 j=1 (x gj x g )(x gj x g ) j=1 = (n 1 1)S 1 + (n 2 2)S (n g )S g (x 2j x 2 )(x 2j x 2 ) where S l is the sample covariance matrix for the l th group (treatment, condition, etc) W (E) is proportional to a pooled estimated of the common Σ SAS IML and MANOVA Slide 17 of 66
18 Between Groups SSCP & Test Statistic With respect to between groups SSCP, g g B = H = n l ( x l x)( x l x) = n lˆτ lˆτ l l=1 l=1 If H o : τ 1 = τ 2 = = τ g = 0 is true, Then B (H) should be close to 0 Between Groups SSCP & Test Statistic withλ Distribution of Wilk s Lambda Λ Other Test Statistics Pillai s Trace and Roy s Largest Root How They are All Related to Λ Which Test Statistic to Use Other Cases and Summary MANOVA To test H o, we consider the ratio of generalized SSCPs, Λ = W W + B = W T where T = W + B (ie, the total corrected SSCP) Λ is known as Wilk s Lambda It s equivalent to likelihood ratio statistic Multivariate MANOVA Slide 18 of 66
19 with Λ Λ is a ratio of generalized sampling variances Between Groups SSCP & Test Statistic withλ Distribution of Wilk s Lambda Λ Other Test Statistics Pillai s Trace and Roy s Largest Root How They are All Related to Λ Which Test Statistic to Use Other Cases and Summary MANOVA Multivariate Λ = W T = p i=1 λ i p i=1 λ i where λ i are eigenvalues of W, and λ i are eigenvalues of T If H o : τ 1 = τ 2 = = τ g = 0 is true then B is close to 0 = T W = λ i λ i = Λ close to 1 If H o : τ 1 = τ 2 = = τ g = 0 is false then B is not close 0 = values on diagonals of T, which will be positive, will be large = λ i < λ i = Λ is small The exact distribution of Λ can be derived for special cases of p and g MANOVA Slide 19 of 66
20 Distribution of Wilk s Lambda Λ Wilk s Λ = SSCP e SSCP e + SSCP h Between Groups SSCP & Test Statistic withλ Distribution of Wilk s Lambda Λ Other Test Statistics Pillai s Trace and Roy s Largest Root How They are All Related to Λ Which Test Statistic to Use Other Cases and Summary MANOVA Number df for variables Hypothesis Sampling distribution for multivariate data p = 1 ν h 1 p = 2 ν h 1 p 1 ν h = 1 p 2 ν h = 2 ( νe ( ν h ) ( 1 Λ Λ ) F νh,ν e ) ( ν e 1 1 ) Λ ν h Λ F 2νh,2(ν e 1) ( ) ( ) νe +ν h p 1 Λ p Λ F p,(νe +ν h p) ( νe +ν h p 1 p )( 1 Λ Λ ) F 2p,2(νe +ν h p 1) Multivariate where ν h = degrees of freedom for hypothesis, and ν e = degrees of freedom for error (residual) MANOVA Slide 20 of 66
21 Other Test Statistics Between Groups SSCP & Test Statistic withλ Distribution of Wilk s Lambda Λ Other Test Statistics Pillai s Trace and Roy s Largest Root How They are All Related to Λ Which Test Statistic to Use Other Cases and Summary MANOVA Multivariate There is more than one way to combine the information in B and W (or H and E) Wilk s Λ Λ = W W + B = E p E + H = i=1 λ i where λ i are eigenvalues of E, and λ i (E + H) Hotelling-Lawely Trace Criteria = trace(e 1 H) = tr(he 1 ) = where λ i is eigenvalue of HE 1 Reject H o when tr(he 1 ) is large When H o is true, ntr(he 1 ) χ 2 p(g 1) p i=1 λ i are eigenvalues of g λ i i=1 Note: df = rank of design matrix ( approach) MANOVA Slide 21 of 66
22 Pillai s Trace and Roy s Largest Root Pillai s Trace Criterion = trace(b(b + W) 1 ) = trace(h(h + E) 1 ) = p i=1 λ i 1 + λ i where λ i is the eigenvalue (root) of HE 1 Between Groups SSCP & Test Statistic withλ Distribution of Wilk s Lambda Λ Other Test Statistics Pillai s Trace and Roy s Largest Root How They are All Related to Λ Which Test Statistic to Use Other Cases and Summary MANOVA Roy s Largest Root Criterion θ = largest root of (E + H) 1 H = largest root of H(E + H) 1 ( ) λ1 = 1 + λ 1 where λ 1 is the largest root of E 1 H = HE 1 Multivariate MANOVA Slide 22 of 66
23 How They are All Related to Λ Let λ i be root of HE 1 ( eigenvalue of H relative to E ) and if all λ i s > 0 (ie, λ 1 λ 2 λ p 0), Between Groups SSCP & Test Statistic withλ Distribution of Wilk s Lambda Λ Other Test Statistics Pillai s Trace and Roy s Largest Root How They are All Related to Λ Which Test Statistic to Use Other Cases and Summary MANOVA Then we can write Λ = = = E E + H 1 I + HE 1 1 p i=1 (1 + λ i ) So Λ is a decreasing function of λ i Note: θ j = because AB = A B λ i 1 + λ i and λ i = θ i 1 θ i Multivariate MANOVA Slide 23 of 66
24 Which Test Statistic to Use Wilk s Λ = likelihood ratio statistic If all statistics lead to the same conclusion, use Λ Between Groups SSCP & Test Statistic withλ Distribution of Wilk s Lambda Λ Other Test Statistics Pillai s Trace and Roy s Largest Root How They are All Related to Λ Which Test Statistic to Use Other Cases and Summary MANOVA If statistics lead to different conclusion, need to figure out why From Simulation studies (power & robustness): Roy s largest root found to be the least useful, except when the population structure is such that groups differ in one dimension and one group is much more different from the rest Others all do pretty good w/rt power (they use more information in E and H than Roys s) Pillai s trace criterion is Least affected by departures from usual population model (ie, more robust against departures from normality) Better for diffuse alternative hypotheses versus sharper ones When roots are approximately equal, it has best power Wilk s and Hotwelling-Lawley have about the same power for a wider-range (spectrum) of alternative hypotheses Multivariate MANOVA Slide 24 of 66
25 Other Cases and Summary MANOVA For cases not covered If H o is true and g l=1 n l = n is large, then ( lnλ n 1 1 ) 2 (p + g) = ( W / B+W ) (n 1 12 ) (p + g) χ 2 p(g 1) Between Groups SSCP & Test Statistic withλ Distribution of Wilk s Lambda Λ Other Test Statistics Pillai s Trace and Roy s Largest Root How They are All Related to Λ Which Test Statistic to Use Other Cases and Summary MANOVA Multivariate You should examine the residual vectors for normality and outliers (ie, ˆɛ lj s) maybe use PCA or methods mentioned in the text Source of variation SSCP df Λ Wilk s Treatment B = g l=1 n l( x l x)( x l x) g 1 W / T (Between) Residual W = g l=1 (Within) n l j=1 (x jl x l )(x jl x l ) n g Total (corrected T = W + B n 1 for mean) = g l=1 n l j=1 (x jl x)(x jl x) MANOVA Slide 25 of 66
26 1 Way MANOVA: Data from Tatsuoka (1988), Multivariate Analysis: Techniques for Educational and Psychological Research, pp An experiment was conducted for comparing 2 methods (A & B) of teaching shorthand to 60 female seniors in a vocational high school (a dated example) Also of interest were the effects of distributed versus massed practice Descriptive Statistics Means and Confidence Regions Hypothesis Test Hypothesis Test continued Test Statistic & Distribution Multivariate C 1 : C 2 : C 3 : 2 hours of instruction/day for 6 weeks 3 hours of instruction/day for 4 weeks 4 hours of instruction/day for 3 weeks So each subject received a total of 12 hours of instruction For now, we ll just look the effect of distributed versus massed practice Note: n l = 20 for l = 1, 2, 3 Two variables (dependent measures): X 1 = speed X 2 = accuracy SAS IML and MANOVA Slide 26 of 66
27 Descriptive Statistics The overall mean vector and mean vectors for each condition: ( ) ( ) ( ) ( x = x 1 = x 2 = x 3 = ) Descriptive Statistics Means and Confidence Regions Hypothesis Test Hypothesis Test continued Test Statistic & Distribution Multivariate The treatment effect vectors (ie, ˆτ i = x i x) ( ) ( ) ˆτ 1 = ˆτ 2 = ˆτ 3 = Sample covariance matrices: ( ) S 1 = S 3 = ( S 2 = ( ( ) ) ) SAS IML and MANOVA Slide 27 of 66
28 Means and Accuracy % confidence regions for µ 1, µ 2 and µ 3 where n l = 20 Descriptive Statistics Means and Confidence Regions Hypothesis Test Hypothesis Test continued Test Statistic & Distribution x1 (2 hours/day for 6 weeks) x x2 (3 hours/day for 4 weeks) x3 (4 hours/day for 3 weeks) Speed Multivariate SAS IML and MANOVA Slide 28 of 66
29 Hypothesis Test No difference between massed versus distributed practice on either speed or accuracy: H o : τ 1 = τ 2 = τ 3 = 0 versus H a : τ l 0 for all l = 1, 2, 3 The within groups (residual) sums of squares and cross-products matrix Descriptive Statistics Means and Confidence Regions Hypothesis Test Hypothesis Test continued Test Statistic & Distribution Multivariate W = (n 1 1)S 1 + (n 2 1)S 2 + (n 3 1)S 3 ( ) ( = ( ) ( ) = ) SAS IML and MANOVA Slide 29 of 66
30 Hypothesis Test continued Descriptive Statistics Means and Confidence Regions Hypothesis Test Hypothesis Test continued Test Statistic & Distribution Multivariate The between groups SSCP matrix: 3 B = n l ( x l x)( x l x) l=1 ( ) ( = (493, 545) + 20 ( ) ( 532, 540) ( ) = T = W + B = ( ) (038, 05) Or T = (n 1)S where S is the covariance matrix computed over all groups and n is the total sample Then B = T W ) SAS IML and MANOVA Slide 30 of 66
31 Test Statistic & Distribution W = (177315)(28995) (41420) 2 = Descriptive Statistics Means and Confidence Regions Hypothesis Test Hypothesis Test continued Test Statistic & Distribution Multivariate SAS IML and T = (282818)(146725) (152575) 2 = Λ = W T = = 0188 For p = 2 and g = 3, we can use the exact sample distribution: ( ) ( n g 1 1 ) ( ) ( Λ n p 2 1 ) Λ = F g 1 Λ p Λ 2(g 1),2(n g 1) which for this example, (60 3 1) (3 1) ( 1 ) = 56 2 ( ) = Since F 4,112 (α = 05) F 4,120 (α = 05) = 245, reject H o that treatment vectors are all equal to 0 The data support the conclusion that there is an effect of massed versus distributed practice MANOVA Slide 31 of 66
32 1 Multivariate contrasts & confidence regions 2 Tests on individual variables (simultaneous confidence intervals for group/treatment differences) 3 Discriminant Analysis Multivariate Contrasts We need the multivariate generalization of the general linear model: X gn p = A gn (g+1) B (g+1) p + E gn p Multivariate SAS IML and where A is the design matrix (it could have g or g + 1 columns depending on the parameterization), and B is a matrix of coefficients (model parameters) some examples MANOVA Slide 32 of 66
33 A is n + g with dummy codes Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 33 of 66 parameters X n+ p = β o1 β o2 β op β 11 β 12 β 1p β 21 β 22 β 2p β g 1,1 β g 1,2 β g 1,p Given the design matrix above, β ok = µ gk, and β lk = µ lk µ gk If p = 1, we would have 1-way ANOVA +E n+ p
34 Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 34 of 66 parameters SAS PROC design: A is gn (g + 1) An alternative design matrix and parameter vector: β o1 β o2 β op β 11 β 12 β 1p X n+ p = β β 22 β 2p β g 1,1 β g 1,2 β g 1,p n + (g+1) Normally, ˆB = (A A) 1 A X; however, the rank of A defined above (and hence A A) is only g = There s no unique solution to A X = A AB +E n+ p
35 What s Interesting We re interested in differences between group means; that is, µ i µ k = (µ + τ i ) p 1 (µ + τ k ) p 1 = τ i τ k Even if we can t get unique estimates of elements of B, we can get unique estimates of differences between parameter estimates, which correspond to differences between group means regardless of what inverse of (A A) is used Moore-Penrose inverses of non-full rank square matrix (A A) is denoted by (A A) Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 35 of 66 parameters SAS PROC uses the Moore-Penrose inverse of (A A) In SAS/PROC IML, the Moore-Penrose inverse is obtained by the command ginv( ), for example giaa = ginv(a *A);
36 Estimable and Testable What we can do is test linear combinations of elements of B if the linear combination is a contrast Estimable: A linear function c B is estimable if (A A)(A A) c = c Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 36 of 66 parameters Testable: A linear function is testable if it only involves the estimable functions of B Contrasts of elements of B are estimable and therefore testable These correspond to differences between means We ll demonstrate multivariate General linear model by example
37 Example: Cameron & Pauling Data Increase in survival of cancer patients given supplemental treatment with vitamin C Increase in survival = the number of days a patient survives minus the number of days matched control survives x 1 = d 1 = increase in survival measured as days from first hospitalization Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 37 of 66 parameters x 2 = d 2 = increase in survival measured days from un-treatability type = type of cancer (1 =stomach, 2 =bronchus, 3 =colon, 4 =rectum, 5 =bladder, 6 =kidney)
38 Example: Descriptive statistics Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 38 of 66 parameters l Type n l x i = d i S l 1 Stomach Bronchus Colon Rectum Bladder Kidney ( ) S pool = (n l 1)S l = l=1
39 Plot of Means and 95% d2 (untreatable) Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 39 of 66 parameters colon x 3 bronchus x 2 x1 stomach x 6 kidney x 4 rectum x 5 bladder d1 (1 st hospitalization) Using S i to the compute regions n 1 = 12, n 2 = 16, n 3 = 16, n 4 = 7, n 5 = 5, n 6 = 7
40 Plot of Means and 95% d2 Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 40 of 66 parameters colon x 3 bronchus x 2 x1 stomach x 6 kidney x 4 rectum Using S pool to the compute regions x 5 bladder d1
41 s of MANOVA Hypothesis Test H o : µ stomach = µ bronchus = µ colon = µ rectum = µ bladder = µ kidney or equivalently Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 41 of 66 parameters H o : τ stomach = τ bronchus = τ colon = τ rectum = τ bladder = τ kidney df type of cancer (hypothesis) ν h = g 1 = 6 1 = 5 df within (error) = ν e = l n l g = 63 6 = 57 Wilk s Λ = det(w)/ det(t) = Since p = 2 dependent variables, Wilk s Λ has an exact sampling distribution that is F, in particular ( ) ( (νe 1) 1 ) Λ F = F 2νh,2ν Λ e ν h F = and p-value = 0005 Reject H o The data support the conclusion that not all of the means (or τ s are equal)
42 Estimated MANOVA parameters ˆµ = ( ) Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 42 of 66 parameters Type hospitalization untreatable Stomach ˆτ 1 = ( , 71543) Bronchus ˆτ 2 = (156055, 59585) Colon ˆτ 3 = ( 88368, ) Rectum ˆτ 4 = (91873, 60111) Bladder ˆτ 5 = ( , 36660) Kidney ˆτ 6 = ( 86698, 64746) Recall that µ l = µ + τ l
43 MANOVA as a multivariate A main effect and six dummy variables (this is what PROC does) So the design matrix looks like } n stomach } n bronchus A n+ 7 = } n colon } n rectum } n bladder } n kidney Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 43 of 66 parameters MANOVA (multivariate general linear model): Estimation: y n+ 2 = A n+ 7B ɛ n+ 2 B = (A A) A y Predicted values: ŷ = AB where ŷ jl = ( x 1l, x 2l )
44 MANOVA as a multivariate (continued) For ŷ jl = ( x 1l, x 2l ), it is the the case that x j1l = b o1 + b l1 and x j2l = b o2 + b l2 So to compare two groups (types of cancer), Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 44 of 66 parameters x il x il = (b oi + b li ) (b oi + b l i) = b li b l i Consider a contrast between means for two types of cancer, for example, stomach and bronchial, c B = (0, 1, 1, 0, 0, 0, 0) b o1 b o2 b 11 b 12 b 21 b 22 b 31 b 32 b 41 b 42 b 51 b 52 b 61 b 62 = ((b 11 b 21 ), (b 12 b 22 )) = (( x 11 x 21 ), ( x 12 x 22 ))
45 With the Parameter Estimates B = Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 45 of 66 parameters stomach: x 11 = b o1 + b 11 = ( 25564) = 7067 x 12 = b o2 + b 12 = ( 6393) = 9491 bronchus: x 21 = b o1 + b 21 = ( 27681) = 4950 x 22 = b o2 + b 22 = ( 5197) = (0, 1, 1, 0, 0, 0, 0)B = (( ), ( )) = (2117, 1196) = (( x 11 x 21 ), ( x 12 x 22 )) = 0 ie, Are effects for stomach and bronchus the same?
46 Testing H o : CB = 0 Our hypothesis tests can be of the form H o : C r g B g p M p s = 0 C defines hypotheses (contrasts) on the elements of columns of B; that is, comparison between means on same variables over groups M defines hypotheses (contrasts) on the elements of rows of B; that is, comparison between means on same group over variables For now M = I and we ll consider hypotheses of the form Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 46 of 66 parameters H o : CB = 0 Specifically, we want to consider H o : 0b 0k +c 1 b 1k +c 2 b 2k + +c g b gk c 1 τ 1 +c 2 τ 2 + +c g τ g where g l=1 c l = 0
47 Testing Contrasts: The H matrix For a simple contrast, such as c = (0, 1, 1, 0, 0) we could do this as a multivariate T 2 test for independent groups; however, we ll stay within the MANOVA and multivariate linear model framework (so we can test multiple ones) Suppose that we have a contrast matrix C r (g+1) where the rows are r orthogonal contrasts, the hypothesis matrix equals H = (CB) (C(A A) C ) 1 (CB) Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 47 of 66 parameters For a balanced design (ie, n 1 = n 2 = = n g = n and a single contrast (ie, r = 1), this reduces to ( g ) ( g ) n H = g l=1 c c l x l c l x l l l=1 l=1 Note: c o = 0
48 Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 48 of 66 parameters Testing Contrasts The Error matrix is W ; that is, E = W = g l=1 n (x jk x l )(x jk x l ) j=1 = X X B (A A)B Wilk s Lambda for the test H o : CB = 0 is Λ = det(e) det(h + E) To find the transformation of this to an F distribution: p = anything ν hypothesis = r ν error = l n l p
49 Example: Five types the same? Three equivalent forms for Hypothesis 1: H o : H o : H o : µ bronchus = µ colon = µ kidney = µ rectum = µ stomach τ bronchus = τ colon = τ kidney = τ rectum = τ stomach β bronchus = β colon = β kidney = β rectum = β stomach Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 49 of 66 parameters where β l is a p 1 column vector of B (ie, a row of B written as a column) For the contrast matrix we need to know the order of the effects in the I re-order them so that they are in alphabetical order, because PROC puts them in alphabetical order (or numerical if groups are coded this way)
50 Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 50 of 66 parameters H o : Four types the same? H o : CB = 0 = So H o : (β 21 β 31 ) (β 22 β 32 ) (β 21 β 41 ) (β 22 β 42 ) (β 21 β 51 ) (β 22 β 52 ) (β 21 β 61 ) (β 22 β 62 ) = β 01 β o2 β 11 β 12 β 21 β 22 β 31 β 32 β 41 β 42 β 51 β 52 β 61 β 61 (τ 21 τ 31 ) (τ 22 τ 32 ) (τ 21 τ 41 ) (τ 22 τ 42 ) (τ 21 τ 51 ) (τ 22 τ 52 ) (τ 21 τ 61 ) (τ 22 τ 62 ) intercept bladder bronchus colon kidney rectum stomach = 0
51 Hypothesis Matrices H = (CB) (C(A A)(ns + nb + nc + nr + nd + nk 6); C ) 1 (CB) ( ) The E error SSCP is the same as W that we used before, which equals Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 51 of 66 parameters E = W = X B(A A)B X ( = Wilk s Lambda, Λ = det(e) det(h + E) = 44877E E13 = and ν h = number of rows of C = 4 ν e = number of rows of X p = 57 )
52 s Referring to the table for transformations of Λ that have sampling distributions that are F, we use the one for p = 2 and ν h 1, which is F = ( νe 1 ν h ) ( 1 Λ Λ ) = ( 56 4 ) ( ) = Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 52 of 66 parameters If the null is true, then this should have a F 2νh,2(ν e 1) sampling distribution Comparing F = 145 to the F 4,112, we find that the p-value is 18 Retain the null hypothesis The data suggest not difference in increased survival of patients over different types of cancer (except bladder)
53 Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 53 of 66 parameters Five versus the Rest H o : τ bladder = τ bronchus +τ colon +τ kidney +τ rectum +τ stomach )/5 or equivalently H o : CB = 0 = (0, 5, 1, 1, 1, 1, 1) β o1 β o2 β 11 β 12 β 21 β 22 β 31 β 32 β 41 β 42 β 51 β 52 β 61 β 61 E is the same as before, but now ( ) H = intercept bladder bronchus colon kidney rectum stomach
54 Multivariate A is n + g with dummy codes SAS PROC design: A is gn (g + 1) What s Interesting Estimable and Testable Example: Cameron & Pauling Data Example: Descriptive statistics s of MANOVA Hypothesis Test MANOVA Estimated MANOVA Slide 54 of 66 parameters The Test and Λ = 44877E E13 = ν e = g l=1 n l g = 57 ν h = 1, the number of rows of C So, for ν h = 1, use ( )( ) νe + ν h p 1 Λ F = p Λ = , which if the null is true (and assumptions valid), F should have a sampling distribution that is F p,(νe +ν h p) Comparing F to F 2,56, we get a p-value< 01 Reject H o In summary, the mean survival of patients with bladder cancer differs from that of those with other types of cancer; however, no support for differences between the other types Question: Are there differences for survival from first hospitalization and/or from time of unreadability?
55 We can construct simultaneous confidence intervals for components of differences τ l τ l (which equal µ l µ l ) Multivariate Using CBM = 0 Confidence interval for CBM Our example: CI for CBM Notes about These CIs Bonferroni Roy s Method A Truly Multivariate Follow-Up MANOVA Slide 55 of 66 or other linear combinations such as τ 1 (τ 2 + τ 3 )/2 There are at least three ways of doing this Specify a matrix M in the hypothesis test H o : CBM = 0 that is a (p 1) vector with all M = (0,, 1 }{{} i th, 0) Bonferroni-type: Same as above but split the α into pieces, on part for each of the planned comparisons Roys method, which is based on the union-intersection principle
56 Using CBM = 0 C picks out which two (or more groups to compare) eg, want to compare bladder with the rest, C = (0, 1, 2, 2, 2, 2, 2) M picks out which variable (or linear combination of variables) eg, Just compare d 1, increase in survival from first hospitalization, M = (1, 0) Multivariate Using CBM = 0 Confidence interval for CBM Our example: CI for CBM Notes about These CIs Bonferroni Roy s Method A Truly Multivariate Follow-Up MANOVA Slide 56 of 66 Putting these together in our example gives us (0, 1, 2, 2, 2, 2, 2) β o1 β o2 β 11 β 12 β 21 β 22 β 31 β 32 β 41 β 42 β 51 β 52 β 61 β 61 ( 1 0 ) = ( β ) 6 β l1 l=2
57 Confidence interval for CBM β We need two things: 6 6 β l1 = τ 11 l=2 l=2 A fudge-factor a value from a probability distribution τ l1 Multivariate Using CBM = 0 Confidence interval for CBM Our example: CI for CBM Notes about These CIs Bonferroni Roy s Method A Truly Multivariate Follow-Up MANOVA Slide 57 of 66 An estimate of the standard error A (1 α)100% confidence statement given vectors C 1 (g+1) and M p 1 is CBM ± F 1,νe (α) (M S pool M)(C(A A) C ) Note: Consider two columns of B, β i and β k, the covariance matrix between them is cov(β i, β k ) = s pool,ik (A A)
58 Our example: CI for CBM β β l1 = ( ) 5 l=2 = (M S pool M)(C(A A) C ) = s pool,11 ( ) = ( )( ) = Multivariate Using CBM = 0 Confidence interval for CBM Our example: CI for CBM Notes about These CIs Bonferroni Roy s Method A Truly Multivariate Follow-Up MANOVA Slide 58 of 66 And F 1,57 (05) = 401 So our 95% confidence interval is ± ± 401(30634) (56003, ) Since 0 is not in the interval, the mean increase in survival from first hospitalization due to bladder cancer is larger than the average of the others means Should we test whether the same is true for increase in survival from time of untreatability?
59 Plot of Means and 95% d2 Multivariate Using CBM = 0 Confidence interval for CBM Our example: CI for CBM Notes about These CIs Bonferroni Roy s Method A Truly Multivariate Follow-Up MANOVA Slide 59 of 66 x 3 x 4 x 2 x 5 x 6 x1 d1
60 Notes about These CIs If you re only looking (testing) the difference between two means, eg, τ li τ l i Multivariate Using CBM = 0 Confidence interval for CBM Our example: CI for CBM Notes about These CIs Bonferroni Roy s Method A Truly Multivariate Follow-Up MANOVA Slide 60 of 66 Then the standard error is just s pool,ii 1/nl + 1/n l When looking at a difference for a variable (eg, above), these confidence statements are equivalent to what you would get from 1-way ANOVA using Fisher s least significant differences; that is, they are univariate CIs When considering a linear combination of variables, these CIs are equivalent to univariate CIs where you ve analyzed a new or composite variable defined by the linear combination In our example, we don t have to worry too much about inflated Type I error rate, because we only did one CI after rejecting the overall test and using multivariate contrasts to narrow down where differences exist If you do all pairwise differences, there are g(g 1)/2 pairs times p variables (eg, 2(6)(5)/2 = 30)
61 Bonferroni If you have planned to look at all pairwise comparisons before looking at the data (ie, m = pg(g 1)/2), then you can use as your fudge factor t νe (α/(2m)) Multivariate Using CBM = 0 Confidence interval for CBM Our example: CI for CBM Notes about These CIs Bonferroni Roy s Method A Truly Multivariate Follow-Up MANOVA Slide 61 of 66 Let n + = g l=1 n l For the model X lj = µ + τ l + ɛ lj with j = 1,,n l and l = 1,,g with confidence at least (1 α), (τ li τ l i) belongs to ( x li x l i) ± t νe (α/(2m)) s pool,ii ( 1 n l + 1 n l for all components (variables) i = 1,,p and all differences l < l = 1,,g No need to do this with our cancer data )
62 Multivariate Using CBM = 0 Confidence interval for CBM Our example: CI for CBM Notes about These CIs Bonferroni Roy s Method A Truly Multivariate Follow-Up MANOVA Slide 62 of 66 Roy s Method This is based on the union intersection principle This is more like the first method that we considered (ie, CBM); however, we use a different distribution for our fudge factor We use Greatest Root, of H θ(e + H) = 0 where H (between groups or hypothesis SSCP matrix) and E = W (error or within groups SSCP matrix) are independent Wishart matrices To apply this result, we need percentiles of the greatest root distribution of the largest root λ of the equation H λe = 0 Percentile can be found in tables This distribution does not depend on Σ but only on df = n g p 1 Recommendation: I ld suggest using Scheffé s method where you do 1-way ANOVA on a linear combination of variables and then specify the contrast that you want
63 d2 1 st PC Multivariate Summary: PCA, MANOVA, DA SAS IML and x 3 x x 4 x 2 x 5 x 6 x1 d1 1 st Discriminant MANOVA Slide 63 of 66
64 SAS IML and SAS IML code using traditional approach & one PROC data=vitc; Multivariate SAS IML and SAS IML and SAS Output class type ; model d1 d2 = type /solution; * Note: The order of the values in the contrast are alphabetical, in this case order is bladder bronchus colon kidney rectum stomach; contrast bronchus=colon=kidney=rectum=stomach type , type , type , type ; contrast bladder vs others type ; manova h=type /printh printe; estimate b vs o type ; lsmeans type; title MANOVA of vitamin C and Cancer ; Alternate MANOVA statement where M is entered as M : manova h=type M=(1 0); MANOVA Slide 64 of 66
65 SAS Output Univariate ANOVAs for each dependent variable If requested, E (printe) and H (printh) SSCP matrices Multivariate p characteristic roots and vectors of E 1 H (ie, discriminant functions) Other requested statistics: contrasts estimates of contrasts cell means etc Test statistics for no overall effect specified in MANOVA statement SAS IML and SAS IML and SAS Output MANOVA Slide 65 of 66
Comparisons of Several Multivariate Populations
Comparisons of Several Multivariate Populations Edps/Soc 584, Psych 594 Carolyn J Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees,
More informationInferences about a Mean Vector
Inferences about a Mean Vector Edps/Soc 584, Psych 594 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees, University
More informationOther hypotheses of interest (cont d)
Other hypotheses of interest (cont d) In addition to the simple null hypothesis of no treatment effects, we might wish to test other hypothesis of the general form (examples follow): H 0 : C k g β g p
More informationApplied Multivariate and Longitudinal Data Analysis
Applied Multivariate and Longitudinal Data Analysis Chapter 2: Inference about the mean vector(s) II Ana-Maria Staicu SAS Hall 5220; 919-515-0644; astaicu@ncsu.edu 1 1 Compare Means from More Than Two
More informationLecture 5: Hypothesis tests for more than one sample
1/23 Lecture 5: Hypothesis tests for more than one sample Måns Thulin Department of Mathematics, Uppsala University thulin@math.uu.se Multivariate Methods 8/4 2011 2/23 Outline Paired comparisons Repeated
More informationChapter 7, continued: MANOVA
Chapter 7, continued: MANOVA The Multivariate Analysis of Variance (MANOVA) technique extends Hotelling T 2 test that compares two mean vectors to the setting in which there are m 2 groups. We wish to
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Comparisons of Two Means Edps/Soc 584 and Psych 594 Applied Multivariate Statistics Carolyn J. Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN c
More informationMultivariate analysis of variance and covariance
Introduction Multivariate analysis of variance and covariance Univariate ANOVA: have observations from several groups, numerical dependent variable. Ask whether dependent variable has same mean for each
More informationMultivariate Linear Models
Multivariate Linear Models Stanley Sawyer Washington University November 7, 2001 1. Introduction. Suppose that we have n observations, each of which has d components. For example, we may have d measurements
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Canonical Edps/Soc 584 and Psych 594 Applied Multivariate Statistics Carolyn J. Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN Canonical Slide
More information5 Inferences about a Mean Vector
5 Inferences about a Mean Vector In this chapter we use the results from Chapter 2 through Chapter 4 to develop techniques for analyzing data. A large part of any analysis is concerned with inference that
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 17 for Applied Multivariate Analysis Outline Multivariate Analysis of Variance 1 Multivariate Analysis of Variance The hypotheses:
More informationTHE UNIVERSITY OF CHICAGO Booth School of Business Business 41912, Spring Quarter 2012, Mr. Ruey S. Tsay
THE UNIVERSITY OF CHICAGO Booth School of Business Business 41912, Spring Quarter 2012, Mr. Ruey S. Tsay Lecture 3: Comparisons between several multivariate means Key concepts: 1. Paired comparison & repeated
More information4.1 Computing section Example: Bivariate measurements on plants Post hoc analysis... 7
Master of Applied Statistics ST116: Chemometrics and Multivariate Statistical data Analysis Per Bruun Brockhoff Module 4: Computing 4.1 Computing section.................................. 1 4.1.1 Example:
More informationApplied Multivariate and Longitudinal Data Analysis
Applied Multivariate and Longitudinal Data Analysis Chapter 2: Inference about the mean vector(s) Ana-Maria Staicu SAS Hall 5220; 919-515-0644; astaicu@ncsu.edu 1 In this chapter we will discuss inference
More informationMultivariate Linear Regression Models
Multivariate Linear Regression Models Regression analysis is used to predict the value of one or more responses from a set of predictors. It can also be used to estimate the linear association between
More informationMANOVA is an extension of the univariate ANOVA as it involves more than one Dependent Variable (DV). The following are assumptions for using MANOVA:
MULTIVARIATE ANALYSIS OF VARIANCE MANOVA is an extension of the univariate ANOVA as it involves more than one Dependent Variable (DV). The following are assumptions for using MANOVA: 1. Cell sizes : o
More informationLeast Squares Estimation
Least Squares Estimation Using the least squares estimator for β we can obtain predicted values and compute residuals: Ŷ = Z ˆβ = Z(Z Z) 1 Z Y ˆɛ = Y Ŷ = Y Z(Z Z) 1 Z Y = [I Z(Z Z) 1 Z ]Y. The usual decomposition
More informationAnalysis of variance, multivariate (MANOVA)
Analysis of variance, multivariate (MANOVA) Abstract: A designed experiment is set up in which the system studied is under the control of an investigator. The individuals, the treatments, the variables
More informationMultilevel Models in Matrix Form. Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2
Multilevel Models in Matrix Form Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Today s Lecture Linear models from a matrix perspective An example of how to do
More informationSTAT 730 Chapter 5: Hypothesis Testing
STAT 730 Chapter 5: Hypothesis Testing Timothy Hanson Department of Statistics, University of South Carolina Stat 730: Multivariate Analysis 1 / 28 Likelihood ratio test def n: Data X depend on θ. The
More informationMULTIVARIATE ANALYSIS OF VARIANCE
MULTIVARIATE ANALYSIS OF VARIANCE RAJENDER PARSAD AND L.M. BHAR Indian Agricultural Statistics Research Institute Library Avenue, New Delhi - 0 0 lmb@iasri.res.in. Introduction In many agricultural experiments,
More informationApplied Multivariate Statistical Modeling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur
Applied Multivariate Statistical Modeling Prof. J. Maiti Department of Industrial Engineering and Management Indian Institute of Technology, Kharagpur Lecture - 29 Multivariate Linear Regression- Model
More informationExample 1 describes the results from analyzing these data for three groups and two variables contained in test file manova1.tf3.
Simfit Tutorials and worked examples for simulation, curve fitting, statistical analysis, and plotting. http://www.simfit.org.uk MANOVA examples From the main SimFIT menu choose [Statistcs], [Multivariate],
More informationMultivariate Regression (Chapter 10)
Multivariate Regression (Chapter 10) This week we ll cover multivariate regression and maybe a bit of canonical correlation. Today we ll mostly review univariate multivariate regression. With multivariate
More informationGroup comparison test for independent samples
Group comparison test for independent samples The purpose of the Analysis of Variance (ANOVA) is to test for significant differences between means. Supposing that: samples come from normal populations
More informationRepeated Measures Part 2: Cartoon data
Repeated Measures Part 2: Cartoon data /*********************** cartoonglm.sas ******************/ options linesize=79 noovp formdlim='_'; title 'Cartoon Data: STA442/1008 F 2005'; proc format; /* value
More informationMANOVA MANOVA,$/,,# ANOVA ##$%'*!# 1. $!;' *$,$!;' (''
14 3! "#!$%# $# $&'('$)!! (Analysis of Variance : ANOVA) *& & "#!# +, ANOVA -& $ $ (+,$ ''$) *$#'$)!!#! (Multivariate Analysis of Variance : MANOVA).*& ANOVA *+,'$)$/*! $#/#-, $(,!0'%1)!', #($!#$ # *&,
More informationRepeated Measures ANOVA Multivariate ANOVA and Their Relationship to Linear Mixed Models
Repeated Measures ANOVA Multivariate ANOVA and Their Relationship to Linear Mixed Models EPSY 905: Multivariate Analysis Spring 2016 Lecture #12 April 20, 2016 EPSY 905: RM ANOVA, MANOVA, and Mixed Models
More informationOne-Way ANOVA. Some examples of when ANOVA would be appropriate include:
One-Way ANOVA 1. Purpose Analysis of variance (ANOVA) is used when one wishes to determine whether two or more groups (e.g., classes A, B, and C) differ on some outcome of interest (e.g., an achievement
More informationApplication of Ghosh, Grizzle and Sen s Nonparametric Methods in. Longitudinal Studies Using SAS PROC GLM
Application of Ghosh, Grizzle and Sen s Nonparametric Methods in Longitudinal Studies Using SAS PROC GLM Chan Zeng and Gary O. Zerbe Department of Preventive Medicine and Biometrics University of Colorado
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Introduction Edps/Psych/Stat/ 584 Applied Multivariate Statistics Carolyn J Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN c Board of Trustees,
More informationANOVA Longitudinal Models for the Practice Effects Data: via GLM
Psyc 943 Lecture 25 page 1 ANOVA Longitudinal Models for the Practice Effects Data: via GLM Model 1. Saturated Means Model for Session, E-only Variances Model (BP) Variances Model: NO correlation, EQUAL
More information20.1. Balanced One-Way Classification Cell means parametrization: ε 1. ε I. + ˆɛ 2 ij =
20. ONE-WAY ANALYSIS OF VARIANCE 1 20.1. Balanced One-Way Classification Cell means parametrization: Y ij = µ i + ε ij, i = 1,..., I; j = 1,..., J, ε ij N(0, σ 2 ), In matrix form, Y = Xβ + ε, or 1 Y J
More informationChapter 9. Multivariate and Within-cases Analysis. 9.1 Multivariate Analysis of Variance
Chapter 9 Multivariate and Within-cases Analysis 9.1 Multivariate Analysis of Variance Multivariate means more than one response variable at once. Why do it? Primarily because if you do parallel analyses
More informationz = β βσβ Statistical Analysis of MV Data Example : µ=0 (Σ known) consider Y = β X~ N 1 (β µ, β Σβ) test statistic for H 0β is
Example X~N p (µ,σ); H 0 : µ=0 (Σ known) consider Y = β X~ N 1 (β µ, β Σβ) H 0β : β µ = 0 test statistic for H 0β is y z = β βσβ /n And reject H 0β if z β > c [suitable critical value] 301 Reject H 0 if
More informationAn Introduction to Multivariate Statistical Analysis
An Introduction to Multivariate Statistical Analysis Third Edition T. W. ANDERSON Stanford University Department of Statistics Stanford, CA WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Contents
More informationWITHIN-PARTICIPANT EXPERIMENTAL DESIGNS
1 WITHIN-PARTICIPANT EXPERIMENTAL DESIGNS I. Single-factor designs: the model is: yij i j ij ij where: yij score for person j under treatment level i (i = 1,..., I; j = 1,..., n) overall mean βi treatment
More informationProfile Analysis Multivariate Regression
Lecture 8 October 12, 2005 Analysis Lecture #8-10/12/2005 Slide 1 of 68 Today s Lecture Profile analysis Today s Lecture Schedule : regression review multiple regression is due Thursday, October 27th,
More informationTopic 17 - Single Factor Analysis of Variance. Outline. One-way ANOVA. The Data / Notation. One way ANOVA Cell means model Factor effects model
Topic 17 - Single Factor Analysis of Variance - Fall 2013 One way ANOVA Cell means model Factor effects model Outline Topic 17 2 One-way ANOVA Response variable Y is continuous Explanatory variable is
More informationMore Linear Algebra. Edps/Soc 584, Psych 594. Carolyn J. Anderson
More Linear Algebra Edps/Soc 584, Psych 594 Carolyn J. Anderson Department of Educational Psychology I L L I N O I S university of illinois at urbana-champaign c Board of Trustees, University of Illinois
More informationMultivariate Statistical Analysis
Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 9 for Applied Multivariate Analysis Outline Addressing ourliers 1 Addressing ourliers 2 Outliers in Multivariate samples (1) For
More informationRegression #5: Confidence Intervals and Hypothesis Testing (Part 1)
Regression #5: Confidence Intervals and Hypothesis Testing (Part 1) Econ 671 Purdue University Justin L. Tobias (Purdue) Regression #5 1 / 24 Introduction What is a confidence interval? To fix ideas, suppose
More informationNeuendorf MANOVA /MANCOVA. Model: X1 (Factor A) X2 (Factor B) X1 x X2 (Interaction) Y4. Like ANOVA/ANCOVA:
1 Neuendorf MANOVA /MANCOVA Model: X1 (Factor A) X2 (Factor B) X1 x X2 (Interaction) Y1 Y2 Y3 Y4 Like ANOVA/ANCOVA: 1. Assumes equal variance (equal covariance matrices) across cells (groups defined by
More informationAnalysis of Longitudinal Data: Comparison Between PROC GLM and PROC MIXED. Maribeth Johnson Medical College of Georgia Augusta, GA
Analysis of Longitudinal Data: Comparison Between PROC GLM and PROC MIXED Maribeth Johnson Medical College of Georgia Augusta, GA Overview Introduction to longitudinal data Describe the data for examples
More informationRejection regions for the bivariate case
Rejection regions for the bivariate case The rejection region for the T 2 test (and similarly for Z 2 when Σ is known) is the region outside of an ellipse, for which there is a (1-α)% chance that the test
More informationCovariance Structure Approach to Within-Cases
Covariance Structure Approach to Within-Cases Remember how the data file grapefruit1.data looks: Store sales1 sales2 sales3 1 62.1 61.3 60.8 2 58.2 57.9 55.1 3 51.6 49.2 46.2 4 53.7 51.5 48.3 5 61.4 58.7
More informationPrepared by: Prof. Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti
Prepared by: Prof. Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti Putra Malaysia Serdang Use in experiment, quasi-experiment
More informationNotes for Week 13 Analysis of Variance (ANOVA) continued WEEK 13 page 1
Notes for Wee 13 Analysis of Variance (ANOVA) continued WEEK 13 page 1 Exam 3 is on Friday May 1. A part of one of the exam problems is on Predictiontervals : When randomly sampling from a normal population
More informationWeek 14 Comparing k(> 2) Populations
Week 14 Comparing k(> 2) Populations Week 14 Objectives Methods associated with testing for the equality of k(> 2) means or proportions are presented. Post-testing concepts and analysis are introduced.
More informationSOME ASPECTS OF MULTIVARIATE BEHRENS-FISHER PROBLEM
SOME ASPECTS OF MULTIVARIATE BEHRENS-FISHER PROBLEM Junyong Park Bimal Sinha Department of Mathematics/Statistics University of Maryland, Baltimore Abstract In this paper we discuss the well known multivariate
More informationTopic 28: Unequal Replication in Two-Way ANOVA
Topic 28: Unequal Replication in Two-Way ANOVA Outline Two-way ANOVA with unequal numbers of observations in the cells Data and model Regression approach Parameter estimates Previous analyses with constant
More informationRandom Matrices and Multivariate Statistical Analysis
Random Matrices and Multivariate Statistical Analysis Iain Johnstone, Statistics, Stanford imj@stanford.edu SEA 06@MIT p.1 Agenda Classical multivariate techniques Principal Component Analysis Canonical
More informationChapter 2 Multivariate Normal Distribution
Chapter Multivariate Normal Distribution In this chapter, we define the univariate and multivariate normal distribution density functions and then we discuss the tests of differences of means for multiple
More informationOne-way ANOVA (Single-Factor CRD)
One-way ANOVA (Single-Factor CRD) STAT:5201 Week 3: Lecture 3 1 / 23 One-way ANOVA We have already described a completed randomized design (CRD) where treatments are randomly assigned to EUs. There is
More informationNeuendorf MANOVA /MANCOVA. Model: X1 (Factor A) X2 (Factor B) X1 x X2 (Interaction) Y4. Like ANOVA/ANCOVA:
1 Neuendorf MANOVA /MANCOVA Model: X1 (Factor A) X2 (Factor B) X1 x X2 (Interaction) Y1 Y2 Y3 Y4 Like ANOVA/ANCOVA: 1. Assumes equal variance (equal covariance matrices) across cells (groups defined by
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More informationLecture 6: Single-classification multivariate ANOVA (k-group( MANOVA)
Lecture 6: Single-classification multivariate ANOVA (k-group( MANOVA) Rationale and MANOVA test statistics underlying principles MANOVA assumptions Univariate ANOVA Planned and unplanned Multivariate ANOVA
More informationSTAT 501 EXAM I NAME Spring 1999
STAT 501 EXAM I NAME Spring 1999 Instructions: You may use only your calculator and the attached tables and formula sheet. You can detach the tables and formula sheet from the rest of this exam. Show your
More informationM A N O V A. Multivariate ANOVA. Data
M A N O V A Multivariate ANOVA V. Čekanavičius, G. Murauskas 1 Data k groups; Each respondent has m measurements; Observations are from the multivariate normal distribution. No outliers. Covariance matrices
More informationI L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
Principal Analysis Edps/Soc 584 and Psych 594 Applied Multivariate Statistics Carolyn J. Anderson Department of Educational Psychology I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN c Board
More informationChapter 12. Analysis of variance
Serik Sagitov, Chalmers and GU, January 9, 016 Chapter 1. Analysis of variance Chapter 11: I = samples independent samples paired samples Chapter 1: I 3 samples of equal size J one-way layout two-way layout
More informationPrincipal component analysis
Principal component analysis Motivation i for PCA came from major-axis regression. Strong assumption: single homogeneous sample. Free of assumptions when used for exploration. Classical tests of significance
More informationMore about Single Factor Experiments
More about Single Factor Experiments 1 2 3 0 / 23 1 2 3 1 / 23 Parameter estimation Effect Model (1): Y ij = µ + A i + ɛ ij, Ji A i = 0 Estimation: µ + A i = y i. ˆµ = y..  i = y i. y.. Effect Modell
More informationSTA442/2101: Assignment 5
STA442/2101: Assignment 5 Craig Burkett Quiz on: Oct 23 rd, 2015 The questions are practice for the quiz next week, and are not to be handed in. I would like you to bring in all of the code you used to
More informationNeuendorf MANOVA /MANCOVA. Model: MAIN EFFECTS: X1 (Factor A) X2 (Factor B) INTERACTIONS : X1 x X2 (A x B Interaction) Y4. Like ANOVA/ANCOVA:
1 Neuendorf MANOVA /MANCOVA Model: MAIN EFFECTS: X1 (Factor A) X2 (Factor B) Y1 Y2 INTERACTIONS : Y3 X1 x X2 (A x B Interaction) Y4 Like ANOVA/ANCOVA: 1. Assumes equal variance (equal covariance matrices)
More informationConfidence Intervals, Testing and ANOVA Summary
Confidence Intervals, Testing and ANOVA Summary 1 One Sample Tests 1.1 One Sample z test: Mean (σ known) Let X 1,, X n a r.s. from N(µ, σ) or n > 30. Let The test statistic is H 0 : µ = µ 0. z = x µ 0
More informationSTAT 525 Fall Final exam. Tuesday December 14, 2010
STAT 525 Fall 2010 Final exam Tuesday December 14, 2010 Time: 2 hours Name (please print): Show all your work and calculations. Partial credit will be given for work that is partially correct. Points will
More informationPOWER AND TYPE I ERROR RATE COMPARISON OF MULTIVARIATE ANALYSIS OF VARIANCE
POWER AND TYPE I ERROR RATE COMPARISON OF MULTIVARIATE ANALYSIS OF VARIANCE Supported by Patrick Adebayo 1 and Ahmed Ibrahim 1 Department of Statistics, University of Ilorin, Kwara State, Nigeria Department
More information22s:152 Applied Linear Regression. Take random samples from each of m populations.
22s:152 Applied Linear Regression Chapter 8: ANOVA NOTE: We will meet in the lab on Monday October 10. One-way ANOVA Focuses on testing for differences among group means. Take random samples from each
More informationOutline. Topic 19 - Inference. The Cell Means Model. Estimates. Inference for Means Differences in cell means Contrasts. STAT Fall 2013
Topic 19 - Inference - Fall 2013 Outline Inference for Means Differences in cell means Contrasts Multiplicity Topic 19 2 The Cell Means Model Expressed numerically Y ij = µ i + ε ij where µ i is the theoretical
More informationUV Absorbance by Fish Slime
Data Set 1: UV Absorbance by Fish Slime Statistical Setting This handout describes a repeated-measures ANOVA, with two crossed amongsubjects factors and repeated-measures on a third (within-subjects) factor.
More information3. (a) (8 points) There is more than one way to correctly express the null hypothesis in matrix form. One way to state the null hypothesis is
Stat 501 Solutions and Comments on Exam 1 Spring 005-4 0-4 1. (a) (5 points) Y ~ N, -1-4 34 (b) (5 points) X (X,X ) = (5,8) ~ N ( 11.5, 0.9375 ) 3 1 (c) (10 points, for each part) (i), (ii), and (v) are
More informationVectors and Matrices Statistics with Vectors and Matrices
Vectors and Matrices Statistics with Vectors and Matrices Lecture 3 September 7, 005 Analysis Lecture #3-9/7/005 Slide 1 of 55 Today s Lecture Vectors and Matrices (Supplement A - augmented with SAS proc
More informationApplied Multivariate Analysis
Department of Mathematics and Statistics, University of Vaasa, Finland Spring 2017 Discriminant Analysis Background 1 Discriminant analysis Background General Setup for the Discriminant Analysis Descriptive
More informationSTAT 263/363: Experimental Design Winter 2016/17. Lecture 1 January 9. Why perform Design of Experiments (DOE)? There are at least two reasons:
STAT 263/363: Experimental Design Winter 206/7 Lecture January 9 Lecturer: Minyong Lee Scribe: Zachary del Rosario. Design of Experiments Why perform Design of Experiments (DOE)? There are at least two
More informationMultivariate Analysis of Variance
Chapter 15 Multivariate Analysis of Variance Jolicouer and Mosimann studied the relationship between the size and shape of painted turtles. The table below gives the length, width, and height (all in mm)
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationInference for the Regression Coefficient
Inference for the Regression Coefficient Recall, b 0 and b 1 are the estimates of the slope β 1 and intercept β 0 of population regression line. We can shows that b 0 and b 1 are the unbiased estimates
More informationCh 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
More informationSTAT 501 Assignment 2 NAME Spring Chapter 5, and Sections in Johnson & Wichern.
STAT 01 Assignment NAME Spring 00 Reading Assignment: Written Assignment: Chapter, and Sections 6.1-6.3 in Johnson & Wichern. Due Monday, February 1, in class. You should be able to do the first four problems
More informationGregory Carey, 1998 Regression & Path Analysis - 1 MULTIPLE REGRESSION AND PATH ANALYSIS
Gregory Carey, 1998 Regression & Path Analysis - 1 MULTIPLE REGRESSION AND PATH ANALYSIS Introduction Path analysis and multiple regression go hand in hand (almost). Also, it is easier to learn about multivariate
More informationMATH5745 Multivariate Methods Lecture 07
MATH5745 Multivariate Methods Lecture 07 Tests of hypothesis on covariance matrix March 16, 2018 MATH5745 Multivariate Methods Lecture 07 March 16, 2018 1 / 39 Test on covariance matrices: Introduction
More informationStatistics - Lecture 05
Statistics - Lecture 05 Nicodème Paul Faculté de médecine, Université de Strasbourg http://statnipa.appspot.com/cours/05/index.html#47 1/47 Descriptive statistics and probability Data description and graphical
More informationLecture 18: Simple Linear Regression
Lecture 18: Simple Linear Regression BIOS 553 Department of Biostatistics University of Michigan Fall 2004 The Correlation Coefficient: r The correlation coefficient (r) is a number that measures the strength
More informationMultivariate Data Analysis Notes & Solutions to Exercises 3
Notes & Solutions to Exercises 3 ) i) Measurements of cranial length x and cranial breadth x on 35 female frogs 7.683 0.90 gave x =(.860, 4.397) and S. Test the * 4.407 hypothesis that =. Using the result
More informationANOVA approaches to Repeated Measures. repeated measures MANOVA (chapter 3)
ANOVA approaches to Repeated Measures univariate repeated-measures ANOVA (chapter 2) repeated measures MANOVA (chapter 3) Assumptions Interval measurement and normally distributed errors (homogeneous across
More informationUNIVERSITY OF TORONTO Faculty of Arts and Science
UNIVERSITY OF TORONTO Faculty of Arts and Science December 2013 Final Examination STA442H1F/2101HF Methods of Applied Statistics Jerry Brunner Duration - 3 hours Aids: Calculator Model(s): Any calculator
More informationUnit 12: Analysis of Single Factor Experiments
Unit 12: Analysis of Single Factor Experiments Statistics 571: Statistical Methods Ramón V. León 7/16/2004 Unit 12 - Stat 571 - Ramón V. León 1 Introduction Chapter 8: How to compare two treatments. Chapter
More informationChapter 5: Multivariate Analysis and Repeated Measures
Chapter 5: Multivariate Analysis and Repeated Measures Multivariate -- More than one dependent variable at once. Why do it? Primarily because if you do parallel analyses on lots of outcome measures, the
More informationGLM Repeated Measures
GLM Repeated Measures Notation The GLM (general linear model) procedure provides analysis of variance when the same measurement or measurements are made several times on each subject or case (repeated
More informationCHAPTER 2 SIMPLE LINEAR REGRESSION
CHAPTER 2 SIMPLE LINEAR REGRESSION 1 Examples: 1. Amherst, MA, annual mean temperatures, 1836 1997 2. Summer mean temperatures in Mount Airy (NC) and Charleston (SC), 1948 1996 Scatterplots outliers? influential
More informationIncomplete Block Designs
Incomplete Block Designs Recall: in randomized complete block design, each of a treatments was used once within each of b blocks. In some situations, it will not be possible to use each of a treatments
More informationTentative solutions TMA4255 Applied Statistics 16 May, 2015
Norwegian University of Science and Technology Department of Mathematical Sciences Page of 9 Tentative solutions TMA455 Applied Statistics 6 May, 05 Problem Manufacturer of fertilizers a) Are these independent
More informationStevens 2. Aufl. S Multivariate Tests c
Stevens 2. Aufl. S. 200 General Linear Model Between-Subjects Factors 1,00 2,00 3,00 N 11 11 11 Effect a. Exact statistic Pillai's Trace Wilks' Lambda Hotelling's Trace Roy's Largest Root Pillai's Trace
More informationMultiple comparisons - subsequent inferences for two-way ANOVA
1 Multiple comparisons - subsequent inferences for two-way ANOVA the kinds of inferences to be made after the F tests of a two-way ANOVA depend on the results if none of the F tests lead to rejection of
More informationMath 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
More informationIntroduction to Statistical Inference Lecture 10: ANOVA, Kruskal-Wallis Test
Introduction to Statistical Inference Lecture 10: ANOVA, Kruskal-Wallis Test la Contents The two sample t-test generalizes into Analysis of Variance. In analysis of variance ANOVA the population consists
More informationMathematical Notation Math Introduction to Applied Statistics
Mathematical Notation Math 113 - Introduction to Applied Statistics Name : Use Word or WordPerfect to recreate the following documents. Each article is worth 10 points and should be emailed to the instructor
More informationCorrelation and the Analysis of Variance Approach to Simple Linear Regression
Correlation and the Analysis of Variance Approach to Simple Linear Regression Biometry 755 Spring 2009 Correlation and the Analysis of Variance Approach to Simple Linear Regression p. 1/35 Correlation
More information