Applied Multivariate and Longitudinal Data Analysis

Size: px

Start display at page:

Download "Applied Multivariate and Longitudinal Data Analysis"

Brenda Webster
5 years ago
Views:

1 Applied Multivariate and Longitudinal Data Analysis Chapter 2: Inference about the mean vector(s) II Ana-Maria Staicu SAS Hall 5220; ; 1

2 1 Compare Means from More Than Two Populations 1.1 Multivariate Analysis of Variance (one-way MANOVA) Wisconsin nursing home (Johnson and Wichern, 2007 ). The Wisconsin Dept of Health and Social Services reimburses nursing homes in the state for the services provided. The department develops a set of formulas for rates for each facility based on factors such as level of care, mean wage rate, and average wage rate in the state. Nursing homes can be classified on the basis of ownership (private, nonprofit, government). One purpose of a recent study was to investigate the effects of ownership on costs. Four costs computed on a per-patient day basis and measured in hours per patient day were selected for analysis: X 1 = cost of nursing labor, X 2 = cost of dietary labor, X 3 = cost of plant operation and maintenance labor, X 4 = cost of housekeeping and laundry labor. A total of n = 516 observations on each of p = 4 cost variables were initially separated according to ownership: n 1 = 271, n 2 = 138, n 3 = 107. Goal: evaluate the effects of ownership on costs. Review of univariate one-way analysis of variance or one-way ANOVA: Assumptions: independent populations; equal variances σ 1 =... = σ g ; normal; g 2 Want to test H 0 : µ 1 =... = µ g X 11,... X 1n1 N(µ 1, σ 2 1) X 21,... X 2n2 N(µ 2, σ 2 2)... X g1,... X gng N(µ g, σ 2 g) One-way ANOVA from a model perspective: write µ l = µ + τ l and thus posit the model X lj = µ + τ l + ɛ lj, l = 1,..., g; j = 1,..., n l with the constraint n l τ l = 0; here ɛ lj N(0, σ 2 ). 2

3 One-way ANOVA Table: Source of variation Sum of squares Degrees of freedom Treatment SSTr = n l ( x l x) 2 g 1 Residual (error) SSE = Total SST = H 0 : τ 1 =... = τ g = 0 Test statistic: F = n l (x lj x l ) 2 g n l g j=1 n l (x lj x) 2 g n l 1 j=1 SSTr/(g 1) SSE/( g n l g) F g 1, g n l g. Tell your neighbor what is x l? and what is x? Reject H 0 if F > F g 1, g n l g(α) Intuitively, we reject H 0 when SSTr/SSE is very large, or equivalently, SSE/SST is small. The multivariate version of ANOVA uses a direct generalization of this latter statistics to higher dimensions. 3

4 One-way MANOVA Assumptions: independent populations; equal covariances Σ 1 =... = Σ g = Σ; multivariate normal; g 2 X 11,... X 1n1 N p (µ 1, Σ) X 21,... X 2n2 N p (µ 2, Σ)... X g1,... X gng N p (µ g, Σ) H 0 : µ 1 =... = µ g One-way MANOVA model: X lj = µ + τ l + ɛ lj, l = 1,..., g; j = 1,..., n l with the constraint n l τ l = 0; here ɛ lj N p (0, Σ). Thus the above null hypothesis is equivalent to H 0 : τ 1 =... = τ g = 0 One-way MANOVA Table: Source Sum of squares and Df cross-products matrix Treatment B = n l ( x l x)( x l x) T g 1 Residual W = n l (x lj x l )(x lj x l ) T g n l g Total B + W = j=1 n l (x lj x)(x lj x) T g n l 1 j=1 To test H 0 : µ 1 =... = µ g use the statistic (Wilks lambda): Λ = W B + W = 1 W 1 B + I p the null distribution of this statistic can be derived in special cases of p and g. For other cases, a modification of Λ is used along with large sample approximations, which are in the F -family. For example when p = 2 and g 2 the test statistic is nl g 1 g 1 and its null distribution is F 2(g 1),2( nl g 1). 1 Λ ; Λ 4

5 Remark. We reject the null if Λ is very small. The Wilks lambda statistics is related to the likelihood ratio statistics. The Wilks lambda statistics is a multivariate generalization of the univariate F-distribution, generalizing the F-distribution in the same way that the Hotelling s T 2 distribution generalizes Student s t-distribution. There are other forms of test statistics: Pillai s statistic, the Lawley-Hotelling statistic, and Roy s largest root statistic. These statistics can be written as particular functions (of the eigenvalues) of W 1 B. Remark For protection against nonnormality and heterogeneity of covariance matrices, the largestroot test should be avoided, while the Pillai-Bartlett trace (Pillai s) test may be recommended as the most robust of the MANOVA tests, with adequate power to detect true differences in a variety of situations (Olson, 1974). Remark For visual assessment use covellipses() function in the heplots package. For more details see Visualizing Tests for Equality of Covariance Matrices by Michael Friendly and Matthew Sigal. Pairwise treatment effect comparison: Question: If we reject H 0, then which effects led to the rejection? Target: c kl = µ l µ k = τ l τ k for l, k = 1,..., g; l < k Estimates of treatment effects: τ l = x l x for l = 1,..., g Estimates of pairwise differences: ĉ kl = x l x k Bonferroni-corrected confidence interval (simultaneous inference): ( ) ( α W jj 1 ( x lj x kj ) ± t n g + 1 ), pg(g 1) n g n l n k l = 1,..., g, j = 1,..., p, where W jj is the j-th diagonal element of W and n = n n g. In class exercise: Why the upper tail probability used in determining the critical value of t is α/{pg(g 1)}? 5

6 1.2 Two-way Multivariate Analysis of Variance (two-way MANOVA) Extrusion of plastic film example (Johnson and Wichern, 2007 ). The optimum conditions for extruding plastic film have been examined using a technique called Evolutionary Operations. In the course of the study, three responses X 1 -tear resistance, X 2 -gloss, and X 3 -opacity, were measured at two levels of the factors rate of extrusion and amount of an additive. The measurements were repeated five times at each combinations of the factor levels. 3 variables: X 1 = tear resistance, X 2 = gloss, X 3 = opacity 2 factors: change in rate of extrusion (low/high); amount of additive (low/high) Number or replications for each combinations of factor levels = 5 Goal: to evaluate main effects of the two factors and their interaction Univariate two-way ANOVA from a model perspective: two-way fixed-effects model with interaction There are g levels of factor 1 There are b levels of factor 2 We observe n independent observations for each of the gb combinations of levels Denote the rth observation at level l of factor 1 and level k of factor 2 by X lkr Population mean at level l of factor 1 and level k of factor 2 is denoted by µ lk Decomposition µ lk = µ + τ l + β k + γ lk, where µ is the overall mean, τ l is the fixed effect of factor 1, β k is the fixed effect of factor 2, and γ lk is the interaction between the two factors We need constraints τ l = β k = γ lk = γ lk = 0 k=1 k=1 6

7 Two-way ANOVA Table: Source of variation Sum of squares Degrees of freedom Factor 1 SS fac1 = bn( x l x) 2 g 1 Factor 2 SS fac2 = Interaction SS int = Residual SS res = Total (Corrected) SS c.tot = gn( x k x) 2 b 1 k=1 k=1 n(x lk x l x k + x) 2 (g 1)(b 1) k=1 r=1 k=1 r=1 n (x lkr x lk ) 2 gb(n 1) n (x lkr x) 2 gbn 1 where the subscript reflects the sum of squares for the main factors, the interactions, the residual and the total (corrected). To test the main effect of factor 1; of factor 2; or the interaction between the two use the test statistics: SS fac1 /(g 1) SS res /{gb(n 1)}, SS fac2 /(b 1) SS res /{gb(n 1)}, SS int /{(g 1)(b 1)} SS res /{gb(n 1)} These tests statsitics follow an F distribution with corresponding degrees of freedom, when the null hypothesis is true. Two-way MANOVA Table is obtained by replacing ( x l x) 2 with ( x l x)( x l x) T, and so on... Denote SS (=sum of square) by SSP (=matrix for sum of squares and cross-products) The analogous test statistics for testing the same null hypothesis are: Λ fac1 = Λ int = SSP res SSP fac1 + SSP res, Λ fac2 = SSP res SSP int + SSP res SSP res SSP fac2 + SSP res, their null distributions is far more complex and will not be given here. 7

8 Some Discussion One multivariate test vs. p univariate tests Individual tests ignore correlation among the p variables, and may give misleading results A single multivariate test is preferable over p univariate tests The result determines whether one should look closer (individual variables or groups) MANOVA: different test statistics Wilks lambda, Lawley-Hotelling trace, Pillai s trace, Roy s largest square root All these tests are nearly equivalent for very large sample sizes For moderate sample sizes, the first three tests behave similarly MANOVA: non-normality and unequal covariance matrices For large sample sizes, non-normality has little effect on the tests If the sample sizes are equal in each group, some difference in the covariance matrices has little effect on tests in MANOVA 8

Lecture 5: Hypothesis tests for more than one sample

1/23 Lecture 5: Hypothesis tests for more than one sample Måns Thulin Department of Mathematics, Uppsala University thulin@math.uu.se Multivariate Methods 8/4 2011 2/23 Outline Paired comparisons Repeated