MULTIVARIATE ANALYSIS OF VARIANCE

MULTIVARIATE ANALYSIS OF VARIANCE RAJENDER PARSAD AND L.M. BHAR Indian Agricultural Statistics Research Institute Library Avenue, New Delhi - 0 0 lmb@iasri.res.in. Introduction In many agricultural experiments, generally the data on more than one character is observed. One common example is grain yield and straw yield. The other characters on which the data is generally observed are the plant height, number of green leaves, germination count, etc. The analysis is normally done only on the grain yield and the best treatment is identified on the basis of this character alone. The straw yield is generally not taken into account. If we see the system as a whole, the straw yield is also important either for the cattle feed or for mulching or manuring, etc. Therefore, while analyzing the data, the straw yield should also be taken into consideration. Similarly, in varietal trials also the data is collected on several plant characteristics and quality parameters. In these experimental situations also the data is generally analyzed separately for each of the characters. The best treatment or genotype is identified separately for each of the characters. In these situations, Multivariate Analysis of Variance (MANOVA) can be helpful. Before discussing about MANOVA, a brief description about Analysis of Variance (ANOVA) is given in Section. A general procedure of performing MANOVA on the data generated from RCB design is given in Section 3. The procedure of MANOVA has been illustrated with the help of an example in Section 4.. Overview of ANOVA The ANOVA looks at the variance within classes relative to the overall variance. The dependent variable must be metric, and the independent variables, which can be many, must be nominal. ANOVA is used to uncover the main and interaction effects of categorical independent variables (called "factors") on an interval dependent variable. A main effect is the direct effect of an independent variable on the dependent variable. An interaction effect is the joint effect of two or more independent variables on the dependent variable. Whereas regression models cannot handle interaction unless explicit crossproduct interaction terms are added, ANOVA uncovers interaction effects on a builtin basis. The key statistic in ANOVA is the F-test of difference of group means, testing if the means of the groups formed by values of the independent variable (or combinations of values for multiple independent variables) are different enough not to have occurred by chance. If the group means do not differ significantly then it is inferred that the independent variable(s) did not have an effect on the dependent variable. If the F test shows that overall the independent variable(s) is (are) related to the dependent variable, then multiple comparison tests of significance are used to explore just which values of the independent(s) have the most to do with the relationship. 3. Multivariate Analysis of Variance Why do MANOVA, when one can get also much more information by doing a series of ANOVAs? Even if all our dependent variables are completely independent of one another,

when we do lots of tests like that, error inflates. But in many ecological or biological studies, the variables are not independent at all. Many times they have strong actual or potential interactions, inflating the error even more highly. In many cases where multiple ANOVAs were done, MANOVA was actually the more appropriate test. Consider an experiment conducted to compare v treatments using a randomized complete block (RCB) design with r replications and the data is collected on p-variables. Let y denote the observed value of the k th response variable for the i th treatment in the j th ijk replication i =,,..., v; j=,,..., r;k=,,..., p. The data is rearranged as follows: Replications Treatments j r Treatment Mean y y y j y r y. y y y j y r y. i y i y i y ij y ir y i. v y v y v y v y v y v. Replication y. y. y. j y. r y.. Mean Here y = ( y y...y... y ) is a p-variate vector of observations taken from the plot ij ij ij receiving the treatment i in replication j. ijk r v v r y i. = yij ; y.j = yij and.. = r v vr j= ijp i= y y. i= j= The observations can be represented by a two way classified multivariate model Ω Ω : y = µ + t + b + e i =,,,v; j =,,,b, (3.) ij i j ij µ = (µ µ µ k µ p ) is the p vector of general means, t i = (t i t i t ik t ip ) are the effects of treatment i on p-characters, and b j =(b j b j b jk b jp ) are the effects of replication j on p-characters. e ij = (e ij e ij e ijk e ijp ) is a p-variate random vector associated with y ij and assumed to be distributed independently as p variate normal distribution N p ( 0, Σ). The equality of treatment effects is to be tested i.e. H 0 : (t i t i t ik t ip ) = (t t t k t p ) (say) i=,,, p against the alternative H : at least two of the treatment effects are unequal. Under the null hypothesis, the model (3.) reduces to Ω : y = α+ b + e (3.) 0 ij j ij ( µ + t µ + t,...,µ p+ t p where α = ). An outline of MANOVA Table for testing the equality of treatment effects and replication effects is ij

MANOVA Source DF SSCPM (Sum of Squares and Cross Product Matrix) Treatment v- = h v H = = ( )( ) b i y i. y.. yi. y.. Replication r- = t b B= v = ( y y )( y y ) Residual Total (v-)(r-) = s vr- j.j...j.. v b R= ( y y y + y )( y y y + y ) i = j = ij i..j.. ij i..j.. v b T= ( y y )( y y ) i = j = ij.. ij.. =H+B+R Here H, B, R and T are the sum of squares and sum of cross product matrices of treatments, replications, errors (residuals) and totals respectively. The residual sum of squares and cross products matrix for the reduced model Ω 0 is denoted by R 0 and is given by R 0 = R+ H. The null hypothesis of equality of treatment mean vectors is rejected if the ratio of R generalized variance (Wilk's lambda statistic) Λ = is too small. Assuming the H+ R normal distribution, Rao (973) showed that under null hypothesis Λ is distributed as the product of independent beta variables. A better but more complicated approximation of the distribution of Λ is Λ Λ / b / b (ab c) ~ F (ph, ab-c) ph where a, b {( p h 4) /( p + h 5) } p h+ = s =, c= ph For some particular values of h and p, it reduces to exact F-distribution. The special cases are given below: For h = and any p, this reduces to For h= and any p, it reduces to ( Λ)(s p+ ) ~ F (p, s p + ) Λ p ( Λ)(s p+ ) ~ F (p, (s p + )) Λ p ( Λ)(s ) For p= and any h: ~ F (h, (s )). Λ h For p =, the statistic reduces to the usual variance ratio statistics. The hypothesis regarding the equality of replication effects can be tested by replacing Λ R by and h by t in the above. B+ R 3

Several other criteria viz. Pillai's Trace, Hotelling-Lawley Trace or Roy's Greatest Root are available in literature for testing the null hypothesis in MANOVA. Wilks' Lamda is, however, the commonly used criterion. Here, we shall restrict to the use of Wilks' Lamda criterion. For further details on MANOVA, a reference may be made to Seber (983) and Johnson and Wichern (988). Remark 3.: One complication of multivariate analysis that does not arise in the univariate case is the ranks of the matrices. The rank of R should not be smaller than p or in other words error degrees of freedom s should be greater than or equal to p (s p). 3. Multivariate Treatment Contrast Analysis If the treatments are found to be significantly different through MANOVA, then the next question is which treatments are significantly different? This question can be answered through multivariate treatment contrast analysis. In the literature, the multivariate treatment contrast analysis is generally carried out using the χ -statistic. The χ -statistic is based on the assumption that the error variance-covariance matrix is known. The error variance-covariance matrix is, however, generally unknown. Therefore, the estimated value of error variance-covariance matrix is used. The error variance-covariance matrix is estimated by sum of squares and cross products (SSCP) matrix for error divided by the error degrees of freedom. As a consequence, test based on χ -statistic is an approximate solution. The procedure using the Wilk s Lambda criterion is also described in the sequel. Suppose the hypothesis to be tested is H 0 : be rewritten as t = against H : t. This hypothesis can i t i ' i t i ' H 0 : = ( t ' ) = 0 against H : = ( t ' ) 0, i t i i t i i t i t i i i i ik i k ip i p t = ( t t t... t t... t t ) (3.3) where ( ' ). Here t ik denote the effect of treatment i for the dependent variable k. The best linear unbiased estimate of ( ti t i ' ) is ( ) y y = ( y y y y... y y... y y ) i. i. i i i i ik i k ip ' where y ik is the mean of treatment i for variable k. 3.. χ Test The statistic based on χ, requires covariance matrix of the contrast of interest. The covariance matrix, in case of a RCB design for elementary treatment contrast is obtained by dividing the SSCP matrix for errors obtained in MANOVA by half of the product of error degrees of freedom and the number of replications. Let this variance-covariance matrix is denoted by Σ c. Under null hypothesis, x = yi. y ' follows p- variate normal i. distribution with mean vector 0 and variance-covariance matrix Σ c. Applying the Aitken's / transformation, it can be shown that z= Σc x follows a p-variate normal distribution with mean vector 0 and variance-covariance matrix I g, where I g, denotes the identity matrix of order g. Then using the results of quadratic forms, it can easily be seen that z z x = Σ x follows a χ distribution with p-degrees of freedom. i p 4

3.. Wilk s Lambda Criterion For testing the null hypothesis (3.3), we obtain a sum of squares and products matrix for the above elementary treatment contrast. Let the SSCP matrix for above elementary treatment contrast be G p p. The diagonal elements of G are then obtained by r g kk = ( yik yi'k) k=,,...,p; i i' =,,..., v (3.4) and the off diagonal elements are obtained by r g kk' = ( y ik y i'k )( y ik' y i'k' ) (3.5) R The null hypothesis is rejected if the value of Wilk's Lambda Λ* = is small, G+ R where R is the SSCP matrix due to residuals as obtained through MANOVA. The hypothesis is then tested using the following F-test statistics based on Wilk's Lambda for h = Λ * edf p+ F(p, s-p+). Λ * p 4. Illustration using Multivariate Techniques In this section, the results obtained from bivariate analysis of variance of the data generated from the experiments conducted under PDCSR are given, where the data on grain yield and straw yield were observed. Illustration 4.: An experiment entitled Studies on the experimentation on conservation of organic carbon in the soil to improve soil condition was conducted at Bhubaneshwar on rice crop. The experiment was initiated in the year 997. The data on grain and straw used for the illustration pertains to the Kharif season of 00. Ten treatments were tried in the experiment. The details of the treatments are given below: T - Recommended N 00% T - Recommended N 00% out of which 0 Kg at first ploughing T3 - Recommended N 00% out of which 0 Kg at first ploughing T4 - Recommended N 00% and add 0 Kg N/ha at first ploughing T5 - Recommended N 00% and add 0 Kg N/ha at first ploughing T6 - Recommended N + 0 Kg N/ha T7 - Recommended N + 0 Kg N/ha T8 - Recommended N + cellulose decomposing enzyme (FYM) T9 - Recommended N + FYM 5 t/ha during Kharif T0 - Recommended N + FYM 5 t/ha during Rabi The results of multivariate analysis of variance are given in the sequel. First the results for each of the two characters are presented separately. ANOVA: Grain Yield (GYLD) Source DF SS MS F Value Pr > F Model 04.6889 8.74 5.94 <.000 Error 7 39.6605.4689 Total 39 44.3494 5

R-Square CV Root MSE GYLD Mean 0.75.9895. 9.3305 Source DF SS MS F -ratio Pr > F REP 3.73 0.904 0.6 0.6083 TRT 9 0.9577.386 7.7 <.000 ANOVA: Straw Yield (SYLD) Source DF SS MS F -ratio Pr > F Model 6.93 3.436900 6.88 <.000 Error 7 5.744.95385 Total 39 3.9067 R-Square CV RMSE SYLD Mean 0.7536.6574.39737.0393 Source DF SS MS F -ratio Pr > F REP 3 4.449.4806 0.76 0.573 TRT 9 56.7503 7.467 8.9 <.000 It can be seen that for both the characters, the replication effects are not significantly different whereas the treatments are significantly different. Therefore, for making all possible paired comparisons, the least significant difference procedure of multiple comparisons was used. The results are given in the sequel: t Tests (LSD) for GYLD Alpha 0.05 Error Degrees of Freedom 7 Error Mean Square.4689 Critical Value of t.058 Least Significant Difference at 5%.7584 t Grouping Mean N TRT A 0.8000 4 8 A 0.6000 4 7 A 0.5575 4 6 A 0.545 4 9 A 0.55 4 0 A 9.835 4 5 A 9.50 4 4 B C 8.055 4 3 D C 7.3550 4 D 5.9375 4 *The treatments with the same alphabet are not significantly different. 6

t Tests (LSD) for SYLD Alpha 0.05 Error Degrees of Freedom 7 Error Mean Square.95385 Critical Value of t.0583 Least Significant Difference.073 t Grouping Mean N TRT A.9375 4 7 A.7750 4 6 B A.4375 4 9 B A.3550 4 0 B A.65 4 8 B A C.7975 4 5 B C 0.745 4 4 D C 9.8875 4 3 E D 8.575 4 E 6.6700 4 *The treatments with the same alphabet are not significantly different. It can be concluded that the treatment T8 is at Rank for GYLD and T7 gets rank for SYLD, although the two treatments are not significantly different among themselves. The treatments T4 and T are not significantly different for GYLD and significantly different for SYLD. Therefore, to rank the treatments collectively for both the characters, the multivariate analysis of variance was carried out. The results obtained are given below: Multivariate Analysis of Variance E = Error SSCP Matrix gyld syld gyld 39.6605 44.60303 syld 44.60303 5.743975 Partial Correlation Coefficients from the Error SSCP Matrix / Prob > r DF = 7 gyld syld gyld.000000 0.975485 <.000 syld 0.975485.00000 <.000 MANOVA Test Criteria and F Approximations for the Hypothesis of No Overall TRT Effects. Statistic Value F-ratio Num DF Den DF Pr > F Wilks' Lambda 0.997 5.45 8 5 <.000 Pillai's Trace.550 5.05 8 54 <.000 Hotelling-Lawley Trace 4.00 5.9 8 39.74 <.000 Roy's Greatest Root 3.475 9.74 9 7 <.000 7

From the above Table, it can be concluded that the treatment effects are significantly different. Now the next question is that what is ranking of treatments? Which treatments are significantly different? This can be achieved through multivariate contrast analysis. However, most of the software, carry out the univariate contrast analysis on the combined average values of all the dependent variables. To account for the correlation structure between the two variables, the principal component analysis was carried out. The results are: Eigenvalues of the Correlation Matrix Eigenvalue Difference Proportion Cumulative.984808.96856056 0.99 0.99 0.05797 0.0079.0000 Prin Prin gyld 0.70707 0.70707 syld 0.70707-0.70707 Eigenvectors It can be seen that the first principal component explains 99.% of the variance. Therefore, the principal component scores of the observations for the first principal components are obtained and the univariate analysis of variance was carried out. The results obtained are: ANOVA: Principal Component Scores Source DF SS MS F -ratio Pr > F Model 6.948.7746 6.48 <.000 Error 7 90.7905 3.366 Corrected Total 39 35.0854 R-Square CV Root MSE Prin Mean 0.74.73.8337 4.4036 Source DF SS MS F -ratio Pr > F REP 3 6.7790.597 0.67 0.5767 TRT 9 54.558 8.795 8.4 <.000 It can be seen that the treatments are highly significantly different. Therefore, multiple comparisons using the least significant difference procedure was used. t Tests (LSD) for prin Alpha 0.05 Error Degrees of Freedom 7 Error Mean Square 3.366 Critical Value of t.058 Least Significant Difference.6605 8

t Grouping Mean N TRT A 6.644 4 7 A 6.499 4 6 A 6.308 4 8 A 6.49 4 9 A 6.70 4 0 B A 5.95 4 5 B A 4.04 4 4 B C.685 4 3 D C.3 4 D 8.95 4 The treatment T7 gets the first rank and is non-significantly different from T8. The treatments T4 and T are significantly different among themselves. This procedure answers the question to some extent. But a multivariate contrasts analysis is the best answer for this situation. The results of multivariate treatment contrast analysis for making all possible paired comparisons of the treatments are given in the sequel. Probabilities of Significance of All Possible Paired Treatment Comparisons using Wilks' Lamda Criterion 3 4 5 6 7 8 9 0 Treats. 0.55. 3 0.0006 0.0388. 4 0.000 0.0938 0.673. 5 0.000 0.0055 0.35 0.3945. 6 0.000 0.0004 0.070 0.063 0.5497. 7 0.000 0.000 0.094 0.053 0.37 0.874. 8 0.000 0.000 0.0006 0.053 0.000 0.0058 0.007. 9 <0.000 0.003 0.08 0.604 0.5636 0.3653 0.657 0.64. 0 <0.000 0.0030 0.059 0.904 0.4866 0.667 0.3 0.88 0.9755. *bold face type shows the treatment pairs that are not significantly different. From the above results, it is seen that treatments T7 and T8 are significantly different where as they were found to be not significantly different when analyzed for individual characters or st principal component score was used. Note: The MANOVA described in Sections and 3 can usefully be employed for the experimental situations where the experiment is continued for several years/ seasons with same treatments and same randomized layout. For a detailed discussion on this one may refer to Parsad et al. (004). 9

SPSS Commands for MANOVA. Enter the data. Click Analyze General Linear Model Multivariate 3. Put dependent variables (grain, straw) in Dependent Variables box and independent variables (treat., rep) in Fixed Factors box 4. Then Click model Custom and bring the independent variables in model box. Then click continue 0

5. If you want contrast analysis, then click contrast and mark the variables for which you want contrast analysis otherwise click continue 6. If you want post-hoc analysis, then click post hoc and bring the required variables in Post hoc test for box and then click continue 7. For other statistics, then click Options and for diagnostics results, then click Save

8. Finally click OK button and get the results References and Suggested Reading Johnson, R.A. and Wichern, D.W. (988). Applied Multivariate Statistical Analysis, nd Edition. Prentice-Hall International, Inc., London. Parsad, R., Gupta, V.K., Batra, P.K., Srivastava, R., Kaur, R., Kaur, A. and Arya, P. (004). A diagnostic study of design and analysis of field exeriments. Project Report, IASRI, New Delhi. Rao, C.R.(973). Linear Statistical Inference and Application. Wiley Eastern Ltd., New Delhi. Seber, G.A.F.(983). Multivariate Observations. Wiley series in Probability and Statistics.