Journal of Modern Applied Statistial Methods Volume Issue Artile 3 5--003 Performing Two-Way Analysis of Variane Under Variane Heterogeneity Sott J. Rihter University of North Carolina at Greensboro, sjriht@ung.edu Mark E. Payton Oklahoma State University, mpayton@okstate.edu Follow this and additional works at: http://digitalommons.wayne.edu/jmasm Part of the Applied Statistis Commons, Soial and Behavioral Sienes Commons, and the Statistial Theory Commons Reommended Citation Rihter, Sott J. and Payton, Mark E. (003) "Performing Two-Way Analysis of Variane Under Variane Heterogeneity," Journal of Modern Applied Statistial Methods: Vol. : Iss., Artile 3. DOI: 0.37/jmasm/05747980 Available at: http://digitalommons.wayne.edu/jmasm/vol/iss/3 This Regular Artile is brought to you for free and open aess by the Open Aess Journals at DigitalCommons@WayneState. It has been aepted for inlusion in Journal of Modern Applied Statistial Methods by an authorized editor of DigitalCommons@WayneState.
Journal of Modern Applied Statistial Methods Copyright 003 JMASM, In. May 003, Vol., No., 5-60 538 947/03/$30.00 Performing Two-Way Analysis of Variane Under Variane Heterogeneity Sott J. Rihter Department of Mathematial Sienes University of North Carolina at Greensboro Mark E. Payton Department of Statistis Oklahoma State University Small sample properties of the method proposed by Brunner et al. (997) for performing two-way analysis of variane are ompared to those of the normal based ANOVA method for fatorial arrangements. Different effet sizes, sample sizes, and error strutures are utilized in a simulation study to ompare type I error rates and power of the two methods. An SAS program is also presented to assist those wishing to implement the Brunner method to real data. Key words: Fatorial arrangement of treatments, heterogeneity of variane Introdution Normal theory methods for analysis of variane depend on the assumption of homogeneity of the variane of the error distribution. For a one-way treatment struture, modifiations are available when the homogeneity of variane assumption is violated. Milliken and Johnson (99) suggest a method due to Box (954) when sample sizes are equal. When samples sizes are unequal, they suggest Welh's (95) test. For multifator layouts, however, there are few options available for testing effets of interation and main effets. A parametri approah to this problem was presented by Weerahandi (995), but it requires omplex and intensive omputing and isn t yet pratial for use on real data. Papers by Akritas (990), Thompson (99) and Akritas and Arnold (994) present nonparametri rank test statistis in a multi-way ANOVA setting. One should see Brunner, et al. (997) for a survey of referenes relating to this topi. Sott Rihter is an assistant professor in the Mathematial Sienes Department at the University of North Carolina at Greensboro. His email address is sjriht@ung.edu. Mark Payton is a professor in the Department of Statistis at Oklahoma State University. His email address is mpayton@okstate.edu. One method that does not require the equal variane assumption is based on a Wald statisti, whih has an asymptoti hi-square distribution. This method tends to rejet too frequently under the null hypothesis for small samples. In fat, simulations of Brunner, et al. (997) show the test to be liberal (by as muh as 0.05) for small to moderate sample sizes, and they suggest a small sample improvement over the Wald statisti. Their approah is to use a generalization of hi-square approximations dating bak to Patnaik (949) and Box (954). Simulation results indiate that this adjustment greatly improves the performane of the Wald statisti, and is effetive for sample sizes as small as n=7 per fator ombination. They also point out that for equal sample sizes, their statisti is idential to the lassial ANOVA F-statisti, and thus their method an be regarded as a robust extension of the lassial ANOVA to heterosedasti designs. They reommend that their method should always be preferred (even in the homosedasti ase) to the lassial ANOVA. However, they do not investigate how the performane of their statisti ompares to the ANOVA F-statisti. In this paper, we present results of a simulation study omparing the performane of the Brunner statisti to the ANOVA F-statisti, make a reommendation for the Brunner statisti for moderate sample sizes ( n 7 ), and also present a SAS program (SAS Institute, Cary, N.C.) for implementing the method. 5
53 TWO-WAY ANALYSIS OF VARIANCE UNDER HETEROGENEITY Brunner Method The method of Brunner et al. (997) is a small sample adjustment to the well-known Wald statisti, whih permits heterogeneous variane but is known to have inflated Type I error rates for small sample sizes. Consider a two-way layout a levels of fator A and b levels of fator B. Assume a set of independent random variables X N( µ, σ ), i=,..., ab. ij i i Let µ = ( µ, µ,..., µ ab ) denote the vetor ontaining the a bpopulation means. Then the hypotheses of no main effets and interation an be written as where H ( ): 0 0 A M Aµ = H ( ): 0 0 B M Bµ = H ( ): 0 0 AB M ABµ = M = P J b M = J a P M = P P A a b B a b AB a b. Here, P = I J, where I is a identity matrix, J a matrix of s, and the symbol represents the Kroneker produt of the matries. The vetor of observed ell means is denoted by = (,..., ab ) X X X and the estimated ovariane matrix is given by ˆ S S ab S N = N diag,...,, where n n ab sample variane and ab N = n. i= i Si is the i th For a omplete ross-lassifiation, the test N XMX statisti is FB =, whih has an tr ( ) ( Sˆ N ) n approximate F distribution with f f num den = tr = tr ( n ) tr ( S N ) ( MSˆ ˆ NMSN) ˆ tr ( Sˆ ) N ( S ˆ N? ) numerator and denominator degrees of freedom, where Λ= diag,..., n nab (Brunner, 997). Results A simulation study was performed using SAS version 8.0 for a two-way layout with a= 4 and b= 3, for various sample sizes. The model used for all simulations was Y = a + b + ab + ε ijk i j ij ijk i =,,3,4, j =,,3, k = n N,..., ij, εijk (0, σij) The lassial F test from ANOVA (denoted by F), assuming normality and equal varianes, and the adjusted F-test (denoted by FB) of Brunner, et al. (997) were alulated for 5000 samples and the probabilities of rejetion estimated using an α = 0.05. Differenes in Type I error rates and powers are investigated for different sample sizes, effet sizes, and variane strutures. Case : Homogeneous errors, equal sample sizes. For this ase, we let k =,..., n, εijk N(0, σi ). Table shows nominal Type I error rate for both methods, for various sample sizes. Note that the FB statisti,.
RICHTER & PAYTON 54 underestimates the nominal level when n is small, but for sample size as small as n = 7, the nominal rates are omparable to the lassial ANOVA test. As sample size inreases beyond n = 7, the nominal rate remains stable near the target α = 0.05. Tables and 3 give proportion of rejetions when fator A effet is present, and when both main effets are present, respetively, for n = 3 and n = 7. When n = 3, the test based on the FB statisti has less power than the F statisti, and underestimates the nominal rate, espeially for the test of interation and when the effet size is small. When n = 7, power and nominal rate are very similar, with the exeption that the nominal rate for interation is still a bit too low. Table 4 shows that when interation only is present, the FB statisti again has less power for the small sample size ase. When the sample size is n = 7, power is omparable for both tests, espeially when effet sizes are not very small. Table. Proportion of rejetions at α = 0.05, normally distributed errors, equal variane, based on 5000 samples, no effets present, equal ell sample sizes. n Test for: Method 3 5 7 0 0 Main Effet A F.049.0496.0478.048.0494.05 FB.030.084.04.0448.046.05 Main Effet B F.0466.05.056.0530.05.0466 FB.04.0360.0448.050.050.0466 Interation F.0458.0470.0474.05.053.0488 FB.0086.0.036.040.0456.046 Table. Proportion of rejetions at α = 0.05, normally distributed errors, equal variane, based on 5000 samples, fator A effet present (a =, a 3 =-), equal ell sample sizes. n = 3 n = 7 Test for: Method.5.0.5.5.0.5 Main Effet A F.3446.930.000.7530.9998.000 FB.64.8876.999.7370.9998.000 Main Effet B F.05.05.05.0530.0530.0530 FB.0360.0360.0360.050.050.050 Interation F.0470.0470.0470.05.05.05 FB.0.0.0.040.040.040
55 TWO-WAY ANALYSIS OF VARIANCE UNDER HETEROGENEITY Table 3. Proportion of rejetions at α = 0.05, normally distributed errors, equal variane, based on 5000 samples, fator A and B effets present (a =b =, a 3 =b =-), equal ell sample sizes. n = 3 n = 7 Test for: Method.5.0.5.5.0.5 Main Effet A F.3440.94.9998.74.000.000 FB.604.8780.9986.776.000.000 Main Effet B F.568.990.000.940.000.000 FB.4576.9830.000.900.000.000 Interation F.0470.0470.0470.05.05.05 FB.0.0.0.040.040.040 Table 4. Proportion of rejetions at α = 0.05, normally distributed errors, equal variane, based on 5000 samples, interation effet present (ab =ab 33 =, ab 3 =ab 3 =-), equal ell sample sizes. n = 3 n = 7 Test for: Method.5.0.5.5.0.5 Main Effet A F.0496.0496.0496.048.048.048 FB.084.084.084.0448.0448.0448 Main Effet B F.05.05.05.0530.0530.0530 FB.0360.0360.0360.050.050.050 Interation F.584.5976.9460.476.988.000 FB.084.4368.8734.3864.976.000 Case : Heterogeneous errors, equal sample sizes. Here we onsider: k = n N = + i j,,...,, εijk (0, σij ( * /) ) (errors inreasing with the levels of A). Tables 5, 6 and 7 are heterogeneous analogs to Tables, 3 and 4, respetively. They ompare the tests under variane heterogeneity. Note that the lassial F- test shows inflated nominal rates for all effets, with the test for interation the most inflated. The inflation beomes more severe as the ratio between smallest and largest varianes beomes larger. The test using the Box-type adjustment, however, maintains the orret nominal rate in all onditions onsidered.
RICHTER & PAYTON 56 Table 5. Proportion of rejetions at α = 0.05, normally distributed errors with unequal variane (variane inreasing with fator A levels, ratio of largest to smallest variane 0 to ), based on 5000 samples, fator A effet present (a =, a 3 =-), equal ell sample size: n i =7. Test for: Method 0.5.5.5 Main Effet A F.059.684.958.9998 FB.0490.384.966.9998 Main Effet B F.0564.0564.0564.0564 FB.048.048.048.048 Interation F.078.078.078.078 FB.0486.0486.0486.0496 Table 6. Proportion of rejetions at α = 0.05, normally distributed errors with unequal variane (variane inreasing with fator A levels, ratio of largest to smallest variane to ), based on 5000 samples, fator A effet present (a =, a 3 =-), equal ell sample size: n i =7. Test for: Method 0.5.5.5 Main Effet A F.065.008.534.967 FB.0488.0750.4408.939 Main Effet B F.06.06.06.06 FB.0488.0488.0488.0488 Interation F.084.084.084.084 FB.0494.0494.0494.0494 Table 7. Proportion of rejetions at α = 0.05, normally distributed errors with unequal variane (variane inreasing with fator A levels, ratio of largest to smallest variane to ), based on 5000 samples, fator A and B effets present (a =b =, a 3 =b =-), equal ell sample size: n i =7. Test for: Method.5.5.5 Main Effet A F.030.534.958 FB.0784.44.90 Main Effet B F.8.7868.9980 FB.04.798.996 Interation F.084.084.084 FB.0494.0494.0494
57 TWO-WAY ANALYSIS OF VARIANCE UNDER HETEROGENEITY Case 3: Homogeneous errors, unequal sample sizes. In this ase we onsider: k =,..., n, ε N(0,), ij where nj = 7, nj = 8, n3j = 9, n 4j = 0. Here there was little differene in the performane of the two tests (See Tables 8 and 9). The Boxadjusted test showed slightly higher power in some ases. Case 4: Heterogeneous errors, unequal sample sizes. Here we onsider: k =,..., n, ε N(0, σ ), ijk ij ijk i with nj = 7, nj = 8, n3j = 9, n4j = 0. When the largest variane was assoiated with the smallest sample the lassial F-test always had inflated nominal Type I error rates (often more than twie the nominal rate) for any effets not present, while the Box-adjusted test maintained expeted nominal Type I error rates (See Tables 0, and ). The lassial F-test had greater power for small effet sizes, but the power advantage beame negligible as the effet size inreased. Although not shown here, when the largest variane was assoiated with the largest sample the power of the two tests was essentially equivalent, with the Box-adjusted test often having a slight power advantage. The lassial F-test tended to underestimate the Type I error rate for effets not present. Table 8. Proportion of rejetions at α = 0.05, normally distributed errors with unequal sample sizes ( n j = 7, n j = 8, n 3 j = 9, n 4 j = 0 ) and equal varianes, based on 5000 samples, fator A effet present (a =, a 3 =-). C Test for: Method 0.5.5 Main Effet A F.048.796.000 FB.0500.858.000 Main Effet B F.058.055.0598 FB.054.054.054 Interation F.0500.050.046 FB.044.044.044 Table 9. Proportion of rejetions at α = 0.05, normally distributed errors with unequal sample sizes n = 7, n = 8, n = 9, n = 0 ) and equal varianes, based on 5000 samples, fators A and B effets ( j j 3 j 4 j present (a =b =, a 3 =b =-). C Test for: Method.5.5 Main Effet A F.800.000 FB.830.000 Main Effet B F.9596.000 FB.9564.000 Interation F.0498.0496 FB.040.040
RICHTER & PAYTON 58 Table 0. Proportion of rejetions at α = 0.05, normally distributed errors with unequal sample sizes ( n j = 7, n j = 8, n 3 j = 9, n 4 j = 0 ) and unequal varianes ( σj = 0, σj = 5, σ3 j =, σ4 j = ), based on 5000 samples, fator A effet present (a =, a 3 =-). Test for: Method 0.5.5 Main Effet A F.056.90.9850 FB.0476.666.94 Main Effet B F.000.04.034 FB.048.048.048 Interation F.44.46.30 FB.0494.0494.0494 Table. Proportion of rejetions at α = 0.05, normally distributed errors with unequal sample sizes ( n j = 7, n j = 8, n 3 j = 9, n 4 j = 0 ) and unequal varianes ( σj = 0, σj = 5, σ3 j =, σ4 j = ), based on 5000 samples, fator A and B effets present (a =b =, a 3 =b =-). C Test for: Method.5.0.5 Main Effet A F.3070.876.9944 FB.634.6660.9788 Main Effet B F.45.9450.999 FB.374.885.9980 Interation F.4.4.08 FB.0494.0494.0494 Table. Proportion of rejetions at α = 0.05, normally distributed errors with unequal sample sizes ( n j = 7, n j = 8, n 3 j = 9, n 4 j = 0 ) and unequal varianes ( σj = 0, σj = 5, σ3 j =, σ4 j = ), based on 5000 samples, interation effet present (ab =ab 33 =, ab 3 =ab 3 =-). C Test for: Method.5.5.5 Main Effet A F.060.046.06 FB.0476.0476.0476 Main Effet B F.03.08.06 FB.048.048.048 Interation F.8.878.9996 FB.0938.634.9898
59 TWO-WAY ANALYSIS OF VARIANCE UNDER HETEROGENEITY Conlusion Based on our results and the results of Brunner, et al. (997), we agree with those authors that there is no reason to use the lassial ANOVA F-test, as long as ell sample size is at least 7. For smaller samples, when the normal theory assumptions hold, we prefer the lassial ANOVA F-test, sine the FB statisti beomes very onservative in this ase. When samples are very small and varianes are not equal, the ANOVA test suffers from inflated nominal levels and thus should be used with aution. The FB test, on the other hand, is always onservative in these situations, and thus is a good hoie for those onerned mostly with avoiding making Type I errors. The obvious tradeoff for small sample sizes, however, is that the FB test is virtually powerless to detet small to moderate effets. Example. We illustrate the method using an example given in Sokal and Rohlf (995). The data are from an experiment to examine differenes in food onsumption when ranid lard was substituted for fresh lard in the diet of rats. The data are lassified by fat (fresh, ranid) and gender (male, female). The amount of food eaten (in grams) is given in the following table: Gender Fats Fresh Ranid Male 709 59 679 538 699 476 Female 657 508 594 505 677 539 A SAS program (available from the first author) was used to ompute the p-values for both the ANOVA F-test and the FB test. Sine ell sample sizes are equal, values of the F and FB statistis are idential. Notie that although the sample sizes are small (n = 3), there is very little differene between the p-values assoiated with the two methods, and only a strong effet of gender is evident from the data. Soure of F p- FB p- variation value value Fats.593 0.46.593 0.53 Gender 4.969 <0.00 4.969 <0.00 Fats*Gender 0.630 0.450 0.630 0.454 Example. This example utilizes data presented in Kuehl (000), page 4. It is a 3x fatorial experiment involving 3 levels of alohol and two levels of base. Note that the data are unbalaned in terms of the number of repliations per treatment ombination. Beause the ell sample sizes are not equal, the alulated test statistis are not the same for the two methods, although the onlusions might be the same for both methods depending upon the level of signifiane the researher adopted. The FB statisti gives stronger evidene for effets of interation and main effets. Alohol Base 3 90.7 89.3 89.5 9.4 88. 87.6 90.4 88.3 90.3 Mean 9.05 89.7 88.93 Std Dev 0.49.5. 87.3 94.7 93. 88.3 90.7 9.5 9.5 Mean 89.03 94.7 9.77 Std Dev.9 ---. Soure of F p- FB p- variation value value Alohol.93 0.95 4.97 0.053 Base 7.67 0.03.858 0.006 Alohol*Base 7.357 0.0 4.087 0.00
RICHTER & PAYTON 60 Referenes Akritas, M.G. (990), The rank transform method in some two-fator designs. Journal of the Amerian Statistial Assoiation, 85, 73-78. Akritas, M.G., & Arnold, S.F. (994). Fully nonparametri hypotheses for fatorial designs I: Multivariate repeated-measures designs. Journal of the Amerian Statistial Assoiation, 89, 336-343. Box, G.E.P. (954). Some theorems on quadrati forms applied in the study of analysis of variane problems, I: Effet of inequality of variane in the one-way lassifiation. The Annals of Mathematial Statistis, 5, 90-30. Brunner, E., Dette, H., & Munk, A. (997). Box-type approximations in nonparametri fatorial designs. Journal of the Amerian Statistial Assoiation, 9, 494-50. Kuehl, R. O. (000). Design of experiments: Statistial priniples of researh design and analysis. ( nd ed.) Paifi Grove, CA: Brooks/Cole. Milliken, G.A., & Johnson, D.E. (99). Analysis of messy data, Volume : Designed experiments. New York: Chapman and Hall. Patnaik, P.B. (949). The nonentral χ and F-distributions and their appliations. Biometrika, 36, 0-3. Sokal, R. R., & Rohlf, F. J. (995). Biometry: The priniples and praties of statistis in biologial researh, New York: W. H. Freeman and Company. Thompson, G.L. (99). A unified approah to rank tests for multivariate and repeated measures designs. Journal of the Amerian Statistial Assoiation, 86, 40-49. Weerahandi, S. (995). ANOVA under unequal error varianes. Biometris, 5, 589-599. Welh, B.L. (95). On the omparison of several mean values. Biometrika, 38, 330-336.