Testing equality of variances for multiple univariate normal populations

University of Wollongong Research Online Centre for Statistical & Survey Methodology Working Paper Series Faculty of Engineering and Inforation Sciences 0 esting equality of variances for ultiple univariate noral populations David Allingha University of Newcastle J C. Rayner University of Wollongong Recoended Citation Allingha, David and Rayner, J C., esting equality of variances for ultiple univariate noral populations, Centre for Statistical and Survey Methodology, University of Wollongong, Working Paper 4-, 0,. http://ro.uow.edu.au/csswp/9 Research Online is the open access institutional repository for the University of Wollongong. For further inforation contact the UOW Library: research-pubs@uow.edu.au

Centre for Statistical and Survey Methodology he University of Wollongong Working Paper 04- esting Equality of Variances for Multiple Univariate Noral Populations David Allingha and J.C.W. Rayner Copyright 008 by the Centre for Statistical & Survey Methodology, UOW. Work in progress, no part of this paper ay be reproduced without perission fro the Centre. Centre for Statistical & Survey Methodology, University of Wollongong, Wollongong NSW 5. Phone 6 4 545, Fax 6 4 4845. Eail: anica@uow.edu.au

esting Equality of Variances for Multiple Univariate Noral Populations David Allingha Centre for Coputer-Assisted Research Matheatics and its Applications School of Matheatical and Physical Sciences, University of Newcastle, NSW 08, Australia David.Allingha@newcastle.edu.au and J.C.W. Rayner Centre for Statistical and Survey Methodology, School of Matheatics and Applied Statistics, University of Wollongong, NSW 5, Australia and School of Matheatical and Physical Sciences, University of Newcastle, NSW 08, Australia John.Rayner@newcastle.edu.au Abstract o test for equality of variances in independent rando saples fro ultiple univariate noral populations, the test of first choice would usually be the likelihood ratio test, the Bartlett test. his test is known to be powerful when norality can be assued. Here two Wald tests of equality of variances are derived. he first test copares every variance with every other variance and was announced in Mather and Rayner (00), but no proof was given there. he second test is derived fro a quite different odel using orthogonal contrasts, but is identical to the first. his second test statistic is siilar to one given in Rippon and Rayner (00), for which no epirical assessent has been given. hese tests are copared with the Bartlett test in size and power. he Bartlett test is known to be non-robust to the norality assuption, as is the orthogonal contrasts test. o deal with this difficulty an analogue of the new test is given. An indicative epirical assessent shows that it is ore robust that the Bartlett test and copetitive with the Levene test in its robustness to fat-tailed distributions. Moreover it is a Wald test and has good power properties in large saples. Advice is given on how to ipleent the new test. Key Words: Bartlett s test; Levene test; Orthogonal and non-orthogonal contrasts; Wald tests.. Introduction Suppose we have independent rando saples, with the th being of size n and being fro a noral N(µ, σ ) population, for =,,. he total saple size is n = n n. We seek to test equality of variances: H: σ = = σ = σ, say, against the alternative K: not H. Probably the ost popular choices for testing H against K are the paraetric Bartlett test, which is known to be not robust to departures fro norality, and the nonparaetric Levene test, which is known to be

robust but is less powerful than Bartlett s test when the data are approxiately noral. In the case of ust two independent rando saples the likelihood ratio test is equivalent to the F test based on the quotient of the (unbiased) saple variances. Using the Wald test, Rayner (997) derived a test based on the difference of the saple variances. his test, and a robust analogue of it, were discussed in Allingha and Rayner (0). For the -saple proble Mather and Rayner (00) gave, without proof, a new test statistic. Here a relatively succinct derivation is given. Using the sae approach a new copetitor test, based on orthogonal contrasts, is derived. It is not iediately obvious, but the two tests are identical. In a sall epirical study we show that for the configurations chosen here the orthogonal contrasts test is slightly inferior to, but certainly copetitive with, the Bartlett test in both size and power. his is consistent with the fact that likelihood ratio and Wald tests are asyptotically equivalent. In the case =, Allingha and Rayner (0) showed that the test derived here is, like the Bartlett test, sensitive to departures fro norality. However it is possible to give an analogue of the test that is robust to non-norality. We argue that this test is a Wald test and hence has good power in large saples. A siilar analogue of the Bartlett test is not apparent. For convenience the Bartlett test statistic B is given here. Define B = ( n )log S ( ) = = ( n )log S, n n in which S is the unbiased saple variance fro the th saple, =,, and S n S / n is the pooled saple variance. he test statistic B has = ( ) ( ) asyptotic distribution χ. Coon practice when norality is in doubt is to use Levene s test. his is based on the ANOVA F test applied to the saple residuals. here are different versions of Levene s test using different definitions of residual. he version eployed here uses the group eans, Xi X i, in an obvious notation.. he distribution of the test statistic, L, say, is approxiately F, n n. In Sections and derivations of the Mather and Rayner (00) and the orthogonal contrasts tests are given. A brief epirical assessent of these tests is given in Section 4. A robust analogue of the orthogonal contrasts test is given in Section 5. An exaple is given in Section 6, while section 7 gives by a brief conclusion, including advice on how to ipleent the recoended robust test.. he Mather-Rayner -Saple est o test H against the alternative K first define an ( ) contrast atrix C. Essentially, the rows of C are used to define contrasts between the population variances σ. hese contrasts ust be specified before sighting the data. If we define

= ( σ,, σ ), then the null hypothesis of equality of variances is equivalent to = C = 0. In this section we take (C) ii = and (C) i(i) =, i =,,, with all other eleents zero. he contrasts are between successive variances: σ σ, σ σ,, σ σ. Clearly there are any other possible choices of contrast atrix. If, as above, S is the unbiased saple variance fro the th saple, then put = ( S ) and = C. A Wald test of H against K ay be based on côv ( ). Using the Rao-Blackwell theore as in Rayner (997), the covariance atrix of, cov( ) = diag(σ /(n )), ay be optially estiated by D = diag(d ), where d = 4 S 4 /(n ) for =,,. Hence the covariance atrix of ay be optially estiated by C diag(d ) C, which for this choice of C is d d d 0 d d d 0 d d 0 0. Note that only variances and adacent eleents of have non-zero covariance. Mather and Rayner (00) gave the test statistic in ters of contrasts between all the saple variances: MR = r= s> r ( n )( n r s r= )( S r r ( n ) / S S ) 4 r s /( S 4 r S 4 s ). (.) hey noted that the derivation required the inversion of the tri-diagonal atrix, and in fact the inverse was calculated for soe sall values of, an inverse guessed and the result proven by a lengthy induction. For the two-saple proble a siple calculation shows that = S S and var( ) = d d, agreeing with the result in Rayner (997). A relatively succinct derivation of MR is given in the Appendix. A Wald est Using Orthogonal Contrasts he coplexity of the derivation of the Wald test can be reduced by requiring that the rows of C be orthogonal. First, define C* = (C u) to be an orthogonal atrix. Should the null hypothesis be true, the coon variance could be estiated by the pooled saple variance, S = ( n ) S /( n ). Put w = (n )/(n ) for =,, and note that w =, since ( n ) = n. Now, define σ = w σ, = ( σ w ), u = ( w ) and C = I uu. Note that C is syetric, idepotent, orthogonal and not of full rank. Its only specification so far, and ultiately, is that its rows are orthogonal to u.

Put = C = (( σ σ ) w ), which is zero if and only if σ = σ for all. Hence we again wish to test H: = 0 against K: 0, albeit for a slightly different than in the previous section. Here, = (( S S ) w ) is an unbiased estiator of, and is asyptotically equivalent to the axiu likelihood estiator of. he covariance atrix of is estiated by côv( ) = CDC, where now D = diag(d w ) with the d as in the previous section. o find the inverse of côv( ) a routine lea is needed. he proof is oitted. A B E F Lea.. If C D G H invertible, then A = E FH G. I 0 = and if E, F, G, H are known with H 0 I he lea is applied with A = côv( ) and E G F H = C* D C* = C*D C* = CD C u D C CD u u D u, so that A = CD C (CD u) (u D C )/(u D u). A Wald test of H: = 0 against K: 0 can be based on OC = Since = ( σ w ), = ( S w ). Substituting gives OC = Now, since C* is orthogonal, C (CD C ) C (u D C C ) /(u D u). I = C* C* = ( u) C C = C C u u, u côv ( ). giving C C = I uu. With u = ( w,, w ) it follows that u = ws = S and (I uu ) = ( w ( S S )) = C, say. hus OC = (I uu )D (I uu ) {u D (I uu ) } /(u D u) = = C D (u D ) /(u D u) C C = ( S S ) { ( S S )} /{ } (.) = = in which = /d for =,,. It is not iediately obvious, but OC and MR are identical. We do not show that here. In Rippon and Rayner (00) orthogonal contrasts and the Moore-Penrose inverse were used to derive a Wald test based on = 4

= ( S S ) = MP say. Clearly MP is ust OC without the final correction ter that can be expected to be sall when the null hypothesis is true. Note that under the null hypothesis of equality of variances, the test statistics MR, MP and OC all have asyptotic distribution χ. Moreover, these three test statistics are invariant under transforations Y k = a(x k b ), for constants a, b, for =,,. 4 Epirical Assessent hroughout this assessent we take saples of the sae size, N, in each of the populations. hus, the total saple size, n, is given by n = N. he test sizes of the copeting tests were copared for a nuber of cobinations of the paraeters involved. Figure shows the proportion of reections in 00,000 Monte Carlo siulations for noinal 5% level tests for varying saple sizes N, each population being noral with ean 0 and variance. he left panel copares = 4 populations, the right panel = 8 populations, for the tests based on B, OC and MP. he test based on MP is clearly not copetitive. he for of B used here (given in Section ) has been adusted fro the likelihood ratio test statistic to iprove its type error rate, and that is reflected in its excellent adherence to the noinal level in Figure. Interestingly the test based on OC perfors far worse for = 8 than it does for = 4. Figure. est sizes for tests based on B, OC and MP for noinal 5% level tests for varying saple sizes N fro noral populations with eans 0 and variances. he left panel copares = 4 populations and the right = 8 populations. 5

he powers of the tests based on B and OC are copared in Figure. Critical values have been estiated so that all are 5% level tests. he test based on MP was excluded because, as noted above, in ters of the agreeent between the achieved and noinal levels, it is not copetitive. Each point is the proportion of reections in 00,000 sets of data, with saples sizes of N = 0, 0 and 50 for each of = 4 populations fro the noral distributions with eans 0 and population variances, Δσ, Δσ, Δσ, where Δσ varies between 0 and in steps of 0.00. hus, at the LHS of the plot, the variances are equal and at the RHS they are,, and 4. Figure. Power functions for the Bartlett test (thin line) and the test based on OC (thick line), based on four noral populations, a significance level of 5% and saple sizes fro each population of N = 0, 0, 0 and 50. As N increases, the powers of the two tests approach each other. his is as expected since both are asyptotically optial tests. he paraeter space is clearly ultidiensional, and here we have chosen a very particular subset. In this space, for saller saple sizes, the power of the Bartlett test is slightly superior to the test based on OC. However, across the entire saple space it could be expected that they will perfor siilarly, with soeties one, and soeties the other, being slightly superior. 6

5. A Robust For of the Orthogonal Contrasts est In the discussion so far we have introduced tests based on MR, OC and MP. he test based on the Moore-Penrose test statistic, MP, has poor adherence to its noinal test size, and the test statistics MR and OC are, in fact, identical. he use of the test based on OC ay be coproised if, like the Bartlett test, it is not robust to departures fro the norality assuption. In fact, as was shown in Allingha and Rayner (0) for the case = and as we shall see in Figure for = 4 and = 8, the agreeent between noinal and actual test sizes for the test based on OC is not good enough for practical use when sapling fro a fat-tailed distribution. We therefore propose an analogue of OC. In deriving that test, the variances var( S ) were estiated optially using the Rao-Blackwell theore. he estiation depends very strongly on the assuption of norality. If norality is in doubt then var( S ) can be estiated using results given, for exaple, in Stuart and Ord (994). For a rando saple Y,, Y n with population and saple central oents µ r and r n r = = ( Y Y ) / n, r =,, respectively, Stuart and Ord (994) gave E[ r ] = µ r O(n ) and var( ) = (µ 4 µ )/n O(n ). µ ay be estiated to O(n ) by, or, /(n ) = S 4, where S is the unbiased saple variance. It Applying Stuart and Ord (994, 0.5), equivalently, by n follows that, to order O(n ), var( ) ay be estiated by ( 4 instead of estiating var( S ) by S 4 /( ), we propose using ( 4 n )/n. herefore 4 S )/n. hus, in the forula for OC given in (.), replace = (n )/( S ) by = n /( 4 4 4 S ). his test statistic will be denoted by AR. A preliinary study found that agreeent of the noinal and actual test size for OC was not good enough for practical use (see Figure ). he Bartlett test statistic, however, has been adusted to iprove this agreeent, and so a coparison between B, OC and AR does not copare like with like. here are various options for such an iproveent, and here we have opted to adust the 5% critical points. Under the assuption of norality and under the null hypothesis of equality of variances, we estiated the 5% critical points for N between 0 and 50, inclusive, using 00,000 siulations of each saple size for = 4 and = 8 populations. Using standard curve fitting techniques, we found that these were well approxiated over that range by c(4, N, 0.05) = χ ( 0.74 8.08N 0.5 4.96N 80.74N.5 ) for = 4 and ;0.05 c(8, N, 0.05) = χ (.64 4.679N 0.5.485N 45.5N.5 ) for = 8. 7;0.05 Using these critical values for = 4 and = 8 and for 0 < N < 50 the actual test sizes vary between 0.046 and 0.054. Use of the asyptotic 5% values χ required very large saple sizes, ;0.05 depending on and the noinal level α. heir use cannot be recoended. 7

Figure. est sizes for the tests based on L (solid line and dots), OC ( and dots), AR using critical values estiated under norality (triangles) and AR using four ter Bartlett correction (squares). For a significance level of 5% we copare four populations (panels (a), (b) and (c)) and eight populations (panels (d), (e) and (f)), and saple sizes of N = 0 (panels (a) and (d)), 0 (panels (b) and (e)), and 50 (panels (c) and (f)). For N > 50, or for levels other than 5%, or for unequal saple sizes, critical values can be estiated using the MALAB or R code available at the URL given in the conclusion. o assess the effect of non-norality, instead of sapling fro the standard noral distribution, we sapled fro tν distributions with varying degrees of freedo, ν. If ν is large, the distribution sapled will be close enough to noral that we could expect the proportion of reections to be close to the noinal. he critical values used in this assessent are both c(, N, 0.05) and the estiated exact critical points. 8

In Figure we plot the proportion of reections - actual test size - for the tests based on L, OC and, with both critical values, AR. Sapling is fro tν distributions for ν =,, 0. We show curves for each test with coon saple sizes N = 0, 0 and 50. It is apparent that the test based on OC perfors increasingly poorly as the degrees of freedo reduce and the tails of the distribution becoe fatter. Although not shown in the figure, this parallels the results for the Bartlett test. he Levene test generally has exact level closer to the noinal level than the tests based on AR except for sall degrees of freedo. However the level of the test based on AR is alost always reasonable, and while for very sall ν the level is not as close to the exact level as perhaps we ay prefer, the sae is the case for the Levene test. In general, a Wald test of H: = 0 against K: 0 is based on a quadratic for!!"!, in which is asyptotically equivalent to the axiu likelihood estiator of and Σ is the asyptotic covariance atrix of. hus, the tests based on OC and AR are both Wald tests and will have the weak optiality these tests enoy. In particular they are asyptotically equivalent to the likelihood ratio test, and thus will have power approaching the power of the Bartlett test in large saples. As the tests based on AR, using either c(, N, 0.05) or estiated exact critical values, have actual test sizes close to the noinal 5% test size, are asyptotically optial and are robust (at least to fat-tailed distributions), they can be recoended. 6 Exaple: National Institute of Standards and echnology Data We analysed data fro the National Institute of Standards and echnology, involving ten groups of ten observations (NIS/SEMAECH, 006). he collection of all unadusted observations is not consistent with norality (Shapiro-Wilk p-value less than %) but if the group eans are subtracted fro each observation the collection of all centred observations is consistent with norality (Shapiro-Wilk p- value 0.5). We conclude the data are consistent with the assuption that they are N(µ, σ ) distributed. Without loss of generality all observations are ultiplied by 000, giving the following standard deviations for the ten groups: 4.46, 5.64,.9777,.858, 7.5785, 9.8860, 7.8775,.67, 4.79 and 5.9. We find that B and AR take the values of 0.786 and 6.55, respectively, with corresponding Monte Carlo p-values, based on 00,000 siulations, of 0.0 and 0.445. he χ 9 p-value for B is 0.04, consistent with the Monte Carlo p-value. he saple sizes N are too sall to use the asyptotic distribution to obtain a p-value for AR. Fro the Bartlett test it appears that there is evidence, at the 5% level, that the variances are not consistent. he Bartlett test is uch ore critical of the null hypothesis than the test based on AR. his is consistent with the powers shown in Figure, where for sall saple sizes the Bartlett test appears to be ore powerful. We doubled and trebled N by duplicating the data within each population and found that the AR test was then equally critical of the data. 9

If the data are not consistent with norality then the Bartlett test cannot be used. However, the test based on AR is valid and can be ipleented by finding a Monte Carlo p-value. his erely involves generating sets of standard noral values for saples of size n,, n and calculating AR for each data set. he proportion of exceedances of the observed AR value is the Monte Carlo p-value. MALAB code to ipleent this procedure is available at the web site given in the conclusion. 7 Conclusion We have given a copact derivation for the test based on MR, introduced in Mather and Rayner (00). A new test, based on OC, is also derived. In fact MR and OC are identical. A test due to Rippon and Rayner (00), based on MP, is OC inus a correction factor. he test based on MP approaches its asyptotic distribution quite slowly. hus unless resapling ethods are used to calculate p-values, the OC test is to be preferred to MP. Although the test new test based on OC cannot be claied to be superior to that based on B in ters of size and power, it is copetitive. he Bartlett test adheres to its noinal significance level ore closely, no doubt in part because the test statistic has been adusted; were the new test to be so adusted it is very probable that both tests will perfor siilarly. For larger saple sizes the powers are siilar. A robustness study shows that when sapling is fro fat-tailed t distributions instead of the noral, the tests based on B and OC do not have actual test size close to noinal. However tests based on L and AR are acceptable in this regard. As the test based on AR is a Wald test it can be expected to have good power in large saples. Even when not sapling fro noral distributions, test statistics of the sae for as AR, quadratic fors with vector asyptotically equivalent to the axiu likelihood estiator of the paraeter of interest and atrix the asyptotic covariance atrix of, are Wald tests and can be expected to have excellent properties in large saples. o apply the test based on AR three options are available. For = 4 and = 8, α = 5% and equal saple sizes N between 0 and 50 fro each population, the Bartlett approxiate critical values c(, N, 0.05) given in Section 5, ay be used. For other configurations of (n,, n ) and other α, critical values ay be estiated using the MALAB or R code available at http://hdl.handle.net/959./97. Code to calculate the Monte Carlo p-value for a data set is available at the sae URL. References Allingha, David and Rayner, J. C. W. (0). A nonparaetric two-saple Wald test of equality of variances. Subitted. Mather, Katrina J. and Rayner, J.C.W. (00). esting equality of corresponding variances fro ultiple covariance atrices. In Advances in Statistics, Cobinatorics and Related Areas. Chandra Gulati, Yan-Xia Lin, Satya Mishra and John Rayner (eds). Singapore: World Scientific Press, 57-66. NIS/SEMAECH (006). Data used for chi-square test for the standard deviation, http://www.itl.nist.gov/div898/handbook/eda/section/eda58.ht. Accessed Noveber 0. 0

Rayner, J.C.W. (997). he asyptotically optial tests. J.R.S.S., Series D (he Statistician), 46(), 7-46. Rippon, Paul and Rayner, J.C.W. (00). Generalised score and Wald tests. Advances in Decision Sciences, Article ID 90, 8 pages. doi:0.55/00/90. Stuart, A. and Ord, J.K. (994). Kendall's Advanced heory Of Statistics, Vol.: Distribution theory (6th ed.), London: Hodder Arnold. Appendix: Derivation of MR First, C is augented by the row vector (0,, 0, ) to for the atrix C C* =. 0 0 he inverse C* is readily shown to satisfy (C* ) i = for > i and 0 otherwise. o see this define * = (, ) = C* so that =,, =, =, whence =, = = * * *, = * *,., = *. hus = C* * with C* as stated. o calculate MR = ( côv ) c ôv ( ) is first required. Now as above c ôv( ) = D so that c ôv( ) = CDC. Siilarly côv( *) = C*DC*, so that côv ( * ) = C* D C* is known. We require côv ( ), the leading ( ) ( ) block of côv ( * ). his ay be found using Lea. with A = c ôv( ) and, recalling that = /d for all, (clearly all d > 0), we have E G F H = C* D C* =. Here E is ( ) ( ), G = (,,, ), F = G and H is the scalar. he Wald test statistic is MR = côv ( ) = - A = W W in which W = E and W = FH G. Now { W = ( ) ( ) ( ) ( ) ( ) } = { } { } = ( ) ( ) and

G = ( ) ( ) = ( ) ( ). hus, writing. =, we have. MR =. { ( ) ( ) } ( ) ( ) { } = = ( )( ). ( )( ). ( ). ( )( ) = ( )( ) ( )( ) ( ) ( )( ) = = ( ) ( ) { } ( ) { } ( ) ( ) { } 4 ( ) { } ( ) ( ) = = ( ) = < i i i i,. In particular for = MR = ( ) ( ) /, which agrees with (.) after substituting for the. Siilarly for = MR = ( ) ( ) ( ) { }( ) /. Routinely substituting for,, and. gives (.).