A Scaled Diæerence Chi-square Test Statistic. Albert Satorra. Universitat Pompeu Fabra. and. Peter M. Bentler. August 3, 1999

Transcription

1 A Scaled Diæerence Chi-square Test Statistic for Moment Structure Analysis æ Albert Satorra Universitat Pompeu Fabra and Peter M. Bentler University of California, Los Angeles August 3, 1999 æ Research supported by the Spanish DGES grant PB , and USPHS grants DA00017 and DA

2 Abstract A family of scaling corrections aimed to improve the chi-square approximation of goodness-of-æt test statistics in small samples, large models, and nonnormal data was proposed in Satorra and Bentler è1994è. For structural equations models, Satorra-Bentler's èsbè scaling corrections are available in standard computer software. Often, however, the interest is not on the overall æt of a model, but on a test of the restrictions that a null model say M 0 implies on a less restricted one M 1.IfT 0 and T 1 denote the goodness-of-æt test statistics associated to M 0 and M 1, respectively, then typically the diæerence T d = T 0, T 1 is used as a chi-square test statistic with degrees of freedom equal to the diæerence on the number of independent parameters estimated under the models M 0 and M 1. As in the case of the goodness-of-æt test, it is of interest to scale the statistic T d in order to improve its chi-square approximation in realistic, i.e., nonasymptotic and nonnormal, applications. In a recent paper, Satorra è1999è shows that the diæerence between two Satorra- Bentler scaled test statistics for overall model æt does not yield the correct SB scaled diæerence test statistic. Satorra developed an expression that permits scaling the diæerence test statistic, but his formula has some practical limitations, since it requires heavy computations that are not available in standard computer software. The purpose of the present paper is to provide an easy way to compute the scaled diæerence chi-square statistic from the scaled goodness-of-æt test statistics of models M 0 and M 1. A Monte Carlo study is provided to illustrate the performance of the competing statistics. Keywords: Moment-structures, goodness-of-æt test, chi-square diæerence test statistic, chi-square distribution, non-normality 1

3 1 Introduction Moment structure analysis is widely used in behavioural, social and economic studies to analyse structural relations between variables, some of which may be latent èi.e., unobservableè; see, e.g., Bollen è1989è, Bentler and Dudgeon è1996è, Yuan and Bentler è1997è, and references therein. Commercial computer programs to carry out such analysis, for a general class of structural equation models, are available èe.g., LISREL of Jíoreskog and Síorbom, 1994; EQS of Bentler, 1995è. In multi-sample analysis, data from several samples are combined into one analysis, making it possible, among other features, to test for across-group invariance of speciæc model parameters. Statistics that are central in moment structure analysis are the overall goodness-of-æt test of the model and tests of restrictions on parameters. Asymptotic distribution-free èadfè methods which do not require distributional assumptions on the observable variables have been developed èbrowne, 1984è. The ADF methods, however, involve fourth-order sample moments, thus they may lack robustness to small and medium-sized samples. In the case of non-normal data, an alternative to the ADF approach is to use a normal-theory estimation method in conjunction with asymptotic robust standard errors and test statistics èsee Satorra, 1992è. Asymptotic robust test statistics, however, may still lack robustness to small and medium-sized samples. As an alternative to asymptotically robust test statistics, Satorra and Bentler è1994; Satorra and Bentler, 1988a,bè developed a family of corrected normal-theory test statistics which are easy to implement in practice, and which have been shown to outperform the asymptotic robust test statistics in small and medium-sized samples èe.g., Chou, Bentler and Satorra, 1991; Hu, Bentler and Kano, 1992; Curran, West and Finch, 1996è. Bentler and Yuan è1999è provide a recent comparison of alternative test methods for small samples. Extension of Satorra-Bentler èsbè's corrections to goodness-of-æt test statistics in the case of the analysis of augmented moment structures, multi-samples and categorical data, have been discussed respectively by Satorra è1992è and Muthçen è1993è. Although SB corrections have been available for some time, formal derivations of SB corrections to the case of nested model comparisons have not been available. The obvious and generally accepted approach of computing separate SB-corrected test statistics for each of two nested models, and then computing the diæerence between them èe.g., Byrne and Campbell, 1999è, 2

4 turns out to be an incorrect way to obtain a scaled SB diæerence test statistic. The diæerence could be even be negative, which is an improper value for achi-square variate. In a recent paper, Satorra è1999è gives speciæc formulae for extension of SB corrections to score èlagrange multiplierè, diæerence and Wald test statistics. He showed that the diæerence between two SB-scaled test statistics does not necessarily correspond to the scaled chi-square diæerence test statistic. The purpose of the present paper is to provide a simple expression that allows a researcher to correctly compute the SB diæerence test statistic when the SB-scaled chi-square goodness of æt tests for the corresponding two nested models are available. The formula is simple to use and provides an alternative scaled test for evaluating a speciæc set of restrictions. The paper is structured as follows. In Section 2 we describe goodness-ofæt tests in weighted least squares analysis, and the corresponding SB scaling corrections. In Section 3 we describe the proposed procedure for computing the SB scaled diæerence test statistic. Section 4 concludes with an illustration. 2 Goodness-of-æt tests Let ç and s be p-dimensional vectors of population and sample moments respectively, where s tends in probability to ç as sample size n! +1. Let p ns be asymptotically normally distributed with a ænite asymptotic variance matrix, èpæpè. Consider the model M 0 : ç = çèçè for the momentvector ç, where çè:è is a twice-continuously diæerentiable vector-valued function of ç, a q-dimensional parameter vector. Consider a WLS estimator ^ç of ç deæned as the minimizer of F V èçè:=ès,çè 0^Vès,çè over the parameter space, where ^V èp æ p è, converges in probability to V, a positive deænite matrix. A typical test statistic used for testing the goodness-of æt-of the model M 0 is T 0 := nf V ès; ^çè, where ^ç := çè^çè. It is widely known that, when the model M 0 holds and V satisæes the asymptotic optimality èaoè condition of V =,,1, then T 0 is asymptotically chi-square distributed with degrees of freedom èdfè r 0 = p,q. In practice, however, AO may not hold, and concern on the quality of the chi-square approximation do arise. For general types of distributions, i.e., when AO does not necessarily hold, T 0 is asymptotically distributed as a mixture of chi-square distributions 3

5 of 1 degree of freedom èdfè èsee Satorra and Bentler, 1986è; that is T 0 L! r X j=1 æ j ç 2 j ; è1è as n!1, where the ç 2 j are independent chi-square variables of 1 df, and the æ j are the non-null eigenvalues of the matrix U 0,, with U 0 := V, V æèæ 0 V æè,1 æ 0 V andæ:=è@=@ç 0 èçèçè. When AO holds, then of course the æ j 's are equal to 1 and the asymptotic exact chi-square distribution applies. In the context of structural models and for general types of distributions, Satorra and Bentler è1994; Satorra and Bentler, 1988a,bè proposed replacing T by the scaled statistic T = T=^c; è2è where ^c denotes a consistent estimator of c := 1 r tr U 0,= 1 r tr fv,g,1 r tr n èæ 0 V æè,1 æ 0 V,V æ o : è3è Note that the SB scaled test statistic has the same mean as the corresponding ç 2 r variate. The SB scaled goodness-of-æt test has been shown to outperform alternative test statistics in a variety of models and non-normal distributions èe.g., Chou, Bentler and Satorra, 1991; Hu, Bentler and Kano, 1992; Curran, West and Finch, 1996è. Of course, when asymptotic optimality holds, this statistic will have the same asymptotic distribution as the unscaled statistic T 0. Note that a consistent estimator ^, of, under general distribution conditions is required to compute the scaling factor ^c. In structural equation models, a consistent estimator of, is readily available from the raw data èe.g., Satorra and Bentler, 1994è. A goodness-of-æt statistic which can be used given any estimation method, is given by T? = nès, ^çè 0 f^,,1, ^,,1 ^æè ^æ 0^,,1 ^æè,1 ^æ 0^,,1 gès, ^çè; è4è When ^, is the èdistribution-freeè consistent estimator of, in è16è below, then T? will be called the asymptotic robust goodness-of-æt test statistic, since it is an asymptotic chi-square statistic regardless of the distribution of observable variables. In the context of single-sample covariance structure 4

6 analysis, this statistic was ærst introduced by Browne è1984è. Its performance was studied by Yuan and Bentler è1998è, who found that very large samples are required to obtain acceptable performance in models with intermediate to large degrees of freedom. 3 Testing a set of restrictions Consider now the case of testing a speciæc set of restrictions on the model. Consider a re-parameterezation of M 0 as ç = ç? èæè with aèæè =a 0, where æ is a èq + mè-dimensional vector of parameters, a 0 is an m æ 1vector of constants, and ç? è:è and aè:è are twice-continuously diæerentiable vectorvalued functions of æ 2 æ 1, a compact subset of R q+m. Our interest now is in the test of the null hypothesis H 0 : aèæè =a 0 against the alternative H 1 : aèæè 6= a 0. Deæne the Jacobian matrices æèpæèq+mèè := è@=@æ 0 èç? èæè and A èm æ èq + mèè := è@=@æ 0 èaèæè; which we assume to be regular at the value of æ associated with ç 0,sayæ 0, with A of full row rank. Let P èèq + mè æ èq + mèè := æ 0 V æ and denote by M 1 the less restricted model ç = ç? èæè. The goodness-of-æt test statistic associated with M 1 is thus T 1 = nf ès; ~çè, where ~ç is the ætted moment vector in model M 1,now with associated degrees of freedom r 1 = r 0, m and scaling factor c 1 given by where c 1 := 1 r 1 tr U 1,= 1 r 1 tr fv,g, 1 r 1 tr n P,1 æ 0 V,V æ o U 1 := V, V æp,1 æ 0 V: When both models M 0 and M 1 are ætted, then we can test the restrictions aèçè =a 0 using the diæerence test statistic T d = T 0,T 1, where under the null hypothesis, it is intended that T d have achi-square distributed with degrees of freedom m = r 0, r 1. In order to improve the chi-square approximation in the case of large values of m and moderate or small sample sizes, we are interested in the SB scaled diæerence test statistic, say ç Td. Extending his earlier work èsatorra, 1989è, Satorra è1999è recently provided formulae for computing such scaled 5 è5è

7 statistics for the diæerence, Score and Wald test statistics. From Satorra's formulae it becomes aparent that the SB scaled diæerence test statistic does not coincide with the diæerence between the two SB scaled goodness-of-æt test statistics that arise when ætting the two nested models; that is, in general çt d 6= ç T0, ç T1, where by ç T0 and ç T1 we denote the SB scaled goodness-of-æt test statistics arising when ætting the models M 0 and M 1 respectively. In Satorra è1999è, the SB scaled diæerence test statistic is deæned as ç T d = T d =^c d where ^c d is a consistent estimator of c d := 1 m tr U d, è6è with U d = V æp,1 A 0 èap,1 A 0 è,1 AP,1 æ 0 V: A practical problem with this expression for the scaled diæerence test statistic is it requires computations that are outside the standard output of current structural equation modeling programs. Furthermore, diæerence tests are usually hand computed from diæerent modeling runs. Here we will show how to combine the scaling corrections c 0 and c 1 associated to the two ætted models M 0 and M 1 in order to compute the scaling correction c d for the diæerence test statistic. It turns out that the computations are extremely simply and can be carried out using a hand calculator. First we show that U d = U 0, U 1. Note that the model M 0 implies a speciæc function æ = æèçè, that by the implicit function theorem is continuous diæerentiable. Consider thus the matrix H 0. Clearly, it holds that æ=æhand AH = 0 èrecall that A is a matrix m æ èq + mè è, that is, the matrix A and H are orthogonal complements. We have since U 0,U 1 =Væèæ 0 V æè,1 æ 0 V, V æhèh 0 æ 0 V æhè,1 H 0 æ 0 V = V æ n P,1, HèH 0 PHè,1 H 0 o æ 0 V P,1, HèH 0 PHè,1 H 0 = P,1 A 0 èap,1 A 0 è,1 AP,1 ; as A and H are orthogonal complements èsee Rao, 1973, p. 77è. We thus have the basic result that U d = U 0, U 1 : è7è 6

8 Now, since r 0 c 0, r 1 c 1 =trèu 0, U 1 è, = tr U d,= mc d ; we obtain c d = èr 0 c 0, r 1 c 1 è=m: This means that consistent estimation of c d is available from consistent estimates of the scaling corrections c 1 and c 1 associated with the null and alternative model respectively. M 0 Thus the proposed practical procedure is as follows. When ætting models and M 1,we obtain the unscaled and scaled goodness-of-æt tests, that is T 0 and ç T 0 when ætting M 0, and T 1 and ç T 1 when ætting M 1. Let r 0 and r 1 be the associated degrees of freedom of the goodness-of-æt test statistics. Then we compute the scaling corrections ^c 0 = T 0 = ç T 0 and ^c 1 = T 1 = ç T 1, and the usual chi-square diæerence T d = T 0, T 1. The SB scaled diæerence test can thus be computed as ç Td = T=^c d ; where ^c d =èr 0^c 0,r 1^c 1 è=m: When the two scaling corrections are equal, i.e. when c 0 = c 1 = c then c d = c and thus ç T d = ç T 0, ç T 1. This is the case, for example, when c 0 = c 1 =1, i.e., when both goodness-of-æt tests are asymptotically chi-square statistics. In general, however, c 0 6= c 1 and then the diæerence between two SB scaled goodness of æt test statistics does not yield the SB scaled diæerence test statistic. Note that the above procedure applies to a general modeling setting. The vector of parameters ç to be modeled may contain various types of moments: means, product-moments, frequencies èproportionsè, and so forth. Thus, the procedure applies to a variety of techniques, such as factor analysis, simultaneous equations for continuous variables, log-linear multinomial parametric models, etc.. It can easily be seen that the procedure applies also in the case where the matrix, is singular, and when the data is composed of various samples, as in multi-sample analysis. The results apply also to other estimation methods, e.g., pseudo ML estimation. It is important to recognize that a competitor to the statistic ç Td will be the diæerence between the robust goodness-of-æt test statistics associated with the models M 0 and M 1 ; that is, an asymptotic chi-square test statistic for H 0 is just T? d := T? 0, T? 1, where T? 0 and T? 1 are the goodness-of-æt test statistics associated to the models M 0 and M 1 respectively. In the next section, we will illustrate using Monte Carlo simulation the small sample size performance of the competing test statistics for the above mentioned null hypothesis H 0. 7

9 4 Illustration In this section we ilustrate in a simple model context of a regression with errors in variables the performance in ænite samples of three test statistics. We consider a regression equation y? gi = æx gi + v gi ; i =1;:::;n g ; è8è where for case i in group g èg =1;2è, y gi? and x gi are the values of the response and explanatory variables, respectively, v gi is the value of the disturbance term, and æ is the regression coeæcient. The model assumes that x gi is unobservable, but there are two observable variables x? and 1gi x? 2gi related to x gi by the following measurement-error equations x? 1gi = x gi + u 1gi ; x? 2gi = x gi + u 2gi ; è9è where u 1gi and u 2gi are mutually independent and also independent ofv gi and x gi. It is assumed that the observations are independent and identically distributed within each group. Equations è8è and è9è with the associated assumptions yield an identiæed model èsee Fuller è1987è for a comprehensive overview of measurement-error models in regression analysisè. Inference is usually carried out in this type of model under the assumption that the observable variables are normally distributed. Write the model of è8è and è9è as z gi =æç gi ; i =1;2;:::;n; è10è where and Deæne z gi := 0 æ:= y? gi x? 1gi x? 2gi æ:=eç gi ç 0 gi = C A ; ç gi := æ B x gi v gi u 1gi u 2gi 1 C C A ç xx ç vv ç uu ç uu 8 1 C A: è11è 1 C A ; è12è

10 and the parameter vector ç := èç vv ;ç xx ;ç uu ;æè 0. Under this set-up, we obtain the moment structure æ := æææ 0 =æèçè; è13è where æè:è, æè:è and æè:è are ètwice-continuously diæerentiableè matrixvalued functions of ç, as deduced from è11è, è12è and è13è. Note that the model restricts the variances of u 1 and u 2 by equality. This is a setting of two-sample data, where the population and sample vectors ç and s are deæned as s =èç 0 1 ;ç0 2è 0 and s =ès 0 1 ;s0 2è 0, where çg 0 =vec S g and s 0 g =vec S g, with S g := 1 n g n X g i=1 z gi z 0 gi : Here ëvec" denotes the column-wise vectorisation operator èsee Magnus and Neudecker, 1999, for full details on this operatorè. We consider the estimation of the model using weighted least squares under the assumption of normality. That is, the matrix V èsee aboveè has the form ^V := block diagè n 1 n ^V 1 ; n 2 n ^V 2 è è14è and ^Vg = 1 2 ès g,1 æ S g,1 è, g =1;2. Clearly, when there is independence across samples, the asymptotic variance matrix of p ns is of the form, = block diagè n n 1, 1 ; n n 2, 2 è; è15è where, g is the asymptotic variance of p n g s g, g =1;2. We further assume that the matrices S g and æ g are positive deænite, and that n g =n! f g é 0, as n! +1 èg =1;2è; in this case, a distribution-free consistent estimator of, is ^, := block diagè n n 1 ^, 1 ; n n 2 ^, 2 è; è16è where ^, g := 1 n g, 1 n X g i=1 èd gi, s g èèd gi, s g è 0 ; è17è with d gi := vec z gi z 0 gi. The Monte Carlo study generates two-sample data from the above model. Two models are ætted. Model M 0 has the parameters restricted across groups, and model M 1 has parameters that are unrestricted across groups. 9

11 For each of the estimated models, we compute the goodness-of-æt test statistics T 0 and T 1, the SB scaled statistics T ç 0 and T ç 1, and the robust test statistics T? 0 and T? 1.To test the hypothesis of parameter invariance acrosssamples, we consider the competing statistics T d = T 0, T 1, T? d = T? 0, T? 1, d ç T = ç T0, ç T1 and ænally, the proposed statistic ç Td. Note that only T? d is asymptotically an exact chi-square statistic. Our conjecture is that for nonnormal data, small samples andèor models with large degrees of freedom, the statistic ç T d will perform the best. We obtained replications of the above statistics for various combinations of sample sizes, ranging from a small sample size to an intermediately large sample size. Results are reported in Table 1 and also in Figure 1, where the empirical p-values of the various statistics are ætted against the theoretical ones corresponding to a uniform distribution. In all the replications we used ç 0 =è1;1;:3;2è 0. The distributions of v and x were independent conveniently scaled zero mean and unit variance chi-squared of 1 df èi.e., a highly nonnormal distributionè; the distribution of u 1 and u 2 were set to be normal, mutually independent, and independent of vand x. The normal-theory GLS estimation method described in Section 2 was used. The restricted model imposed across-group invariance of model parameters. In each replication, we computed the statistics mentioned above, corresponding to the null hypothesis of invariance of cross-sample model parameters. Clearly, in our Monte Carlo set-up, the null hypothesis holds true, with the null distribution of the statistics being chi-square with m = 4 degrees of freedom. Note that in our Monte Carlo set-up, severe non-normality of random constituents of the model requires the use of robust andèor corrected versions of the diæerence test statistic. We note that the normal-theory chi-square goodness-of-æt T 1 of the unrestricted model èi.e., the model that does not restrict parameters across groupsè is an asymptotic chi-square statistic despite non-normality of the data èthis follows from the asymptotic robustness theory for multi-samples; cf., Satorra, 1992è. In contrast, the normal-theory chi-square goodness of æt T 0 of the restricted model èi.e., the model that imposes parameter invariance across-samplesè is not necessarily an asymptotic chi-square statistic èsince variances of non-normal constituents of the model are restricted by equality across-groups; cf., Satorra, 1992è. As shown in Table 1, in our speciæc model context, in the smaller sample, the SB scaled statistic, ç Td, seems to outperform the alternative robust test statistic T? d. As expected from theory, in the case of the large sample, T? d 10

12 Table 1: Monte Carlo results: empirical signiæcance levels of test statistics nominal signiæcance levels: 1è 5è 10è 20è n 1 = 100 and n 2 = 120 çt d T d? T d dt ç n 1 = 800 and n 2 = 900 çt d T d? T d d T ç outperforms the alternative test statistics. Especially interesting is that the statistic d ç T = ç T 0, ç T 1 performs very badly indeed. That is, doing the presumably natural thing, simply computing the diæerence between two SB scaled chi-square statistics, yields a very poorly performing test when evaluated by the chi-square distribution. References ë1ë Bentler, P. M. è1995è. EQS structural equations program manual. Encino, CA: Multivariate Software. ë2ë Bentler, P. M., and Dudgeon, P. è1996è. Covariance structure analysis: Statistical practice, theory, and directions. Annual Review of Psychology, 47, 541í570. ë3ë Bentler, P. M., and Yuan, K. -H. è1999è. Structural equation modeling with small samples: Test statistics. Multivariate Behavioral Research, 34, 183í

13 ë4ë Bollen, K. A. è1989è. York: Wiley. Structural equations with latent variables. New ë5ë Browne, M. W. è1984è. Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62í83. ë6ë Byrne, B. M. and Campbell, T. L. è1999è. Cross-cultural comparisons and the presumption of equivalent measurement and theoretical structure: A look beneath the surface. Journal of Cross-cultural Psychology, 30, 557í576. ë7ë Chou, C. -P., Bentler, P. M., and Satorra, A. è1991è. Scaled test statistics and robust standard errors for non-normal data in covariance structure analysis: A Monte Carlo study. British Journal of Mathematical and Statistical Psychology, 44, 347í357. ë8ë Curran, P. J., West, S. G., and Finch, J. F. è1996è. The robustness of test statistics to nonnormality and speciæcation error in conærmatory factor analysis. Psychological Methods, 1, 16í29. ë9ë Fuller, W. A. è1987è. Measurement error models. New York: Wiley. ë10ë Hu, L., Bentler, P. M., and Kano, Y. è1992è. Can test statistics in covariance structure analysis be trusted? Psychological Bulletin, 112, 351í362. ë11ë Jíoreskog, K., and Síorbom, D. è1994è. LISREL 8 user's reference guide. Mooresville, IN: Scientiæc Software. ë12ë Magnus, J., and Neudecker, H. è1999è. Matrix diæerential calculus with applications in statistics and econometrics. New York: Wiley. ë13ë Muthçen, B. è1993è. Goodness of æt test with categorical and other nonnormal variables. In K. A. Bollen and J. S. Long èeds.è, Testing structural equation models èpp. 205í234è. Newbury Park: Sage. ë14ë Rao, C.R., è1973è. Linear Statistical Inference and Its Applications, 2nd. edit. New York: Wiley. 12

14 ë15ë Satorra, A. è1989è. Alternative test criteria in covariance structure analysis: A uniæed approach. Psychometrika, 54, 131í151. ë16ë Satorra, A. è1992è. Asymptotic robust inferences in the analysis of mean and covariance structures. Sociological Methodology, 22, 249í278. ë17ë Satorra, A. è1999è. Scaled and adjusted restricted tests in multi-sample analysis of moment structures. To appear in Innovations in Multivariate Statistical Multivariate Analysis: A Festschrift for Heinz Neudecker èr.d.h. Heijmans, D.S.G. Pollock, and A. Satorra, edts. è, Dordrecht: Kluwer Academic Publishers ë18ë Satorra, A., and Bentler, P. M. è1986è. Some robustness properties of goodness of æt statistics in covariance structure analysis ASA Proceedings of the Business and Economic Statistics Section, 549í554, Alexandria, VA: American Statistical Association. ë19ë Satorra, A., and Bentler, P. M. è1988aè. Scaling corrections for chisquare statistics in covariance structure analysis. ASA 1988 Proceedings of the Business and Economic Statistics Section, 308í313, Alexandria, VA: American Statistical Association. ë20ë Satorra, A., and Bentler, P. M. è1988bè. Scaling corrections for statistics in covariance structure analysis. UCLA Statistics Series è 2, University of California, Los Angeles. ë21ë Satorra, A., and Bentler, P. M. è1994è. Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye and C. C. Clogg èeds.è, Latent variables analysis: Applications for developmental research èpp è. Thousand Oaks, CA: Sage. ë22ë Yuan, K. -H., and Bentler, P. M. è1997è. Mean and covariance structure analysis: Theoretical and practical improvements. Journal of The American Statistical Association, 92, 767í774. ë23ë Yuan, K. -H., and Bentler, P. M. è1998è. Normal theory based test statistics in structural equation modelling, British Journal of Mathematical and Statistical Psychology, 51,