10. Comparative Tests among Spatial Regression Models. Here we revisit the example in Section 8.1 of estimating the mean of a normal random

Transcription

1 Part III. Areal Data Aalysis 0. Comparative Tests amog Spatial Regressio Models While the otio of relative likelihood values for differet models is somewhat difficult to iterpret directly (as metioed above), such likelihood ratios ca i may cases provide powerful test statistics for comparig models. I particular, whe two models are ested i the sese of expressio (9.4.) above, it turs out that the asymptotic distributio of such ratios ca be obtaied uder the (ull) hypothesis that the simpler model is the true model. To develop such tests, we begi i Sectio 0. below with a simple oe-parameter example where the geeral ideas to be developed ca be give a exact form. 0. A Oe-Parameter Example Here we revisit the example i Sectio 8. of estimatig the mea of a ormal radom variable, Y N(, ), with kow variace,, give a sample, y ( y,.., y ), of size. The relevat likelihood fuctio is the give by expressio (8..) as i i L( ) L ( y, ) log ( y ) (0..) ad the resultig maximum-likelihood estimate of, is agai see from expressio (8..) to be precisely the sample mea, ˆ y. But rather tha simply estimatig, suppose that we ow wat to test whether 0, or more geerally to test the ull hypothesis, H 0: 0, for some specified value, 0. The uder H 0 the likelihood value i (0..) becomes: (0..) 0 0 i i 0 L( ) L ( y, ) log ( y ) As show i Figure 0. below, it seems reasoable to argue that the likelihood of 0 relative to the maximum likelihood at ˆ should provide some idicatio of the stregth 0 ˆ L( ˆ ) L( 0) L( ) Figure 0. Likelihood Comparisos ESE 50 III.0- Toy E. Smith

2 Part III. Areal Data Aalysis of evidece i sample y for (or agaist) hypothesis H 0. I terms of log likelihoods, such relatios are expressed i terms of the differece betwee L( ˆ ) ad L( 0). But followig stadard covetios, we here refer to such log-differeces as likelihood ratios. Moreover, sice L( ˆ ) L( 0) by defiitio, it is atural to focus o the oegative differece, L( ˆ ) L( 0). If the distributio of L( ˆ ) L( 0) ca be determied uder H 0, the this statistic ca be used to test H 0. I particular, if L( ˆ ) L( 0) is sufficietly large, the this should provide statistical grouds for rejectig H 0. With this i mid, observe that by cacelig the commo terms i the log likelihood expressios, ad recallig that ˆ y, we see that this likelihood ratio ca be writte as (0..3) L ( ˆ ) L ( 0) ( y ˆ ) ( i y i 0) i i Thus it follows that [( y y ) ( y ) ] i i i i 0 [( y y y y ) ( y y )] i i i i i 0 0 [ yy y y ] i i 0 0 y y y y y y 0 0 y ( ) i i 0 i i 0 0 (0..4) y 0 L 0 [ L( ˆ ) ( )] / But uder the ull hypothesis, H 0, the stadardized mea i brackets is stadard ormal: (0..5) y 0 / ~ N(0,) So the right-had side of (0..4) is distributed as the square of a stadard ormal variate, which is kow to have a chi square distributio,, with oe degree of freedom, i.e., 3 (0..6) y / 0 ~ ESE 50 III.0- Toy E. Smith

3 Part III. Areal Data Aalysis where the desity of is plotted o the right. So we may coclude that this likelihoodratio statistic is chi-square distributed (up to a factor of ) as: (0..7) [ L( ˆ ) L( )] ~ 0 [As metioed i Sectio 9, this factor of is closely related to the same factor appearig i the pealized likelihood fuctios developed there.] Note that we are implicitly comparig two models here, oe with a sigle free parameter ( ) ad the other a ested special case where has bee assiged a specific value, 0 (typically, 0 0 ). But the same likelihood-ratio procedure ca be used for much more geeral comparisos betwee a full model ad some special case, deoted as the restricted model. Here we simply summarize the mai result. Suppose that the full model is represeted by a log likelihood fuctio, L( y), with parameter vector, (,.., K ), ad that the restricted model is defied by imposig a set of m K restrictios o these parameters that are represetable by a vector, g ( g j : j,.., m), of (smooth) fuctios as relatios of the form, (0..8) g ( ) 0, j,.., m j I our simple example above, there is oly oe relatio, amely, g ( ) 0 0. If the maximum-likelihood estimate for full model is deoted by ˆ, ad if the maximumlikelihood estimate, ˆg, for the restricted model is take to be the (uique) solutio of the costraied maximizatio problem, (0..9) L( ˆ g y) max { : g( ) 0} L( y) the it agai follows that the relevat likelihood-ratio statistic, L( ˆ y) L( ˆ g y), is oegative. I this more geeral settig, if it is hypothesized that the restricted model is true (i.e., that the true value of satisfies restrictios, g ), the uder this ull hypothesis it ca be show that L( ˆ y) L( ˆ g y) is ow asymptotically chi square distributed (up to a factor of ) with degrees of freedom, m, equal to the umber of restrictios defied by g : (0..9) [ L( ˆ y) L( ˆ y)] ~ g m This result, kow as Wilk s Theorem, is developed, for example, i Sectio 3.9 of the olie Lecture Notes i Mathematical Statistics (003) by R.S. Dudley at MIT ( mathematical-statistics-sprig-003/lecture-otes/). ESE 50 III.0-3 Toy E. Smith

4 Part III. Areal Data Aalysis This family of likelihood-ratio tests provides a geeral framework for comparig a wide variety of ested models. Moreover, as i the oe-parameter case of (0..7) above, the basic ituitio is essetially the same for all such tests. I particular, sice the full maximum likelihood, L( ˆ y), is almost surely larger tha the restricted maximum likelihood, L( ˆ g y), the oly questio is whether it is sigificatly larger. If so, the it ca be argued that the restricted model should be rejected o these grouds. If ot, the this suggests that the full model adds little i the way of statistical substace, ad thus (by Occam s razor) that the simpler restricted model should be preferred. For example, i the OLS case above, the key questio is whether a give parameter, such as, is sigificatly differet from zero (all else beig equal). If so, the this idicates that the larger model icludig variable, x, yields a better predictor of y tha the same model without x. I the followig sectios, we shall employ this strategy to compare the SEmodel ad SL-model from a umber of perspectives. 0. Likelihood-Ratio Tests agaist OLS Here we begi by observig that sice SEM ad SLM are o-ested models i the sese that either is a special case of the other, it is ot possible to compare them directly i terms of likelihood-ratio tests. But sice OLS is precisely the 0 case of each model, both SEM ad SLM ca be compared with OLS i terms of such tests. Thus, by usig OLS as a bechmark model, we ca costruct a idirect compariso of SEM ad SLM. For example, if the improvemet i likelihood of SEM over OLS is much greater tha that of SLM over OLS for a give data set, ( yx, ), the i this sese it ca be argued that SEM provides a better model of ( yx, ) tha does SLM. To operatioalize such comparisos, we start with SEM ad for a give data set, ( yx, ), let ( ˆ SEM, ˆ SEM, ˆ SEM ) deote the maximum likelihood estimates obtaied usig the SEM likelihood fuctio, L(,, y, X), i (7.3.4) above [as i expressios (7.3.0) through (7.3.)]. The the correspodig SEM maximum-likelihood value ca be deoted by: (0..) Lˆ L( ˆ, ˆ, ˆ y, X) SEM SEM SEM SEM Similarly, if for OLS we let ( ˆ OLS, ˆ OLS ) deote the maximum-likelihood estimates i (7..6) ad (7..9) obtaied for ( yx, ) by maximizig (7..4), the the correspodig OLS maximum-likelihood value ca be deoted by (0..) Lˆ L ˆ ˆ y X OLS ( OLS, OLS, ) Oe may ask how this likelihood-ratio test i the OLS case relates to the stadard (Wald) tests of sigificace, such as i expressio (8.4.) above (with 0 ). Here it ca be show [as for example i Sectio 3.4 of Davidso ad MacKio (993)] that these tests are asymptotically equivalet. ESE 50 III.0-4 Toy E. Smith

5 Part III. Areal Data Aalysis Fially, sice the likelihood fuctio i (7..4) is clearly the special case of (7.3.4) with 0 [or more precisely, with g(,, ) i (0..8) ], it follows from the geeral discussio above that uder the ull hypothesis, 0, it must be true that the likelihood ratio, LR ˆ ˆ SEM / OLS [ LSEM LOLS ], is distributed as chi square with oe degree of freedom, i.e., that (0..3) LR [ L ˆ Lˆ ] ~ SEM / OLS SEM OLS Similarly, if ( ˆ SLM, ˆ SLM, ˆ SLM ) deotes the maximum likelihood estimates obtaied usig the SLM likelihood fuctio, L(,, y, X), i (7.4.) above [as i expressios (7.4.) through (7.4.4) ], the we may deote the resultig SLM maximum-likelihood value by: (0..4) Lˆ L( ˆ, ˆ, ˆ y, X) SLM SLM SLM SLM The i the same maer as (0..3), it follows that uder the ull hypothesis that 0 for SLM, we also have (0..5) LR [ L ˆ Lˆ ] ~ SLM / OLS SLM OLS For the Eire case, these two likelihood ratios ad associated p-values are reported i Figure 7.7 as (0..6) LR LR / ( Pval.0066) ad SEM OLS (0..7) LR LR / ( Pval.00007) SLM OLS So for example, if OLS were the correct model, the the chace of obtaiig a likelihood ratio, LR SLM / OLS, as large as would be less tha 7 i 00,000. Moreover, while the p-value for LR SEM / OLS is also quite small, it is relatively less sigificat tha for SLM. Thus a compariso of these p-values provides at least idirect evidece that SLM is more appropriate tha SEM for this Eire data. But give the idirect ature of this compariso, it is atural to ask whether there are ay more direct comparisos. Oe possibility is developed below, which will be see to be especially appropriate for the case of row ormalized spatial weights matrices. ESE 50 III.0-5 Toy E. Smith

6 Part III. Areal Data Aalysis 0.3 The Commo-Factor Hypothesis Here we start by recallig from Sectio 6.3. that if X ad are partitioed as X [, X v] ad ( 0, v ), respectively, the a alterative modelig form is provided by the Spatial Durbi model (SDM), (0.3.) Y WY X WX N I, ~ (0, ) 0 v v v But this model ca be viewed as a special case of the SLM model i the followig way. If we group terms i (0.3.) by lettig X [, X, WX ] ad (,, ) so that SDM v v SDM v (0.3.) 0 X SDM SDM [ Xv WXv ] v 0 Xvv WXv, the (0.3.) ca be rewritte as, (0.3.3) Y WY X N I SDM SDM, ~ (0, ) which is see to be a istace of SLM i expressio (6..). Moreover, if W is row ormalized, the SEM ca i tur be viewed as a special case of SDM. To see this, observe first that the reduced form of SEM i expressio (6..9) ca be expaded ad rewritte as follows: (0.3.4) Y X ( I W) ( I W) Y ( I W) X Y WY ( X WX) Y WY X WX So by employig the otatio i (0.3.), we see that (0.3.5) Y WY [ 0 Xvv] W[ 0 Xvv] WY X [ W WX ] 0 v v 0 v v Fially, if W is row ormalized, the by expressio (3.3.30) it follows that W. So by lettig b 0 ( ) 0, ad groupig the two uit vector terms, we see fially that the SEM model i (0.3.4) becomes ESE 50 III.0-6 Toy E. Smith

7 Part III. Areal Data Aalysis (0.3.6) Y WY b0 X WX ( ) v v v v which is precisely SDM i (0.3.) uder the coditio that (0.3.7) v This coditio is usually formulated as a ull hypothesis, desigated as the Commo Factor Hypothesis, ad writte as (0.3.8) H : 0 CF v Uder this hypothesis, it follows that SEM is formally a restrictio of SDM i the sese of expressio (0..8), where the relevat vector, g, of restrictio fuctios is ow give by g(, 0, v,, ) v. The umber of restrictios (i.e., dimesio of g) is here simply the umber of explaatory variables, k. Give this relatioship, oe ca the employ likelihood-ratio methods to test the appropriateess of SDM versus SEM. To do so for ay give ay data set, ( yx, ), we ow let ( ˆ SDM, ˆ SDM, ˆ SDM ) deote the maximum likelihood estimates obtaied by applyig the SLM likelihood fuctio, L( SDM,, y, X), i (7.4.) to the SLM form of SDM i (0.3.3) above. I these terms, the resultig SDM maximum-likelihood value is the give by: (0.3.9) Lˆ L( ˆ, ˆ, ˆ y, X) SDM SDM SDM SDM Fially, if we let Lˆ ˆ SEM L( SEM, ˆ SEM, ˆ SEM y, X) deote the maximum-likelihood value of the SE-model i (0.3.6) [viewed as a SD-model restricted by (0.3.8)], the uder the SEM ull hypothesis, we ow have (0.3.0) LR [ Lˆ Lˆ ] ~ SDM / SEM SDM SEM k where agai, k, is the umber of explaatory variables i SEM. The results of this comparative test are part of the SEM output, deoted by Com-LR. For the case of Eire, the result reported i Figure 7.7 is (0.3.) Com-LR = ( Pval = ) ad shows that SDM fits this Blood Group data far better tha SEM. This ca largely be explaied by otig from (0.3.) ad (0.3.3) that the reduced form of the SDM model is give by (0.3.) Y B ( X WX ) B 0 v v v ESE 50 III.0-7 Toy E. Smith

8 Part III. Areal Data Aalysis ad thus cotais the Rippled Pale term, B Xv v ( B x), which was show to yield a strikig fit to this data. So a strog result is ot surprisig i this case. Fially, it should be oted that while the above aalysis has focused o row-ormalized matrices i order to iterpret the SLM versio of SEM as a Spatial Durbi model, this restrictio ca i priciple be relaxed. I particular, whe W, it is possible to treat the vector, W, as represetig the sample values of a additioal explaatory variable ad thus modify (0.3.) to (0.3.3) 0 X [ ] v SDM SDM Xv W WXv 0 Xvv 0W WXv 0 With this additio, SEM ca still be viewed formally as a istace of SLM. Moreover, if the additioal restrictio, 0 0 0, is added to yield a set of k restrictios, the this ew likelihood ratio must ow be distributed as k uder the ull hypothesis of SEM. So while the problematic ature of this artificial explaatory variable complicates the iterpretatio of the resultig test, it ca still be argued that the presece of the spatial lag term, WY, suggests that SLM may yield a better fit to the give data tha SEM. 0.4 The Combied-Model Approach A fial method of comparig SEM ad SLM is provided by the combied model (CM) developed i Sectio 6.3. above, which for ay give spatial weights matrix, W, ca be writte as [see also expressio (6.3.3) ]: (0.4.) Y WY X u uwu N I,, ~ (0, ) Here is clear that SEM is the special case with 0, ad SLM is the special case with 0. So these two models are see to lie betwee OLS ad the Combied Model, as i Figure 0. below: Combied Model SEM SLM OLS Figure 0.. Model Relatios ESE 50 III.0-8 Toy E. Smith

9 Part III. Areal Data Aalysis I the same way that OLS served as a lower bechmark for comparig SEM ad SLM, the Combied Model ca thus serve as a upper bechmark. Here the oly issue is how to estimate this more complex model. To do so, we start by observig from (6.3.4) that the reduced form of this model ca be writte as: (0.4.) Y X, ~ N(0, V ) where (0.4.3) (0.4.4) (0.4.5) ( ) X I W X ( I W) ( I W) V ( I W) ( I W) ( I W) ( I W) So it should be clear that this model is simply aother istace of GLS, where i this case coditioig is o the pair of spatial depedece parameters, ad. So for the parameter vector, (,,, ), the correspodig likelihood fuctio takes the form: (0.4.6) L( y) log( ) log( ) log V ( yx ) V ( y X ) ad the correspodig coditioal maximum-likelihood estimates for ad give ad ow take the respective forms: (0.4.7) (0.4.8) ˆ ( ) X V X X V y ˆ ( ) ( ) ˆ ˆ y X V y X By substitutig (0.4.7) ad (0.4.8) ito (0.4.6), we may the obtai a cocetrated likelihood fuctio for ad, deoted by: (0.4.9) ˆ L (, ) (, ˆ,, ) c y L y Fially, by maximizig this two-dimesioal fuctio to obtai maximum-likelihood estimates, ˆ ad ˆ, we ca substitute these ito (0.4.7) ad (0.4.8) to obtai the correspodig maximum-likelihood estimates, ˆ ˆ ˆ ad ˆ ˆ ˆ. This estimatio procedure is programed i the MATLAB program, sac.m, (Spatial Autocorrelatio Combied) writte by James Lesage, ad ca be foud i the class directory at: >> sys50/matlab/lesage_7/spatial/sac_models ESE 50 III.0-9 Toy E. Smith

10 Part III. Areal Data Aalysis While the parameter estimates, ˆ ad ˆ, obtaied by this procedure ofte ted to be colliear (i view of their commo role i modifyig the same weight matrix, W), the correspodig maximum-likelihood value, (0.4.0) Lˆ L( ˆ, ˆ, ˆ, ˆ y) CM ˆ ˆ ˆ ˆ cotiues to be well defied ad umerically stable. This value ca thus be used to test the relative goodess of fit of the two restricted models, SEM ad SLM. I particular, it follows by the same argumets as above that uder the SEM ull hypothesis ( 0 ) we have (0.4.) Lˆ [ Lˆ Lˆ ] ~ CM / SEM CM SEM ad similarly, that uder the SLM ull hypothesis ( 0) we have (0.4.) Lˆ [ Lˆ Lˆ ] ~ CM / SLM CM SLM The results of these respective tests for the Eire case are as follows: (0.4.) LR / 0.9 ( Pval.0009) CM SEM (0.4.3) LR /.49 ( Pval.45) CM SLM Thus the Combied Model is see to yield a sigificatly better fit tha SEM, but ot SLM. So relative to this CM bechmark, it ca agai be cocluded that SLM yields a better fit to the Eire data tha does SEM. ESE 50 III.0-0 Toy E. Smith