Spatial Regression 9. Specification Tests (1) Luc Anselin http://spatial.uchicago.edu 1
basic concepts types of tests Moran s I classic ML-based tests LM tests 2
Basic Concepts 3
The Logic of Specification Testing some of the standard regression assumptions may be violated in practice how to detect if this is the case assumptions pertain to error terms BUT diagnostics are based on regression residuals 4
Null Hypothesis classic linear regression model y = Xβ + e with E[ee ] = σ 2 I correct specification no non-linearities no omitted variables error terms have constant variance and are not correlated error terms are not correlated with explanatory variables (exogeneity) 5
Some Possible Alternatives heteroskedasticity: error variance is not constant spatially correlated errors (e.g., a SAR process), i.e., nuisance spatial autocorrelation spatial correlation due to an omitted spatially lagged dependent variable, i.e., substantive spatial autocorrelation more than one alternative possible 6
H1: Spatial Lag Model true model is y = ρwy + Xβ + e (1) model estimated is y = Xβ + e (2) (2) is obtained from (1) by setting ρ = 0 (2) is constrained model (ρ is constrained to 0), (1) is unconstrained model (ρ can take on any value) 7
H1: Spatial SAR Error Model true model is y = Xβ + e with e = λwe + u (1) model estimated is y = Xβ + e (2) (2) is obtained from (1) by setting λ = 0 (2) is constrained model (λ is constrained to 0), (1) is unconstrained model (λ can take on any value) 8
Types of Tests 9
Formal Structure of Hypotheses H 0: constrained, H1: unconstrained For Spatial Lag Model H0: ρ = 0, H1: ρ 0 For Spatial Error Model H0: λ = 0, H1: λ 0 10
Types of Tests no specific alternative = diffuse tests specific alternative model = focused tests 11
Diffuse Tests reject absence of spatial autocorrelation alternative is presence of spatial autocorrelation, but not specified what form or process example: Moran s I test for regression residuals 12
Focused Tests reject the null hypothesis against a fully specified alternative model, such as a spatial lag model or a spatial SAR error model based on maximum likelihood (ML) principles examples: Wald test (= asymptotic t-test), Likelihood Ratio test (LR), Lagrange Multiplier test (LM) 13
Moran s I 14
Moran s I regression residuals: u = y - Xb I = [ u Wu/S0 ] / [ u u/n ] with S0 = Σij wij for row-standardized weights, S 0 = N and I = u Wu / u u formal similarity to Durbin-Watson statistic for time series (serial) error correlation 15
Inference based on a normal approximation OLS residuals are not error terms but are related to the error terms: u = y - X [ (X X) -1 X y ] = [ I - X(X X) -1 X ](Xβ + e) = (Xβ - Xβ) + [ I - X(X X) -1 X ]e = Me M is an idempotent matrix: MM = M = M = M M 16
Inference (continued) relation between u and e allows one to connect expressions in residuals to the properties of the error terms (under the null hypothesis) example: E[u u] = E[e Me] = E[tr(Mee )] = σ 2 tr(m) = σ 2 (N-k) E[u Wu] = E[e M WMe] = E[tr(M WMee )] = σ 2 tr(m WM) = σ 2 tr(wmm ) = σ 2 tr(wm) since under the null E[ee ] = σ 2 I 17
Moments of Moran s I E[I] = tr(mw) / (N-K) V[I] = (T / [(N-K)(N-K-2)]) - (E[I])2 with T = tr(mwmw ) + tr(mw) 2 + [tr(mw)] 2 standardized z-value: z(i) = (I - E[I]) / V[I] 18
Inference (again) use z-value in standard normal approximation (a large sample approximation) also exact finite sample results (under assumption of normality) permutation is NEVER an option for Moran s I derived from residuals 19
Moran s I default calculation - no inference Moran s I with inference highly significant even though value is only 0.12 20
Interpretation lack of rejection means the model is correctly specified rejection of the null for Moran s I needs to be interpreted with caution: not clear what the alternative is (e.g., error or lag) Moran s I has power against many alternatives (such as non-normality, heteroskedasticity) Moran s I is a great misspecification test, but not that useful in a specification search - not clear what the next step should be 21
Classic ML Based Tests 22
Three Different Perspectives on Testing always: compare constrained to unconstrained model value of estimate compared to null hypothesis fit of estimate: maximized log-likelihood vs. constrained log-likelihood slope of likelihood functions vs. zero (at maximum) 23
Test Statistics principle is to compare the value of the constrained to the unconstrained model the parameter value, fit or slope of the likelihood function if unconstrained and constrained model are different enough then reject the null hypothesis based on variance of the statistics 24
fit slope value maximum likelihood (ML) based tests 25
Three Classic Tests Wald test (or asymptotic t-test): based on value of estimate Likelihood Ratio (LR) test: based on difference in fit Lagrange Multiplier (LM) or Rao Score test: based on slope of likelihood function asymptotically equivalent in finite samples: W LR LM 26
Wald Test (or Asymptotic t-test) requires the estimation of the alternative model, i.e., either a spatial lag or a spatial error difference between the ML estimate of the spatial parameter and zero requires variance estimate t = θ / (asy Var[θ]) N(0,1) or W = t 2 χ 2 (1) 27
>>> Example: ML-Lag estimation Wald test as square of asymptotic t-test W = 7.478 2 = 55.9 28
Likelihood Ratio Test test on difference of maximized log likelihood between spatial model and null model requires estimation of two models: both the null (non-spatial) and the alternative (spatial) LR = 2[L(θ) - L(θ=0)] χ2 (1) 29
constrained model: L(θ=0) = -855.2 30
unconstrained model: L = -832.9 31
LR test 2[L(θ) - L(θ=0)] = 2[-832.9 + 855.2] = 2[22.3] = 44.6 32
Lagrange Multiplier Test test on significance of slope (gradient) of likelihood function (score) only requires estimation of null model based on OLS residuals 33
LM Test Statistic LM = d I -1 d χ 2 (1) d = L/ θ θ=0, d is score I = -E[ 2 L/ θ θ ] θ=0, with I information matrix derivations based on the alternative (spatial) model, but parameter values from the null model (OLS residuals) 34
LM Tests 35
LM Error Test 36
Null and Alternative Hypothesis H0: y = Xβ + e, E[ee ]= σ 2 I H 1: SAR error or SMA error locally equivalent alternatives e = λwe + u (SAR), H0: λ = 0 e = λwu + u (SMA), H0: λ = 0 37
LM or RS Approach partition parameter vector [ λ β, σ 2 ] score evaluated at λ=0 L/ λ λ =0 partioned information matrix evaluated at λ = 0 -E[ 2 L/ λ λ ] λ=0 38
Score L/ λ = - tr(i - λw) -1 W + (1/σ 2 )u Wu evaluated at λ=0 yields d = tr(w) + (1/σ 2 )u Wu with tr(w) = 0 (zero elements on diagonal) d = (1/σ 2 )u Wu 39
Information Matrix -E[ 2 L/ λ λ ] = trw(i - λw) -1 W(I - λw) -1 + trw'(i - λw ) -1 W(I - λw) -1 evaluated at λ=0 yields tr WW + trw'w = tr(ww+w W) 40
LM Statistic LMERR = d I-1 d = (u Wu/σ 2 )[tr(ww+w W)] -1 (u Wu/σ 2 ) = [u Wu/σ 2 ] 2 / [tr(ww+w W)] χ 2 (1) 41
LMError and Moran s I MI = u Wu / u u = u Wu / nσ 2 = (u Wu/σ 2 )/n u Wu/σ 2 = nmi LMError = (nmi)2 / tr(ww+w W) 42
LM Lag Test 43
Principle same as LM Error Test score and information matrix from ML lag model impose ρ = 0 complex expressions 44
LM Lag Test (Anselin 1988) LM-Lag = [ e Wy / σ 2 ] 2 / T1 χ 2 (1) T1 = (WXb) M(WXb)/σ 2 + T first term: residual sum of squares of WXb on X, i.e., regression of spatial lag of predicted values (Xb) on the original regressors T same trace term as for LM-Error T = tr(ww + W W) 45
LM-Lag = 51.4 LM-Error = 53.5 both reject the null hypothesis 46