1.5 Testing and Model Selection

Transcription

1 1.5 Testing and Model Selection The EViews output for least squares, probit and logit includes some statistics relevant to testing hypotheses (e.g. Likelihood Ratio statistic) and to choosing between specifications (e.g. Akaike information criterion). Similar statistics are produced when other modelsarefittedandiwillexplainwhattheymean. To fix ideas consider a simple 1st year example of testing. Supposewehave16independentobservationsy 1,...,y 16 froman(µ,1)populationandwewishtotestthenull hypothesish 0 againstthealternativehypothesish A where H 0 : µ=10 H A : µ 10. Weknowthatthepopulationvarianceis1. The usual procedure is to base the test on the samplemeany. ThedistributionofY isknowntoben(µ, 1 16 ). IfH 0 istrue, Y 10 1/4 isn(0,1). At significance level 5% we would reject H 0 if Y 10 1/4 >z =1.96. E.g. ify=9.5,wehave Y 10 =2>1.96and 1/4 werejecth 0 atthe5%level. Alternatively we compute the probability value corresponding to 2, viz Most packages including EViews report the outcomes of significance tests in terms of p-values

2 1. The first compares the MLE of µ, the sample mean 9.5 here, with the hypothesised value 10 and asks whether the numbers are close, given sampling fluctuations. The variance of the MLE is 1/16 (σ 2 /n). In more complicated situations this is called the Wald test after Abraham Wald who discussed its use in more general settings in the 1940s Thereare3waysofproducingthisoneprocedure. In more complicated cases the 3 methods lead to different procedures. All 3 are used in practical econometrics. Thenext2methodsarebestunderstoodusingadiagramoflnL(µ;y)theloglikelihoodfunctionforµ lnl log likelihood 2. This method compares the value of the log likelihoodat9.5(itsmaximumvalue)withitsvalue at µ = 10; if the difference is is close to zero (taking into account sampling fluctations) this is evidence in favour of H 0. The difference in the loglikelihoodvaluesisthelogoftheratioofthe likelihoods and this is called the likelihood ratio test. Itwasintroducedinthelate1920sbyNeyman and Egon Pearson 3. Thethirdconsidersthefirstderivativeofthelog likelihood at the hypothesised value µ=10 and µ

3 asks if it is close to zero given sampling fluctations. Ifitis,theevidencefavoursH 0. Thisprocedure has two names the score test because the slopeoftheloglikelihoodiscalledthescoreand the Lagrange multiplier test because we could look at the multiplier associated with the constraint µ = 10 and reject the hypothesis that µ = 10 if the multiplier is big. The multiplier turnsouttobethesameastheslope! Thescore comesfromc.r.raointhe40sandthelmfrom S.D.Silveyinthe50s. After some algebra all of these principles lead to the procedure given above see Exercises. In more complex situations the procedures generally produce different test statistics. The trinity in practice Thedescriptionsofthe3methodsgiveonlythecore idea of each method. Econometric models rarely have only one unknown parameter and so the methods, as implemented, are more complex. Also the estimators concerned are not usually exactly normal but only normal in large samples. Important extensions θisavectorandweareinterestedinonecomponent of this vector, H 0 : θ i =θ i versus H A : θ i θ i. Thus in the probit model say θ = (β 0,β 1 ) and H 0 :β 1 =0.Herethehypothesisdoesnotinvolve β 0.

4 θ is a vector and we are interested in a function hofθ, H 0 : h(θ)=0versus H A : h(θ) 0. Thus in a regression model θ = (β 0,β 1,β 2,σ 2 ) we might be interested in whether β 0 = β 1 β 2, i.e. intestingh 0 :β 0 β 1 β 2 =0. Wedescribeingeneraltermshowthe3testprinciples work when there is a model with parameter vector θ andwewishtotest H 0 : h(θ)=0versus H A : h(θ) 0. We call h(θ) = 0 the restriction. The hypothesis H 0 says that the restriction is satisfied. There could be more than one restriction being tested, e.g. in regressionwemighttestβ 2 =0,β 3 =0. 0 subject to the usual qualification about sampling variability. TheWaldtesttakesthe θunrestrictedmaximumlikelihoodestimateofθandaskswhetherh( θ)iscloseto Thelikelihoodratiotesttakes θtheunrestrictedmaximum likelihood estimate and evaluates the likelihood at this value, L( θ). It also takes θ R the restricted maximum likelihood estimate and evaluates the likelihood at this value, L( θ R ). The likelihood ratio is L( θ R )/L( θ). The test statistic EViews calculates as the likelihood ratio test is a transform of this, 2lnL( θ R )/L( θ)= 2(lnL( θ R ) lnl( θ)). The Lagrange multiplier test uses θ R the restricted maximum likelihood estimate and evaluates the score atthispoint. Weaskwhetherthescoreiscloseto0, with the usual qualification. There is a large sample for these tests linked to the large sample theory of maximum likelihood estimators.

5 In large samples the estimators are approximately normally distributed and the test statistics are approximatelychi-squaredχ 2. Notes on the relationship between the standard normalandχ 2 : if Z N(0,1) then Z 2 is a chi-squared random variablewith1degreeoffreedom,writtenχ 2 1. ifz 1 andz 2 areindependentn(0,1)thenz Z 2 2 isachi-squaredrandomvariablewith2degree offreedom,writtenχ 2 2. ifz 1,...,Z k areindependentn(0,1)thenz Zk 2 is a chi-squared random variable with k degree of freedom, written χ 2 k. The probability densities for different degrees of freedom look so Figure 1: If the hypothesis has 2 components, e.g. β 2 = 0,β 3 = 0, the test statistic (W, LR or LM) is a chi-squared random variable with 2 degree of freedom. The EViews output for the modal split exercise includes the values Loglikelihood ,i.e. lnl( θ) Restrictedloglikelihood ,i.e. lnl( θ R )

6 LRstatistic(1df) ,i.e. 2lnL( θ R )/L( θ)= 2(lnL( θ R ) lnl( θ)) Probability(LR stat) 4.30E-05, based on the uppertailoftheχ 2 1 distribution. Thehypothesisbeingtestedisβ 1 =0.Whenashere there is a single restriction the tests can be presented in2equivalentforms,eitherasaχ 2 1 orasastandard normalwhichisthesquarerootofaχ 2 1. ML and Model Selection Criteria One way to choose between different specifications (e.g. betweentheprobitandlogitmodels)istousea model selection criterion. Several of these appear in EViews. All are associated with maximum likelihood. The best known is the Akaike information criterion (AIC). The Akaike information criterion(aic) is an adjusted log likelihood value adjusted for the number of parameters in the model. AIC=lnL( θ) p where p is the number of estimated parameters, the numberofelementsinθ. TheAICiscalculatedforallthemodelsunderconsideration and the model with the highest AIC chosen. In the probit/logit models of modal split there are 2 parameters β 0,β 1 and so p = 2. Here with an equal number of parameters the choice between the models would depend on the value of the maximised loglikelihoodlnl( θ). There are other criteria based on different penalties forthenumberofparameters. Insteadofthepofthe AIC,thepenaltymayalsodependonthesamplesize, n.

7 TheSchwarzcriterionhas 1 2 plnn The Hannan-Quinn criterion has p ln(ln n). These criteria were introduced because they have a consistency property. It is reasonable to require that as the number of observations tends to infinity the probability of choosing the right model should tend to unity. The AIC (and the R 2 and R 2 from regression theory) do not have this consistency property. Schwarz and Hannan-Quinn have this property.