Econ 583 Homework 7 Suggested Solutions: Wald, LM and LR based on GMM and MLE Eric Zivot Winter 013 1 Wald, LR and LM statistics based on generalized method of moments estimation Let 1 be an iid sample of Bernoulli random variables; that is, each has density (; 0 )= 0(1 0 ) 1 Instead of using ML estimation, consider instead estimation of 0 using the generalized method of moments (GMM). Here 1. Consider GMM estimation of using the single moment condition ( )= Show that [( )] = 0 for = 0,and[( )] 6= 0for 6= 0 Additionally, show that = [( 0 ) ]= 0 (1 0 ) [( )] = [ ] = 0 Clearly, [( )] = 0 for = 0,and[( )] 6= 0for 6= 0 Next, = [( 0 ) ]=[( 0 ) ]=( )= 0 (1 0 ) since has a Bernoulli distribution with parameter 0 Note that = ( 0 ) 1. Using the sample moment () = P 1 ( ), show that the GMM estimator of is ˆ = P 1 What is the value of the GMM objective function ( ˆ 1 ˆ )= (ˆ ) 0 ˆ 1 (ˆ )= (ˆ ) ˆ 1? How would you consistently estimate? 1
Equating the sample moment with the population moment gives the estimating equation (ˆ ) = 1 ( ˆ )= 1 ˆ =0 ˆ = 1 = Hence, the GMM estimator is the same as the ML estimator. The GMM objective function evaluated at ˆ is! ( ˆ 1 ˆ )= (ˆ ) ˆ 1 = 1 ˆ ˆ 1 =0 regardless of the value of ˆ since ˆ = P 1 This is to be expected since with one moment condition and one parameter to estimate the model is exactly identified. Since = 0 (1 0 ) a nature estimate is ˆ = ˆ (1 ˆ ) Alternatively, one could use the the usual non-parametric estimator of : ˆ = 1 ( ˆ ) = 1 ( ˆ ) 3. Determine the asymptotic normal distribution for ˆ Is this distribution the same as the asymptotic normal distribution for ˆ? Using the asymptotic normality result for GMM estimators we have Now, so that and ˆ ( 0 1 ( 0 1 ) 1 ) ( 0 ) = = [( 0 ) ] 0 ( 0 ) 0 = 0 ( 0 )= 1 = 0 (1 0 ) ( 0 1 ) 1 = = 0 (1 0 ) ˆ ( 0 1 0 (1 0 )) which is equivalent to the asymptotic distribution for the MLE. Next, consider testing the hypotheses 0 : = 0 vs. 1 : 6= 0 using a 5 significance level.
4. and 5. Derive the Wald, LR and LM statistics for testing the above hypothesis. Because is a scalar, these statistics have the form = (ˆ 0 ) ( (ˆ ) ˆ 1 ) = ( ( 0 ) ˆ 1 ( 0 )) ( ( 0 ) ˆ 1 ) 1 = ( ˆ 1 0 ) ( ˆ 1 ˆ ) where () = () For the Wald and LR statistic it is useful to estimate using ˆ For the LM statistic, it is useful to an estimate of imposing = 0 Are the GMM Wald, LM and LR statistics numerically equivalent? Compare the GMM Wald, LR and LM statistics with the ML equivalents. Do you see any similarities or differences? The Wald statistic has the form = (ˆ 0 ) ( (ˆ ) ˆ 1 ) = (ˆ 0 ) ˆ (1 ˆ ) which is equivalent to the ML Wald statistic. For the LM statistic, first note that ( 0 ) = 1 ( 0 ) = 1 0 = ˆ 0 ˆ = 0 (1 0 ) Then, the LM statistic has the form = ( ( 0 ) ˆ 1 ( 0 )) ( ( 0 ) ˆ 1 ) 1 = (ˆ 0 ) 0 (1 ˆ 0 ) which is identical to the ML LM statistic. For the LR statistic, recall that (ˆ ˆ 1 )=0since the model is just identified. Therefore, the LR statistic has the form = ( ˆ 1 0 ) ( ˆ 1 ˆ ) = ( ˆ 1 0 )= ( 0 ) 0 ˆ 1 ( 0 )= ( 0 ) ˆ 1 = (ˆ 0 ) ˆ (1 ˆ ) 3
which is not equivalent to the ML LR statistic. However, the GMM LR statistic is equivalent to the GMM Wald statistic. If you use ˆ = 0 (1 0 ) then the GMM LR statistics is equivalent to the the GMM LM statistic. Note: for GMM, = = if the same ˆ is used for all statistics. If ˆ = P 1 ( ˆ ) = P 1 ( ˆ ) is used then the GMM statistics willnotbeequaltothemlstatisticsinfinitesamples,butwillbeasthesamplesize gets large. Wald, LR and LM statistics based on maximum likelihood estimation Let 1 be an iid sample of Bernoulli random variables; that is, each has density (; 0 )= 0(1 0 ) 1 First, consider estimation of 0 using maximum likelihood (ML). Let =( 1 ) 1. Derive the log-likelihood function, ln ( x) score function, ( x) = ln ( x) Hessian function, ( x) = ln ( x) and information matrix, ( 0 x) = [( 0 x)] = (( 0 x)) Verify that [( 0 x)] = 0 and show that the ML estimate for is ˆ = P 1 The joint density/likelihood function for the sample is given by Y (x; ) =( x) = (1 ) 1 = (1 ) The log-likelihood function is ln ( x) = ln (1 )! = ln()+ ln(1 ) The score function for the Bernoulli log-likelihood is ln ( x) ( x) = = 1 1 1 P = ( ) (1 ) Using the chain rule it can be shown that ( x) = (1 ) ( x) = (1 ) µ + ( x) ( x) = (1 ) 4!
where = 1 P Therefore, the information matrix is ( 0 x) = [( 0 x)] = + [( 0 x)] 0 [( 0 x)] 0 (1 0 ) = 0 (1 0 ) since [( 0 x)] = P ([ ] 0 ) 0 (1 0 ) =0 since [ ]= 0 The MLE solves (ˆ x) = P ( ˆ ) ˆ (1 ˆ =0 ) ˆ = 1 =. Determine the asymptotic normal distribution for ˆ Using the asymptotic properties of the MLE for a regular model, ˆ ( 0 ( 0 x) 1 ) ( 0 x) 1 = 0(1 0 ) Since 0 is unknown, a practically useful result replaces ( 0 x) 1 with (ˆ x) 1 Next, consider testing the hypotheses 0 : = 0 vs. 1 : 6= 0 usinga5significance level. 3. Derive the Wald, LR and LM statistics for testing the above hypothesis. These statistics have the form = (ˆ 0 ) (ˆ x) = ( 0 x) ( 0 x) 1 = [ln ( 0 x) ln (ˆ x)] Are these statistics numerically equivalent? For a 5% test, what is the decision rule to reject 0 for each statistic? 5
The Wald statistic has the form For the LM statistic, note that = (ˆ 0 ) (ˆ x) = (ˆ 0 ) ˆ 1 ˆ = (ˆ 0 ) ˆ 1 ˆ so that P ( 0 x) = ( ) (1 ) ( 0 x) 1 = 0(1 0 ) = (ˆ 0 ) 0 (1 0 ) = ( 0 x) ( 0 x) 1 (ˆ 0 ) 0 (1 0 ) = 0 (1 0 ) = (ˆ 0 ) 0 (1 0 ) Notice that the LM statistic has the same form as the Wald statistic. The difference between the two statistics is based on how ( x) is estimated. With the Wald statistic, ( x) is estimated using (ˆ x) andwiththelmstatistic,( x) is estimated using ( 0 x) For the statistic, note that! ln ( x) = ln(ˆ )+ ln(1 ˆ ) Therefore ln ( 0 x) = ln( 0 )+! ln(1 0 ) = [ln ( 0 x) ln (ˆ x)] "! # = (ln 0 ln )+ ln(1 ) ln(1 ˆ ) µ µ 0 = ˆ ln +(1 ˆ 1 0 )ln ˆ 1 ˆ 6
4. ProvethattheWaldandLMstatisticshaveasymptotic (1) distributions. First, consider the Wald statistic: = (ˆ 0 ) ˆ 1 ˆ = (ˆ 0 ) rˆ 1 ˆ From the asymptotic normality of the MLE, we have that (ˆ 0 ) (0 0 (1 0 )) and from the consistency of the MLE we have that ˆ 0 Slutsky s theorem then gives rˆ 1 ˆ p 0 (1 0 ) Finally, by a combination of Slutsky s theorem and the continuous mapping theorem (CMT) we have (ˆ 0 ) (0 0 (1 0 )) rˆ 1 ˆ p 0 (1 0 ) = (0 1) = (1) where (1) denotes a chi-square random variable with 1 degree of freedom. A similar proof works for the LM statistic: = (ˆ 0 ) 0 (1 0 ) (ˆ 0 ) = p 0 (1 0 ) (0 0 (1 0 )) p = (1) 0 (1 0 ) 7