1 Covariance Estimation

Transcription

1 Eco 75 Lecture 5 Covariace Estimatio ad Optimal Weightig Matrices I this lecture, we cosider estimatio of the asymptotic covariace matrix B B of the extremum estimator b : Covariace Estimatio Lemma 4. ad Assumptios EE2(i) ad CF(iv)* combie to yield bb 2 = ^Q ( b )! p B : () Hece, it remais to d a cosistet estimator of. The geeral priciple employed is that of formig estimators by replacig expectatios with sample averages ad ukow parameters with cosistet estimators of them. The, Lemma 4. ca be used to establish cosistecy of the resultig estimator b : We cosider each of the examples i tur: () ML Estimator: Let bb = b = 2 log f(w i; b ) ad (2) log f(w i; b ) log f(w i; b ): We obtai b! p by verifyig coditios (i), (ii), ad (iii) of Lemma 4.. Coditio (i) holds by cosistecy of b. Coditios (ii) ad (iii) hold by the ULLN i Sectio 4 provided log f(w; ) (or, equivaletly, f(w; ) ad f(w; )) is cotiuous i o 8w 2 W (as is assumed i Lecture 4) ad E sup log f(w i; ) 2 where is a compact eighborhood of : 2 < ; If the model is correctly speci ed, the B = ad the covariace matrix B B ca be estimated by B b b B b, B b, or b. Note that b oly requires calculatio of the rst derivative of f(w; ), whereas B b requires calculatio of the secod derivatives. The otes for this lecture is largely adapted from the otes of Doald Adrews o the same topic I am grateful for Professor Adrews geerosity ad elegat expositio. All errors are mie. Xiaoxia Shi Page:

2 Eco 75 (2) LS Estimator: Let bb = g(x i; b ) g(x i; b ) (Y i g(x i ; b 2 )) g(x i; b ) ad b = (Y i g(x i ; b )) 2 g(x i; b ) g(x i; b ): (3) We obtai b! p by verifyig the three coditios of Lemma 4.. Coditio (i) holds by cosistecy of b. Coditios (ii) ad (iii) hold by ULLN provided g(x; ) ad g(x; ) are cotiuous i o 8x 2 X (as is assumed i Lecture 4) ad E sup (Y i g(x i ; )) g(x i; ) 2 where is a compact eighborhood of : 2 < ; If the regressio model is correctly speci ed (i.e., E(Y i jx i ) = g(x i ; ) a.s.), the B simpli es ad b B ca be simpli ed correspodigly. Let eb = g(x i; b ) g(x i; b ): (4) I the correctly speci ed case, e B! p B whe CF(iv) holds. So, a cosistet covariace matrix estimator for a correctly speci ed regressio model is eb b e B : (5) Note that this estimator allows for coditioal heteroskedasticity of the errors i.e., it is a heteroskedasticity cosistet covariace matrix estimate. If the model is correctly speci ed ad the errors are coditioally homoskedastic, the = 2 B ad b ca be replaced by the estimator b 2 e B, where b 2 = (Y i g(x i ; b ) 2 : (6) For a liear regressio model, b 2 e B equals b 2 X ix i : (3) GMM Estimators: Let bb = b = g(w i; b ) A A g(w i; b ) A A g(w i; b ) ad (7) g(w i; b )g(w i ; b ) A A g(w i; b ): Xiaoxia Shi Page: 2

3 Eco 75 Note that the de itio of b B does ot iclude the secod summad of 2 ^Q ( b ) i Equatio (33) i Lecture 4. The reaso is that the secod summad coverges i probability to zero sice Eg(W i ; ) = ad, hece, ca be omitted. Each compoet of b has bee show i Lecture 4 to coverge i probability to the correspodig compoet of. The oly exceptio is the compoet g(w i; b )g(w i ; b ). The latter coverges i probability to Eg(W i ; )g(w i ; ) by Lemma 2. ad Theorem.3 provided b! p, g(w; ) is cotiuous i o 8w 2 W ad where is a compact eighborhood of. (4) MD Estimators: Let E sup 2 jjg(w i ; )jj 2 < ; bb = b = g(b ) A A g(b ) ad (8) g(b ) A A V b A A g(b ); where V b is some cosistet estimator of V, the asymptotic covariace matrix of p (b ). Note that the de itio of B b does ot iclude the secod summad of 2 ^Q ( b ) i Equatio (37) i Lecture 4, because the latter coverges i probability to zero give that = g( ). Each compoet of b has bee show i Lecture 2 to coverge i probability to the correspodig compoet of, except b V. We simply assume b V! p V here, because the speci c form of V has ot bee stated. (5) TS Estimators: Let bb = G ( b ; b ) A A G ( b ; b ) ad (9) b = G ( b ; b ) A A ( V b + b V b 2 + V b 2 b + b V3 b b ) A A G ( b ; b ), where b = G ( b ; b ) ad b V j is some cosistet estimator of V j for j = ; 2; 3. If is zero, as occurs i some cases, such as feasible GLS estimatio, the oe ca take b = ad the estimators b V 2 ad b V 3 are ot required. Xiaoxia Shi Page: 3

4 Eco 75 2 Optimal Weight Matrices for GMM, MD, ad TS Estimators The GMM, MD, ad TS estimators have asymptotic covariace matrices of the form ( C ) C C ( C ) ; () where C = A A ad is a symmetric positive semi-de ite (psd) matrix that depeds o the estimator. We will show that the optimal choice of weight matrix A is a choice such that A A =, where A p! A: () This choice miimizes the asymptotic covariace matrix of b : Whe () holds, the asymptotic covariace matrix i () simpli es to ( ). We will show that ( C ) C C ( C ) ( ) ; (2) where deotes is psd. Note that F G if ad oly if G F. Thus, (2) holds if ad oly if The left-had side of (2) equals =2 = HP H = HP (HP ) ; C ( C C ) C : (3) h i I k =2 C ( C C ) C =2 =2 (4) where H = =2, P = I k =2 C ( C C ) C =2, ad the secod equality uses the fact that P is a projectio matrix (i.e., P is symmetric ad idempotet, P 2 = P ). A matrix of the form HP (HP ) is ecessarily psd, sice z HP (HP ) z = jjp H zjj 2 8z 2 R d : I sum, the optimal weight matrix for the GMM, MD, ad TS estimators depeds o the asymptotic covariace matrix of p ^Q ( ), which is = estimators, = V ad the optimal weight matrix A is such that C C. For the GMM ad MD A A p! A A = V : (5) Xiaoxia Shi Page: 4

5 Eco 75 For the TS estimator, the optimal weight matrix A is such that A A p! A A = (V + V 2 + V 2 + V 3 ) : (6) Two-Step GMM It is usually desirable to use the optimal weight matrix rather tha a arbitrary weight matrix whe doig GMM ad MD estimatios. However, the optimal weight matrices deped o the covariace of the momet fuctios i the GMM case ad the asymptotic variace of the iitial estimator i the MD case. Either the covariace or the asymptotic variace is ot kow. Therefore, we eed to obtai cosistet estimators for them. I the case of MD, the asymptotic variace of the iitial estimator ^ ca be estimated from the iitial procedure used to obtai ^. Deote the estimator by V. I the case of GMM, the covariace = E (g (W i ; ) g (W i ; )) ca be cosistetly estimated by ^ = P g W i ; ^ ; g W i ; ^ ; for some cosistet estimator ^. The cosistet estimator ^ may be obtaied usig GMM with the idetity matrix as the weight matrix. After estimatig V ad ^, we ca use A = sqrtm(v ) ad A = sqrtm(^ ) as the estimated optimal weight matrix to carry out GMM ad MD estimatio, respectively. The GMM/MD estimators obtaied have the same asymptotic variace as GMM/MD estimators usig the (ifeasible) optimal weight matrices. The reaso simply is that our estimators for the optimal weight matrices are cosistet. The GMM estimators obtaied usig the above procedure are called two-step GMM estimators because i this procedure, GMM estimatio is carried out twice. Multi-step GMM: Now that the two-step GMM estimators are "better" (i a secod order asymptotic sese) tha a oe-step GMM estimator with idetity weight matrix (or ay other arbitrary weight matrix that is ot the optimal weight matrix). Suppose that we estimate agai usig the two-step GMM estimators. The estimated ca be reasoably believed to be better tha the ^. You may woder if we should ru GMM agai usig the better covariace matrix estimator, ad obtai a 3-step GMM estimator. You may also wat to keep the iteratio goig. (Iterative GMM) The multi-step GMM estimators do ot improve upo the two-step versio i rst order (cosistecy) or secod order (asymptotic variace) asymptotics. Some argues that ite sample property (like mea-bias or media-bias) might be better for iterated GMM. Cotiuous Updatig GMM (CUE): ulike the iterative or two-step GMM, CUE does ot take the weight matrix as give. Istead, it treats the weight matrix as a fuctio of, ad miimizes: ^Q () = g () ^ () g (), (7) Xiaoxia Shi Page: 5

6 Eco 75 where ^ () = g (W i; ) g (W i ; ). (8) The CUE is cosistet ad has the same asymptotic variace as the two-step or the iterative GMM estimators. All three procedures are used i practice ad oe domiates the others. See Hase, Heato ad Yaro (994) for a Mote Carlo experimets that compare the three i ace applicatios. (Homework questio: d a GMM problem i a cross-sectio settig ad compare the three procedures by simulatio) Xiaoxia Shi Page: 6