Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday 1 st March, 2019 (h17:43) P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 1 / 25 Summary Liear model with edogeous regressors The GMM estimator ad its special case (IV estimator) Asymptotic distributio of the GMM estimator Cosistet estimatio of error variace Cosistet estimatio of stadard errors Efficiecy ad the 2-steps efficiet GMM P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 2 / 25
Itro ad otatios Here we preset a framework where the model is specified i terms of momet coditios (MC). IV are aturally embodied i this framework. Now we cosider the case where we observe three types of variables y i : the usual respose/outcome/idepedet variable z i : a set of L regressors for y i. The primary goal is to kow their partial effect o the expected y (that is E[y i z i ]/ z i ). However, some of these variables may be edogeous. x i : a set of K exogeous variables. x i ad z i ca share some of the variables. x i usually cotais IVs ad exogeous regressors Remark: the iferece is based o asymptotic approximatios, hece max {K, L} < P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 3 / 25 Assumptios (C1): liearity Let z i R L be a vector of regressors, δ R L a vector of L coefficiets, {ε i } is a uobservable error term, ad y i =δ 1 + δ 2 z i2 + δ 3 z i3 +,..., +δ L z il + ε i for i = 1, 2,..., =z iδ + ε i z i1 = 1 for all i = 1, 2,..., if we have a costat term i the model. (C2): ergodic statioarity Let x i be a K-dimesioal vector of istrumets, let w i be the uique ad ocostat term of (y i, z i, x i). Assume {w i } is joitly statioary ad ergodic. P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 4 / 25
Deote: g i = g(w i, δ) = x i ε i = x i (y i z iδ) (C3): momet equatios, orthogoality coditios All the K elemets of x i are predetermied i the sese that E[x ik ε i ] = 0 for all i, k; or E[g i ] = 0. Exogeous variables that are also regressors are called predetermied regressors. The rest of regressors are called edogeous regressors (C3) gives a key tool: a system of K momet equatios with L ukows: (eq. 1) E[x i1 (y i z iδ)] = 0 (eq. 2) E[x i2 (y i z iδ)] = 0. (eq. K) E[x ik (y i z iδ)] = 0 Vector form: E[g(w i, δ)] = E[x i (y i z iδ)] = 0 P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 5 / 25 (C4): rak coditio The (K L) matrix E[x i z i] = Σ xz has rak equal to L (full colum rak). Propositio (idetificatio) Uder (C2), (C3) ad (C4), the parameter vector δ is uiquely idetified. Proof. Based o (C3) we kow that there exists a δ such that That is E[x i (y i z iδ)] = E[x i y i ] E[x i z i]δ = 0 E[x i z i]δ = E[x i y i ] (13.1) Based o (C2) we kow that these momets exists ad do ot deped o i. From liear algebra we kow that the there exists a uique δ satisfyig (13.1) if ad oly if rak(e[x i z i]) = L. Based o (C4) the ukow parameter δ is uiquely determied based o momets obtaied from the joit distributio of the observables. Therefore, δ is idetified. P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 6 / 25
We use the otatio σ xy = E[x i y i ], ad eq. (13.1) is rewritte as Σ xz δ = σ xy. Note that rak(σ xz ) mi {K, L}, therefore a ecessary (but ot sufficiet) coditio for idetifiability is that K L (order coditio). The Model is: uderidetified: order coditio is ot satisfied because K < L. It s hopeless to estimate δ because it s ot idetified. just idetified: rak coditio is satisfied with K = L. We ca estimate δ overidetified: rak coditio is satisfied with K > L We ca estimate δ P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 7 / 25 (C5): martigale depedece {g i } is a MDS. The (K K) matrix of cross momets, S = E[g i g i] is o sigular. (C5) allows to derive CLT-types approximatios. It also allows to ivert S A sufficiet coditio for (C5) is E[ε i ε i 1, ε i 2,..., ε 1, x i, x i 1,..., x 1 ] = 0 which implies that the error term is orthogoal to both past ad preset values of the exogeous variables x i. P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 8 / 25
Empirical aalog of (C3) Deote s xy = 1 x i y i ad S xz = 1 x i z i these are sample couterparts of σ xy, Σ xz Rewrite the sample versio of E[g i ] g (δ) = 1 g(w i ; δ) = 1 x i (y i z iδ) = 1 ( ) 1 x i y i x i z i δ =s xy S xz δ P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 9 / 25 Now the empirical versio of the system of momet equatios g (δ) = 1 x i (y i z iδ) = 0 is equivalet to (eq. 1) 1 x i1 (y i z iδ) = 0 (eq. 2) 1 x i2 (y i z iδ) = 0.. (eq. K) 1 x ik (y i z iδ) = 0 P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 10 / 25
K = L (model just idetified): IV estimator The typical case is that some of the regressors are edogeous, ad we have a IV for each edogeous variable. Recall: if a regressor is exogeous, it ca be istrumeted by itself. Suppose we have L regressors icludig the itercept. Assume that the last q regressors are edogeous z i = (1, z i,2, z i,3,..., z i,(l q), z i,(l q+1),..., z il ) A typical choice for the exogeous vector is: x i = (1, z i,2, z i,3,..., z i,(l q), x i1, x i2,..., x iq ) K = q istrumets + (L q) exogeous regressors, the x i R L. Data ca be arraged i two desig matrices P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 11 / 25 Z is the desig matrix with the L regressors where the first L K colums are samples of the exogeous regressors 1 z 1,2... z 1,L 1 z 2,2... z 2,L Z =...... 1 z,2... z,l X is the desig matrix of the K = L exogeous variables. The last q colums are the sample values of the IVs 1 z 1,2 z 1,(L q) x 1,1 x 1,2 x 1,q 1 z 2,2 z X = 2,(L q) x 2,1 x 2,2 x 2,q............ 1 z,2 z,(l q) x,1 x,2 x,q P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 12 / 25
If K = L by (C4) Σ xz is square ivertible. Moreover, by (C2) as S xz Σ xz, this meas that S xz is ivertible with probability 1 for sufficietly large. The for large we ca estimate δ by solvig the sample versio of the momet equatios. That is, we look for ˆδ IV such that g (ˆδ IV ) = 0: g (ˆδ IV ) = 0 = S xz ˆδIV = s xy = S 1 xz S xz ˆδIV = S 1 xz s xy ˆδ IV = S 1 xz s xy = ( 1 x i z i ) 1 ( 1 ˆδ IV is called istrumetal variable estimator ˆδ IV is a method of momets estimator (MM) If z i = x i for all i, the we go back to OLS ) x i y i P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 13 / 25 K > L (model overidetified): GMM Sice K > L we have more equatios tha ukow. There may be multiple solutios to g (δ) = 0. We may look for a δ that makes g (δ) as close as possible to 0. Maybe a δ that miimizes the orm g (δ) 2 However if the elemets of g ( ) are ot orthogoal, g ( ) 2 is ot really good at measurig the size of g (δ). I these cases we kow that we ca measure the size of g (δ) based o a quadratic form (of Mahalaobis type) with some appropriate symmetric ad PD matrix W : J(δ, W ) := g (δ) W g (δ) P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 14 / 25
GMM estimator Defiitio (GMM: geeralized method of momets estimator) Suppose W is a symmetric ad PD (K K) matrix. Assume that there exists Ŵ such that Ŵ p W. ˆδ(Ŵ ) is the GMM estimator if ˆδ(Ŵ ) := arg mi δ g (δ) Ŵ g (δ) Ŵ ca be costat or radom. The latter covers the case whe W is data-depedet Ŵ weights the equatios P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 15 / 25 g (δ) is the sample versio of the K momet equatios g (δ) = g (1) g (2) g (K) (δ) (δ). (δ) where g (k) (δ) = 1 x ik (y i z iδ) Here Ŵ = (ŵ k,j). It is easily see that Ŵ weights the samplig momet equatios ito the objective fuctio J(δ, Ŵ ) =g (δ) Ŵ g (δ) K K = ŵ k,j g (k) (δ) g (j) (δ) k=1 j=1 P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 16 / 25
I vector form J(δ, Ŵ ) = (s xy S xz δ) Ŵ (s xy S xz δ) J( ) is quadratic wrt ψ = (s xy S xz δ) ψ is liear i δ. FOC for the miimum of J(δ, Ŵ ) are S xzŵ s xy S xzŵ S xzδ = 0 Uder (C2) ad (C4), S xz is full colum rak with probability 1, for large Ŵ is PD with probability 1. Hece, S xzŵ S xz is osigular with probability 1. Pre-multiply both sides by (S xzŵ S xz) 1 ad we get the GMM estimator δ(ŵ ) = (S xzŵ S xz) 1 S xzŵ s xy P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 17 / 25 Properties of the GMM estimator For ay suitable choice of Ŵ Cosistecy: uder (C1)-(C4), δ(ŵ ) Normality: uder (C1)-(C5) where ( δ( Ŵ ) δ) p δ d N (0, AVar( δ(ŵ ))) AVar( δ(ŵ )) = (Σ xzw Σ xz ) 1 (Σ xzw SW Σ xz ) (Σ xzw Σ xz ) 1 Recall: Σ xz = E[x i z i], S = E[g i g i] = E[ε 2 i x i x i], Ŵ p W P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 18 / 25
By (C2) S xz = 1 x i z i as Σ xz Assume you ca fid ay estimator Ŝ p S, the by cotiuous mappig theorem we get a cosistet estimator for the asymptotic variace AVar( δ(ŵ )) = (S xzŵ S xz) 1 (S xzŵ ŜŴ S xz) (S xzŵ S xz) 1 How to fid Ŝ? Aswer: aalog priciple applied to the followig populatio momet E[g i g i] = E[ε 2 i x i x i] Problem: as usual oe caot observe ε i directly, we have to rely o its recostructio P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 19 / 25 Cosistet estimatio of the error variace Residual e i = y i z i δ(ŵ ) (C6): existece of momets of {z i } E[ z i z i ] < Uder (C1), (C2), ad (C6), it ca be show that 1 ei 2 p E[ε 2 i ] = Var[ε i ] The proof of this statemet is almost idetical to the oe for the case of large sample OLS. P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 20 / 25
(C7): existece of higher order momets E[(x ik z il ) 2 ] <, for all k = 1, 2,..., K ad l = 1, 2,..., L. (C7) implies that the joit distributio of regressors ad istrumets has a ice tail behavior Uder (C1), (C2), ad (C7): Ŝ = 1 ei 2 x i x i p S We ow have a cosistet estimator for the asymptotic variace. So we have everythig we eed to make iferece o δ Wait a miute... what about Ŵ? P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 21 / 25 Efficiet GMM Efficiecy of δ depeds crucially o Ŵ. We would like to have a Ŵ p W such that AVar( δ(ŵ )) is as small as possible i a large sample. Efficiecy lower boud: it ca be proved that (Σ xzs 1 Σ xz ) 1 AVar( δ(w )) for ay appropriate choices of Ŵ we achieve the lower boud with Ŵ = S 1 therefore the best choice is Ŵ = Ŝ 1 p S 1 P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 22 / 25
δ (Ŝ 1 ) is called efficiet GMM estimator. It s asymptotic variace is ow give by It is estimated by AVar ( δ( Ŝ 1 ) ) 1 ) = (Σ xzs 1 Σ xz AVar ( δ( Ŝ 1 ) ) 1 ) = (S xzŝ 1 S xz Ŝ depeds o e i which depeds o the estimated δ, which i turs depeds o Ŵ. P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 23 / 25 2-steps efficiet GMM Step 1: Step 2: take ay Ŵ 1 that coverges to a (K K) positive symmetric matrix. Ŵ 1 = I K, or Ŵ 1 = S 1 xx are commo choices. compute the cosistet δ(ŵ 1) compute e i = y i z i δ(ŵ 1), ad get the correspodig cosistet Ŝ1 set Ŵ 2 = Ŝ 1 1 compute the cosistet δ(ŵ 2) δ(ŵ 2) is the efficiet GMM. Hece, δ(ŵ 2) has the lower boud variace-covariace matrix. P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 24 / 25
Cocludig remarks Whe K = L the GMM is equivalet to the IV estimator. IV estimator is a particular case of the GMM estimator. Whe errors are homoscedastic, the efficiet GMM is equivalet to a classical estimator called 2SLS= 2-stages least square. 2SLS played a importat role i the begiig of IV literature. P. Coretto MEF Liear Model with Edogeous Regressors ad the GMM estimator 25 / 25