DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet choices of A. Next we will show that the efficiet correspods to A such that A A p Ω. Theorem 1 a A lower boud for the asymptotic variace of the class of estimators idexed by A is give by Q Ω Q. b The lower boud is achieved if A A p Ω. Proof. I order to prove part a we eed to show that Q Ω Q Q A AQ Q A AΩA AQ Q A AQ is egative semi-defiite for ay A that has rak l. Equivaletly we ca show that Q Ω Q Q A AQ Q A AΩA AQ Q A AQ 1 is positive semi-defiite. Sice the iverse of Ω exists Ω is positive defiite we ca write Ω = C C where C is ivertible as well. Write 1 as Q C CQ Q A AQ Q A AC C A AQ Q A AQ = Q C I C A AQ Q A AC C A AQ Q A AC CQ. Defie ad ote that usig this defiitio becomes H = C A AQ Q C I H H H H CQ. The above matrix is positive semi-defiite if I H H H H is positive semi-defiite. Next I H H H H I H H H H = I H H H H + H H H H H H H H = I H H H H. Therefore I H H H H is idempotet ad cosequetly positive semi-defiite. This completes the proof of part a. For part b if A A p A A = Ω the the asymptotic variace becomes Q Ω Q Q Ω ΩΩ Q Q Ω Q A atural choice for such A A is = Q Ω Q. Ω. This suggests the followig two-step procedure: 1
1. Set A A = I l. Obtai the correspodig iefficiet estimates of β say β. Usig the iefficiet but cosistet estimator of β obtai Ω. For example i the liear case Ω = Ûi Z i Z i where Û i = Y i X i β ad i the geeral case Ω = g W i β g W i β.. Obtai the efficiet estimates of β by miimizig where Ω comes from the first step. g W i b Ω A alterative to Ω i the first step is g W i β g W i β g g W i b W i β g W i β the cetered versio of Ω. The two versios are asymptotically equivalet sice E g W i β / b = 0. However the cetered versio ofte performs better. I the liear case a better choice for the first stage weight matrix is A A = Z i Z i 3 = Z Z. The reaso for this become clear i the ext sectio. The variace-covariae matrix of the efficiet estimator ca be estimated cosistetly by Q Ω Q where Q was defied i Lecture 11. Oe ca use Ω from the first stage or compute it agai usig the efficiet estimator to compute Ûi s i the liear case or g/ b i the geeral case. Two-stage Least Squares SLS Cosider the liear IV regressio model ad assume that E U i Z i = σ. 4 I this case Ω = E Ui Z i Z i = E E Ui Z i Zi Z i = σ E Z i Z i.
A atural estimator of E Z i Z i is which gives the optimal weight matrix as i 3. Note that i this case the efficiet estimator ca be obtaied without the first step sice the weight matrix i 3 does ot deped o Ûi s. The efficiet is give by SLS = X i Z i Z i Z i Z i X i X i Z i Z i Z i Z i Y i = Z i Z i X Z Z Z Z X X Z Z Z Z Y. We have that 1/ SLS β d N 0 σ EX i Z i EZ i Z i EZ i X i. The above estimator is also called the two stage LS estimator for the followig reaso. Defie X = Z Z Z Z X = P Z X the orthogoal projectio of the matrix of regressors X oto the space spaed by the istrumets Z. Sice P Z is idempotet we ca write SLS = X X X Y. Thus ca be obtaied usig the two-step procedure. First regress X agaist istrumets ad obtai the fitted values X. The first step removes from X i the correlatio with the error U i. I the secod step oe should ru the regressio of Y agaist the fitted values X. The SLS estimator is ot efficiet whe the coditioal homoskedasticity assumptio 4 fails. I this case the efficiet estimator is = X i Z i Ûi Z i Z i Z i X i X i Z i Ûi Z i Z i Z i Y i Exactly idetified case Whe the umber of istrumets is equal to the umber of regressors l = k ad the k k matrix Z X is of full rak the SLS estimator reduces to the IV estimator discussed i Lecture 10: SLS = X Z Z Z Z X X Z Z Z Z Y = Z X Z Z X Z X Z Z Z Z Y = Z X Z Y IV =. The IV estimator is a example liear of the exactly idetified case. I this case the weight matrix A plays o role. If the model is exactly idetified the we have k equatios i k ukows. Therefore it is possible to solve g W i b = 0 exactly. As a result the solutio to the miimizatio problem mi b B A g W i b 3
does ot deped o A. Sice i the exactly idetified case Q is k k ad ivertible the asymptotic variace-covariace matrix takes the followig form Q A AQ Q A AΩA AQ Q A AQ = Q A A Q Q A AΩA AQQ A A Q = Q Ω Q = Q Ω Q idepedet of A ad aturally efficiet. Cofidece itervals ad hypothesis testig i the framework I this sectio we discuss costructig of cofidece itervals ad hypothesis testig. Let be the efficiet estimator with the asymptotic variace-covariace matrix V = Q Ω Q. Let V deote a cosistet estimator of V. Sice is approximately ormal i large samples a cofidece iterval with the coverage probability 1 α for elemet j of β is give by [ [ ] ] j z 1 α/ V / / j + z 1 α/ [ V ] for j = 1... k. SLS For example i the liear ad homoskedastic case the asymptotic variace of is ad its cosistet estimator is V = σ = σ V = σ EX i Z i EZ i Z i EZ i X i X i Z i X Z Z Z Z X Z i Z i Z i X i where σ = Y i X i SLS. Therefore the 1 α asymptotic cofidece iterval for βj is give by [ ] j SLS ± z 1 α/ σ X Z Z Z Z X. Oe ca costruct a test of the ull hypothesis H 0 : β j = β 0j agaist H 1 : β j β 0j by usig the followig test statistic: T j = j β 0j [ ]. V / Sice uder the ull hypothesis T j d N 0 1 the asymptotic α-size test is give by Reject H 0 if T j > z 1 α/. Oe ca use a Wald statistic i order to test H 0 : β = β 0 agaist H 1 : β β 0 : W = β 0 V β 0. 4
More geerally suppose that the ull ad alterative are give by H 0 : h β = 0 ad H 1 : h β 0 where h : R k R q. By the delta method uder the ull 1/ h Therefore the Wald statistic is give by W = h d N 0 h β V h h β β V β h β. β h. The asymptotic α-size test is give by Testig overidetified restrictios Reject H 0 if W > χ q. I this sectio we discuss a specificatio test that allows oe to test whether the momet coditio Eg W i β = 0. Cotrary to the tests discussed before this is ot a test of whether β takes o some specific value but rather whether the model as defied by the momet coditios is correctly specified. The ull hypothesis is that there exists some β such that Eg W i β = 0. The alterative hypothesis is that Eg W i β 0 for all β R k. Note that whe the model is exactly idetified the system of k equatios i k ukows Eg W i b = 0 ca be solved exactly. Thus we ca test validity of momet restrictios oly if the model is overidetified. Whe the model is overidetified i geeral it is impossible to choose b such that g W i b is exactly zero. However if the momet coditio Eg W i β = 0 holds we should expect that g W i β is close to zero ad further / If we use the efficiet matrix A the I this case the weighted distace / g W i β d N 0 Eg W i β g W i β g W i β = N 0 Ω. A A p Ω. 5 A A / g W i β asymptotically has the χ l distributio the degrees of freedom are determied by the l momet restrictios. It turs out that whe β is replaced by its efficiet estimator the degrees of freedom chage from l to l k. We have the followig result. Uder the ull hypothesis H 0 : Eg W i β = 0 for some β R k ad provided that A satisfies 5 ad is efficiet / g W i A A / g W i d χ l k. The reaso for chage i degrees of freedom is that we have to estimate k parameters β before costructio the test statistic. Aother explaatio is that we eed k restrictios to estimate β. Thus we ca test oly additioal overidetified l k restrictios. 5
Cosider the liear ad homoskedastic case. The efficiet estimator is the SLS estimator ad the efficiet weight matrix is give by Z iz i. Oe should reject the ull of correctly specified model if = / > χ l k1 α Û i Z i Y i X i Z i Z i / Û i Z i/ σ Z i Z i Z i Y i X i Z i / σ where σ is ay cosistet estimator of σ = EUi such as Y i X i. Note that here we test joitly exogeeity of the istrumets ad other assumptios such as liearity of the model. 6