Endogenous Regressors and Instrumental Variables. James L. Powell Department of Economics University of California, Berkeley

Transcription

1 Endogenous Regressors and Insrumenal Variables James L. Powell Deparmen of Economics Universiy of California, Berkeley Endogenous Regressors and Inconsisency of LS he endogenous regressor linear model, a workhorse of economeric applicaions, assumes ha he dependen variable and regressors are boh random and saisfy he linear relaion y = β + ε, bu he usual assumpion of zero condiional mean of he error erms given he regressors is no saisfied, i.e, E( 0 ε) 6= 0= E(ε ) 6= 0. his is a more serious deparure from he assumpions of he classical linear model han was he case for he Generalized Regression model, which mainained E(ε ) =0bu permied nonconsan variances and/or nonzero correlaions across error erms; unlike he Generalized Regression model, he classical leas squares esimaor will be inconsisen for β if he errors are correlaed wih he regressors. Wriing ˆβ LS =( 0 ) 0 y = β +( 0 ) 0 ε, applicaion of an appropriae version of he Law of Large Numbers will imply ha plim 0 plim x x = plim E(x x 0 ) M xx, which is rouinely assumed o be inverible for he classical regression models, and plim 0 ε = plim x ε = plim E(x ε ) M xε. Since E(x ε ) 6= 0generally implies M xε 6=0(a leas for saionary daa), i follows ha plim ˆβ LS = β + plim ( 0 ) ( 0 ε) = β + M xx M xε β 6= β.

2 Of course, he inconsisency of ˆβ LS for β is only a problem if you wan o esimae he primiive parameer β raher han β, which is he vecor of bes linear predicor coefficiens for y given x.several examples below will ouline cases in which β is a naural srucural parameer for behavior even hough i canno be inerpreed as he vecor of bes linear projecion coefficiens. An alernaive inerpreaion of he inconsisency of classical leas squares lies in he fac ha i solves a sample momen condiion of he form 0= 0 (y ˆβ LS )= x (y x 0 ˆβ LS ) (which are called he normal equaions), bu he analogous condiion for he populaion daa disribuion is no saisfied, i.e., E(x (y x 0 β)) = E(x ε ) 6= 0. hus, leas squares solves he wrong momen condiion in he sample. he classical soluion o his problem of endogenous regressors supposes ha here is some L-dimensional vecor of insrumenal variables, denoed z below, which is observable and saisfies E(z ε ) M zε =0 for all values of. hus, hough he regressors x are correlaed wih he error erms ε,heinsrumenal variables (or insrumens ) z are no. If only some of he componens of x are correlaed wih he errors, he remaining componens can be included in he insrumen vecor z ; depending on he paricular applicaion, oher ransformaions of observable random variables may be suiable insrumens. In addiion o being uncorrelaed wih he error erms, i will be necessary ha he insrumenal variables z are fully correlaed wih he regressors x :hais,if E[z x 0 ] M zx is he (L K) marix of produc-momens of z wih x, we will need his marix o be of full column rank, i.e. rank(m zx )= K. his rank condiion, whichwillbeasufficien condiion for idenificaion of he parameer vecor β from he observable daa, implies a weaker order condiion which is a necessary condiion for idenificaion 2

3 namely, ha L K, i.e., he number of insrumenal variables mus be a leas as large as he number of regressors. Models of Endogenous Regressors Before urning o he deails of esimaion of he parameer β using observaions on y,x,andhe insrumenal variables z, i may be informaive o consider some examples of linear models wih correlaion beween he regressors and error erms, and he ways ha insrumenal variables migh be derived. While no general mehod for cooking up insrumens is available, many models have naural insrumens associaed wih hem. () Auocorrelaed Errors and Lagged Dependen Variables: Here he linear model of ineres is y = x 0 β + γy + ε, where he error erms are assumed o be firs-order auoregressive, ε = ρε + u, wih u assumed o be i.i.d. wih zero mean and variance σ 2 u, and are independen of {x s } for all s. Since E(y ε ) = E(y (ρε + u )) = ρe(y ε ) = ρe((α + βx + γ 2 + ε )ε ) = ργe(y ε )+ρe(ε ), hen assuming {x } are saionary implying he {y } are saionary as well i follows ha E(y ε )=E(y 2 ε ), so ha E(y ε ) = ργe(y ε )+ρe(ε 2 ) = ρe(ε 2 )/( ργ) = ρσ 2 u/( ρ 2 )( ργ), 3

4 which is differen from zero when ρ 6= 0. hus, he regressor y is correlaed wih he error erm ε. For insrumens, we can use he curren and lagged values of he regressors; for example, we can use z =(x 0,x 0 )0 as insrumens for he se of regressors (x 0,y ) 0. Since he lagged values of he regressors, x, show up in he equaion for y, i is easy o see ha he IVs should be correlaed wih he righ-hand side variables, and hey are uncorrelaed wih he error erms ε by assumpion. Of course, we could also use more lagged values, i.e., z =(x,x,x 2,...), bu ypically he exra lagged values of x aren as highly correlaed wih y, and so migh be less useful as IVs. (2) Omied Variables: In a long regression seing wih for cross-secion daa, y i = x 0 iβ + w 0 iγ + u i x 0 iβ + ε i, wih ε i wi 0γ + u i,wemayonlyobservey i and x i, and only care abou esimaing β, bu if x i and he missing regressors w i are correlaed wih x i in he populaion, x i and ε i will be correlaed in he shor regression of y i on x i. Ofen, i is hard o hink of a convincing example where variables ha are no included in he observable x i (like geographic dummy variables) would be correlaed wih x i bu uncorrelaed wih w i,soheinsrumenal variable sraegy may problemaic in his example. Occasionally, hough, a naural experimen will provide a variable z i which affecs x i direcly, bu clearly is independen of w i. A well-known example in economerics is J. Angris s sudy of he effec of miliary service (a regressor in x i ha is possibly correlaed wih an unobserved abiliy variable w i ) on fuure earnings y i. As an insrumenal variable, Angris used an indicaor variable for wheher individual i had a high or low draf loery number during he Vienam war years; his would clearly be correlaed wih miliary service bu should be independen of individual unobserved abiliy. An earlier biosaisics sudy used he same insrumenal variable o esimae he effec of miliary service on life expecancy. hese and similar papers led o a paradigm shif oward naural experimen approaches o esimaion of causal effecs using microeconomic daa over he pas wo decades. (3) Measuremen Error: M. Friedman s classic model for he permanen income hypohesis posis an individual consumpion funcion as y i = α + βp i + u i, 4

5 where y i is measured consumpion and p i is permanen income for individual i. As Friedman noed, we don see permanen income, bu only measured income, which is assumed o be of he form x i = p i + v i, where v i is ransiory income (jus like u i is ransiory consumpion ). here are wo sors of assumpions on he observable daa ha yield insrumenal variables here. In one approach, he repeaed measuremen approach, i is assumed ha some oher variable relaed o permanen income, such as financial wealh w i, is observed. Supposing ha wealh is linearly relaed o permanen income, w i = γ + δp i + η i, where he ransiory componens (u i,v i,η i ) are independen of p i, and furher supposing ha E(u i ) = E(v i )=E(η i )=0, E(u i η i ) = E(v i η i )=0 (i.e., he shock o wealh is uncorrelaed wih ransiory consumpion or income), hen y = α + βx + ε, wih ε i = u i + βv i, so ha E(x i ε i ) 6= 0bu E(w i ε i )=E((γ + δp i + η i )(u i + βv i )) = 0 and E(w i x i )=E((γ + δp i + η i )(p i + v i )) = δe(p 2 i ) 6= 0, provided δ 6= 0(i.e., w i is really relaed o p i ). So z i =(,w i ) 0 can be used as a vecor of insrumenal variables for (,x i ) 0. Adifferen soluion, proposed by A. Zellner, assumes a causal model relaing permanen income o observable variables w i (including, say, financial wealh, educaion, work experience, ec.). Wriing his model as p i = wiδ 0 + η i, 5

6 where (u i,v i,ε i ) is assumed independen of w i wih E(u i )=E(v i )=E(ε i )=0, ec. hen w i is clearly correlaed wih p i if δ 6= 0, E(w i x i )=E(w i (w 0 iδ + v i + η i )) = E(w i w 0 i)δ 6= 0, bu E(w i ε i )=E(w i (u i + βv i )) = 0, so we can use z i (,wi 0)0 as insrumens for (,x i ) 0. hough he repeaed measuremen and causal model approaches are quie differen in he former, E(p i η i )=0,E(w i η i ) 6= 0, and vice versa in he laer we ge similar insrumens in eiher case. (3 ) Raional Expecaions Models (Measuremen Error): A variaion on he previous measuremen error model migh be y = α + βe (x + )+u, wih u i.i.d., independen of {x s }, ec. For example, y migh be curren invesmen, and x migh be curren sales, so curren invesmen would respond o curren expecaions of fuure sales. Assuming we observe only (y,x ),where we can wrie x + = E (x + )+v +, E(v + x,y,x,y,...)=0, y = α + βx + + ε, ε u + βv +, and can use pas values of y and curren and pas values of x as insrumenal variables in his regression, i.e., z =(,x,y,x,y 2,...). (4) Keynesian Cross: In his shopworn example, discussed by virually all inroducory economerics exs, an aggregae consumpion funcion c = α + βy + ε 6

7 is paired wih an income ideniy y = c + i, where c,y,andi are aggregae consumpion, income, and auonomous invesmen. Assuming E[ε ]=0=E(ε i s ) (so ha invesmen is deermined by animal spiris, and does no respond o consumpion or income), we can ake z (,i ) 0 as IV s for (,y ) 0 in esimaing he consumpion funcion. his is possibly he simples example of a simulaneous equaions model, in which he variables c and y appearing in one equaion (he consumpion funcion) are joinly deermined as he soluion of a sysem of equaions, raher han he lef-hand variable (here, c ) being deermined by he independen causal effecs of he righ-hand-side regressors and errors. (4 ) Supply and Demand Model: In anoher simple canonical model of simulaneiy, quaniy q and price p are deermined o equae supply and demand in a paricular marke. Wriing he demand funcion as q = α + βp + γy + u, i is assumed ha some oher variable y besides price (say, aggregae income) shifs he demand funcion, bu does no affec supply. Similarly, he inverse supply funcion is assumed o be p = δ + φq + ψw + v, where some supply shifer variable w (e.g., weaher ) is included in he inverse supply funcion bu is excluded from he demand equaion. Assuming (y,w ) are exogenous, i.e., y u E y v =0, w u w v hen z =(,y,w ) 0 will be uncorrelaed wih (u,v ), assuming E(u )=E(v )=0;ifψ 6= 0,E(w p ) 6= 0, and if γ 6= 0,E(q y ) 6= 0,sowecanusez =(,y,w ) 0 as insrumenal variables for x d (,p,y ) 0 in he demand equaion and for x s (,q,w ) 0 in he inverse supply equaion. Jus-Idenificaion and Insrumenal Variables Esimaion he foregoing examples illusrae how L-dimensional insrumenal variables z saisfying he condiions E[z ε ]=M zε =0and rank(e[z x 0 ]) = rank(m zx )=K can be obained in cerain applicaions. Now, 7

8 assuming such variables exis, we urn o he quesion of wha o do wih hem, i.e., how o use hem o ge consisen esimaors of β. In he special case where L = K (known as he jus idenified case, since he order condiion for idenificaion is barely saisfied), we can define he insrumenal variables (IV) esimaor of β for he linear model as ˆβ IV (Z 0 ) Z 0 y = " z x# 0 " # z y, which is generally well-defined because Z 0 is a square (K K) marix. his is an obvious generalizaion of he classical leas squares esimaor ˆβ LS,wih Z 0 replacing 0 in ha formula hroughou. I is easy o show consisency of his esimaor if he condiions M zε =0and rank(m zx )=K are saisfied; again wriing he IV esimaor ˆβ in erms of he rue parameer and error erms, ˆβ IV = β +( Z0 ) ( Z0 ε), a law of large numbers and Slusky s heorem will imply ha} plim µ Z0 plim à z x 0! = M zx and so plim Z0 ε = plim z ε =lim E(z ε ) M zε =0, ˆβ IV p β +(M zx ) M zε = β, as required for consisency. his insrumenal variables esimaor can be inerpreed as a mehod of momens esimaor, because ˆβ IV solves a varian of he normal equaions, he IV esimaing equaions 0= Z0 (y ˆβ IV )= z (y x 0 ˆβ IV ), which correspond o he correc populaion momen condiions E[z (y x 0 β)] = E(z ε ) M zε =0. In effec, he IV esimaor ses he sample covariance of he insrumenal variables and residuals equal o zero (assuming a consan erm is included in he se of insrumens). 8

9 Assuming some cenral limi heorem can be invoked o show ha Z 0 ε z ε d N (0,V 0 ) for some marix V 0 (wih V 0 = E[ε 2 z z 0 ] if he original observaions on y and x are i.i.d.), hen i is easy o show ha he IV esimaor is approximaely normally disribued, (ˆβ β0 )=( Z0 ) Z 0 ε d N (0,M zx V 0 M xz ), where M xz Mzx 0 = plim x z 0 plim ˆM xz. A consisen esimaor of M zx is, of course, ˆMzx ; wih a consisen esimaor ˆV of V 0 using eiher he Eicker-Whie (for serially uncorrelaed daa) or Newey-Wes (for serially correlaed daa) mehods applied o (y x 0 ˆβ IV )z, large-sample confidence regions and hypohesis ess can be consruced using normal sampling heory. Overidenificaion and wo-sage Leas Squares (2SLS) If here are more insrumenal variables han regressors, i.e., L>K (called he overidenified case), his mehod-of-momens approach mus be generalized, since he IV esimaing equaions would require soluion of an overdeermined sysem of linear equaions (i.e., more equaions han unknown ˆβ componens) which will no exis excep in rare (probabiliy zero) cases. Of course, we could always hrow ou some of he exra insrumens o make L = K. A more general sraegy is o premuliply he L-vecor z by some (K L) marix ˆΠ 0 which could, in general, be esimaed (random) hen use he K-vecor ˆΠ 0 z as insrumens. A special case would be ˆΠ 0 =(I K, 0), which would delee he las L K componens of z ;in general, we will require ha he square marix ˆΠ 0 Z 0 will have full rank K, which implies ha he rank of ˆΠ mus be K (wih probabiliy one). Since he dimension of Z ˆΠ ishesameasfor, namely,n K, we can jus subsiue Z ˆΠ ino he IV esimaor formula o define a generalized insrumenal variable or generalizedmehodofmomensesimaor of β: ˆβ GIV = ˆβ GIV (ˆΠ) =(ˆΠ 0 Z 0 ) (ˆΠ 0 Z 0 y), whose asympoic disribuion will depend on he paricular sequence of ˆΠ marices used, excep in he jus-idenified case L = K, where his formula reduces o he previous IV formula (because he K K 9

10 marix ˆΠ mus hen be inverible by assumpion). Assuming ˆΠ p Π for some fixed, full-rank (L K) marix Π, and assuming ha Z0 p M zx and Z 0 ε d N (0,V 0 ) as before, hen i is easy o verify ha he asympoic disribuion of he GIV esimaor is (ˆβ β) d N (0, [Π 0 M zx ] (Π 0 V 0 Π)[M xz Π] ), which depends on Π. As before, we can consisenly esimae his asympoic covariance marix using eiher he Eicker-Whie or Newey-Wes approaches, depending on wheher he daa are serially- correlaed. he radiional choice of he combinaion marix ˆΠ is ˆΠ =(Z 0 Z) Z 0, so ha ˆΠ is he (L K) marix of leas-squares coefficiens for he regression of he columns of on he marix Z, which should cerainly yield a marix Z ˆΠ of fied values of his regression which will be correlaed wih. he resuling esimaor of β is called he wo-sage leas squares (2SLS) esimaor, and has he algebraic form ˆβ 2SLS =( 0 Z(Z 0 Z) Z 0 ) ( 0 Z(Z 0 Z) Z 0 y). =(ˆ 0 ) ˆ0 y, where ˆ = Z ˆΠ is he marix of prediced values of from he regression of on Z. In his procedure, we firs fi a reduced form regression for x in he firs sage, x = Π 0 z + v, and use he fied values as IVs in he second-sage regression. By a law of large numbers, we expec ha ˆΠ p Π 0 {E(z z 0 )} E(z x ) M zz M zx ; subsiuing his ino he previous formula for he asympoic disribuion of he GIV esimaor gives a mighy unwieldy general formula for he asympoic disribuion of he 2SLS esimaor, (ˆβ β) d N (0, [M xz M zz M zx ] (M xz M zz V 0 M zz M zx )[M xz M zz M zx ] ). 0

11 If he error erms ε are assumed o be independen of z, as was radiionally assumed, his expression simplifies quie a bi; hen so ha V o = E[ε 2 z z 0 ]=E[ε 2 ] E[z z 0 ]=σ 2 M zz, (ˆβ β) d N (0,σ 2 [M xz M zz M zx ] ) when he errors are independen of he insrumens. he consan variance σ 2 could be esimaed by he sample average of he squared values of he residuals e = y x 0 ˆβ 2SLS, as for he classical regression model, and plim ˆ 0 ˆ = Mxz M zz M zx. Because Z(Z 0 Z) Z 0 is an idempoen (projecion) marix, here are several oher algebraically equivalen ways of wriing he 2SLS esimaor. In heil s inerpreaion of he 2SLS esimaor, ˆβ 2SLS =(ˆ 0 ˆ) ˆ0 y =[( 0 Z(Z 0 Z) Z 0 )(Z(Z 0 Z) Z 0 )] ˆ0 y, so β is esimaed by geing he fied values of on Z, hen regressing y on ˆ using he classical leas squares formula. Anoher inerpreaion, suggesed by Basmann, uses he fac ha ˆβ 2SLS =(ˆ 0 ˆ) ˆ0ŷ =( 0 P zz P zz ) 0 P zz P zz y =( 0 P zz ) 0 P zz y =(ˆ 0 ) ˆ 0 y, for P zz Z(Z 0 Z) Z 0 and ŷ = P zz y he vecor of fied values of he leas-squares regression of y on Z. Oher inerpreaions of 2SLS, some o be found on homework problems, exis. Generalized Mehod of Momens (GMM) In he general case where he errors are heeroskedasic and/or serially correlaed, so ha V 0 6= σ 2 M zz, he 2SLS esimaor will no have he smalles asympoic covariance marix. o obain an efficien esimaor, we wan o choose he combinaion marix Π = plim ˆΠ o minimize he asympoic covariance marix AV (ˆβ GIV (Π)) = [Π 0 M zx ] Π 0 V 0 Π[M xz Π] of he generalized IV esimaor (in he marix sense). he soluion can be found by ransforming he model o an asympoic version of he Generalized Regression model, and hen finding he appropriae GLS

12 esimaor for he ransformed model. Firs, premuliply he linear model y = β + ε by he normalized marix (/ )Z 0,soha ỹ Z 0 y ³ = Γ Z 0 ³ β + Z 0 ε β + ε, which defines a new linear model relaing he L-dimensional vecor ỹ o he (L K) marix of ransformed regressors. By he Cenral Limi heorem and he assumpion M zε =0=E[ ε], we know ha ε N (0,V 0 ), and we can also show ha he asympoic covariance of and ε is zero, so for large he ransformed variables ỹ and obey he Generalized Regression model (wih approximaely normal errors, no less!). he appropriae (infeasible) GLS esimaor of β for his model is = ³ 0 ZV 0 β GLS ( 0 V 0 Z 0 ) 0 V 0 ỹ ³ 0 ZV 0 Z 0 y =[ 0 ZV 0 Z 0 ] 0 ZV 0 Z 0 y ˆβ GMM, which is called he opimal generalized mehod of momens esimaor. his is in he form of a generalized IV esimaor, wih combinaion marix of he form. ˆΠ = V 0 ( Z0 ). o consruc a feasible version of his esimaor, we would need a consisen esimaor of V 0 = AsyV ar( ε) µ = AsyV ar Z 0 ε, 2

13 which we could obain by applying he Eicker-Whie or Newey-Wes procedures o he residuals from a 2SLS fi ofy on using Z as insrumens. For eiher he infeasible or feasible versions of his GMM esimaor, he asympoic disribuion will be normal, and of he form (ˆβGMM β) d N (0, [M xz V 0 M zx ] ), which reduces o he asympoic disribuion of 2SLS in he special case ha ε is independen of Z. In general, hough, we can show ha he asympoic covariance marix of 2SLS will be a leas as large (in he marix sense) as ha for GMM using he same argumens ha show ha he asympoic variance of LS exceeds ha of GLS for he Generalized Regression model. Many variaions on 2SLS and GMM exis. For example, we may have a sysem of equaions y j = j β j + ε j, j =,...,J, wih conemporaneous correlaion across he componens of ε j over j. Acombinaionof2SLSandZellner s Seemingly Unrelaed Equaions (SUR) esimaion mehod yield somehing known as hree-sage leas squares (3SLS), which is efficien if he error erms are normal and independen of Z; oherwise, he GMM and SUR approaches can be combined o ge more efficien join esimaors of all he unknown coefficien vecors {β j }. he IV and GMM approaches are also naurally applicable o nonlinear sysems of equaions, bu he esimaors do no have nice closed-form expressions for such models, and he derivaion of heir asympoic properies is much more complicaed. 3