Omitted Variables, Instrumental Variables (IV), and Two-Stage Least Squares (TSLS)

Size: px

Start display at page:

Download "Omitted Variables, Instrumental Variables (IV), and Two-Stage Least Squares (TSLS)"

Wilfrid Cummings
5 years ago
Views:

1 Omtted Varables, Istrumetal Varables (IV, ad Two-Stage Least Squares (TSLS Greee Ch.8,, Keedy Ch. 9 R scrpt mod4sa, mod4sb, mod4sc Assumpto 3 of the CLRM stpulates that the eplaatory varables are ucorrelated wth the error term. I may real-world applcatos ths assumpto wll ot hold. Eamples clude: Omtted varable (a varable that affects y AND s correlated wth oe or more regressors s omtted from the model. Measuremet error o oe or more regressors Lagged depedet varables used as regressors ( autoregresso Smultaeous equatos Models wth sample selecto All these codtos wll result the same ecoometrc problem: basedess ad cosstecy of the OLS (or MLE estmator. For ths chapter we wll focus o least squares (LS estmato ad the remedal methods of IV ad TSLS that have bee devsed for the LS settg. Orthogoalty ad Covarace I the followg, we wll make ample use of the cocepts of "orthogoalty", sample covarace, ad sample varace. Let's take a closer look at these terms. Cosder two vectors ad z. Orthogoalty mples that z z = z = 0 ( = The sample varace of s gve as var ( = s = ( ( = ( = = = = = = ( ( ( The last epresso wll be useful below. The sample covarace betwee ad z ca be wrtte as:

2 cov ( z, = sz = ( ( ( z z = ( z z = = = z z = = = ( ( (3 Aga, the last epresso wll be useful later. It follows that orthogoalty aloe does NOT mply a zero covarace. For a zero covarace we eed ether z = z (a somewhat ulkely codto, or ether of the sums the last = = = term s zero addto to orthogoalty. By the same toke, strctly speakg a zero sample covarace does ot mply orthogoalty. However, orthogoalty lkely holds that case, otherwse we would eed aga z = z for the covarace to be zero. = = = We should also ote that the sample varace coverges to the populato varace (call t σ as goes to fty. We ca show ths va covergece mea square error: ( ( ( ( ( ( ( ( ( σ µ σ µ σ σ ( ( ( ( E s ( ( ( = E E = E E = = V + E V E = ( ( + = = (4 Thus lm E( s = σ. Smlarly, t ca be show that ( s lm var = 0. Thus plm s = σ. See also Greee p By aalogy, the sample covarace approaches the populato covarace (call t σ uder creasg sample sze. z I the followg, whe we eame the covarace (or correlato betwee cluded regressors ad the compoets of the error term we usually thk of ths as the populato covarace. However, some cocepts are more clearly llustrated usg the sample covarace. Ether oe s sutable llustratg the O.V. problem. The Omtted varable problem Cosder the stpulated model y = Xβ + ε ad the true model y = Xβ + Xβ + ν. Asssume for ow E X ν = E X ν = 0. The frst model, estmated va OLS, produces that ( (

3 3 ( ( ( b = XX Xy= XX X Xβ + Xβ + ν = ( ( + + β XX XXβ XX Xν wth ( = + = (.. E b X β P β where P XX XX (5 The last term (5 wll oly be zero f the two X-matrces are perfectly orthogoal or f β = 0,.e. the omtted varables have o effect o y (whch, of course, we ruled out upfrot whe we wrote dow the "true" model. Alteratvely you ca thk of each colum P. as a set of LS coeffcets from a regresso of the correspodg colum of X o the etre X matr. I other word, the stpulated or "flawed model" wll lead to ubased (ad cosstet estmates oly f the cluded regressors ( X are orthogoal (ad thus, essece, ucorrelated wth the ecluded varables ( X ad thus wth the error term ε= Xβ + ν. Here s a closer look at ths stuato usg a smple stpulated model wth a costat term ad a sgle regressor, ad a sgle omtted varable. Thus, we have y = β + β + ε ε= γz+ ν or [ ] y = Xβ+ ε X= (6 Ths mples ( = = z = = = = = PX.Z = XX Xz = = z = = = = = = z z = = = = ( = = = = = s z z z z s z = = = s = = (7 where we use the results from ( ad (3 the last trasformato. Ths shows some addtoal detals for the omtted varable problem. If the sample covarace betwee ad z s zero, oly the tercept wll 3

4 4 be based. If ad z are orthogoal (or have zero covarace ad = z = 0, ether term wll be based. Thus, the omtted varable problem ca be qute subtle eve for ths smple case, assumg γ 0. I a multvarate regresso model, O.V. bas wll geerally affect ALL coeffcets ukow drecto, eve f there s oly oe O.V. correlated wth oly oe regressor! Ths s due to the usual correlato betwee cluded regressors (.e. ( XX wll usually be a full k by k matr wth ozero elemets. Returg to the geeral case, we ca alteratvely epress ths problem as a volato of Assumpto 3 of the CLRM,.e. E ( X ε = γ 0 (8 As a result, the OLS estmator s based ad cosstet (.e. asymptotcally based,.e. E ( b X = β+ ( XX γ ( ( plm b= β+ plm XX plm Xε = β+ Q - XXγ (9 R scrpt mod4sa provdes a emprcal llustrato of the O.V. problem usg the hedoc property value data from PS3, Q3. The Istrumetal Varables (IV Estmator ad TSLS Assume there are k colums of ( by k matr X that are potetally correlated wth the error term. We wll call these varables troublemakers. Assume the remag k colums are clea wth respect to O.V. problems (although ot ecessarly ucorrelated wth the remag colums X. Assume that for each troublemaker colum cj we ca fd oe or more varables that are ot the orgal models, ad that are hghly correlated wth c j, but ot wth ε. Such varables are called strumets. Istrumets are crutches that ultmately allow us to derve the effect of the orgal X o y wthout currg O.V. bas. Thus, clea, or "vald" strumets are ot correlated wth the outcome varable y other tha through ther correlato wth c j. Let matr Z collect all clea colums of the orgal X, plus the strumets. Assume Z has dmesos by l, wth l k. Net, cosder the followg -step procedure: Frst, the troublemakers X eed to be purged of ther O.V.-causg effects. Secod, the purged X s used the orgally desred regresso. The frst step s accomplshed by regressg each colum X (troublemaker or ot agast Z ad compute ftted values. Ths wll leave the clea colums of X uchaged, but leaves the troublemakers wth oly the effects that ca also be eplaed by Z. Formally: 4

5 5 ( - Xˆ = P X= Z ZZ ZX (0 Z I the secod step, ˆX s used leu of X the orgal regresso, ad OLS s appled. Ths leads to the Two-Stage-Least-Squares (TSLS Estmator ( ˆ ˆ ˆ ( ( ( b = XX Xy = XZ ZZ ZX XZ ZZ Zy ( TSLS Ths epresso ca be smplfed the specal case where l = k,.e. each troublemaker s replaced by eactly oe correspodg strumet. I that case, ZXs a square matr wth a (presumably wellbehaved verse, ad we ca wrte ( ( ( ( ( b = ZX ZZ XZ XZ ZZ Zy = ZX Zy = b ( TSLS I ths specal case the TSLS estmator s usually called IV estmator or b IV. However, the two terms are ofte used terchageably. Just remember that the epresso ( always holds, whle ( oly holds f l = k. As show Greee, Ch. 8, b IV (used the geeral sese of b TSLS s cosstet ad asymptotcally ormally dstrbuted. It s asymptotc varace ca be estmated by ( εε ˆˆ ( ( ˆ ˆ ( Vˆ a btsls = ˆ σ XX = ˆ σ XZ ZZ ZX where ˆ σ = ad εˆ = y Xb (3 TSLS Note that the orgal X, ot ˆX, s used the computato of the resduals ˆε! The asymptotc varace of the TSLS estmator ca show to be larger tha that of the OLS estmator, especally whe the strumets are poor (.e. ot hghly correlated wth the troublemaker(s. I other words, the TSLS estmator s less effcet tha the OLS estmator. However, effcecy s ot a very meagful vrtue f the estmator s cosstet! Specfcato tests for OV-type problems Hausma Test (H-test my jargo The Hausma test s a type of Wald test that eames f the dfferece betwee sets of estmates flowg from dfferet models, weghted by the dfferece ther asymptotc varace-covarace matr, s large eough to reject the ull hypothess that they are the same. The ratoale s that the frst model cosdered s kow to geerate cosstet estmates uder OV-type problems or other ms-specfcato ssues, whle the secod model s cosstet IF there are deed OV type problems or ms-specfcatos. However, the frst estmator s always less effcet tha the secod. So f there are o OV-type or ms-specfcato problems, t would be better to choose the secod model. If there are OV type problems, we should use the frst model (sce cosstecy s geerally more mportat tha effcecy. IV 5

6 6 For the case at had the IV (or TSLS model s model cosstet uder OV-problems, but less effcet. The OLS model s model more effcet, but cosstet uder OV-problems. The H-test eames f the two estmators are close eough to coclude that OLS s fe,.e. that there are o OVtype problems (ths s the ull hypothess. If the weghted dfferece betwee the estmators s too large, the test would reject the ull. The H-test statstc s thus derved as ( ˆ ( ˆ ( ( ˆ ˆ a TSLS a ( ~ χ ( H = d V b V b d= d s XX s XX d J - where = (, ˆ e'e d btsls b X= Z( ZZ ZX, s = k (4 By coveto the squared regresso error (s from the OLS model s used for both varaces ths formula. Beg a type of Wald test, the test statstc s dstrbuted ch-square wth J degrees of freedom, where J s equal to the umber of troublemakers the orgal model (.e. the colums of X that are beg replaced by strumets. A commo problem that arses wth ths test s that ( XX ˆ ˆ ( XX may ot be full rak sce there are usually some commo colums ˆX ad X. Thus, the verse of the dfferece varaces may ot est. To get aroud ths problem the Moore-Perose Geeralzed Iverse s used stead. I R, just use pseudoverse ( the corpcor lbrary stead of solve. Wu test A equvalet test that avods ths verse problem s the Wu test. It proceeds steps:. Pck the troublemakers from X ad collect them a ew matr, say X. Regress each colum X agast Z ad obta the ftted values,.e. geerate ˆ = ( - X Z ZZ ZX.. Regress y agast the orgal X ad X ˆ. The ull hypothess for the test s that the coeffcets for X ˆ are jotly zero. A rejecto of the ull would dcate OV problems the OLS model, ad would suggest swtchg to a IV approach. If the ull s ot rejected, OLS s fe. Ths s mplemeted as a F-test wth (-k ad J degrees of freedom, where J s aga the umber of troublemakers, here equal to the colums X ad k s the umber of regressors the fal model step. Eamples Eample : OV-type problems through edogeety Edogeety bas arses whe oe or more eplaatory varables the ma equato of terest are themselves drectly or drectly drve by the depedet varable. Ths s also kow as smultaeous equato bas. Ofte tmes the varables are lked by some uderlyg theoretcal relatoshp ( whch case we ca usually atcpate trouble... The upshot s that ths edogeety troduces a correlato betwee X ad ε, rederg the OLS estmator based ad cosstet. 6

7 7 Scrpt mod4sb llustrates ths effect. It s based o Greee s (5 th edto eample 5.3. The ma equato of terest s a regresso of atoal cosumpto epedtures at tme t agast atoal dsposable come at tme t,.e. y = β + β dp + ε (5 t t t However, the atoal accoutg detty stpulates that atoal come s the sum of cosumpto, vestmet, govermet spedg, ad et eports. Thus, at least at the macroecoomc level, come may well be drve by cosumpto. Let s assume for smplcty that ths relatoshp s aga lear, such that: dp = γ + γ y + δ + µ (6 t t t t where t cludes other varables that affect dp t, ad the frst equato to the secod, we get µ t s a well behaved CLRM error term. Pluggg γ+ γβ µ t γ tδ + dpt = γ+ γ( β+ βdpt + ε t + tδ + µ t dpt = + + εt (7 γβ γβ γβ For the frst equato, ths mples that γ+ γβ 0 µ t γ tδ + γ COV ( dpt, εt = COV + + εt, εt = σε 0 γβ γβ γβ γβ (8 As ca be see from the last term the stregth ad drecto of the lk betwee dp t, ad the error term depeds o the magtude ad sg of γ ad β. Eample : OV-type problems through measuremet error Assume the correct form of your CLRM equato of terest s gve by y = β + β, + 3:k, 3:k + ε, β (9 Further assume that regressor s measured wth error for the etre sample (or at least for more tha a hadful of observatos. We oly observe. Assume that the relatoshp betwee ad s gve as ( σ = + μ wth μ~ 0, µ I (0 Ths mples that the measuremet error s perfectly radom. Other types of measuremet error are possble. Ether way, we ru to the followg dlemma (show for smplcty at the dvdual level: 7

8 8 ( β β ( y = β + β µ + + ε = β + β + + ε β µ = 3:k, 3:k 3:k, 3:k β + β + β + ω ad ( 3:k, 3:k ( COV ( ω = + µ ε βµ =βσ u COV,, Aga, we ed up wth a ozero correlato betwee a cluded regressor ad the error term of the estmated regresso. Smlar problems arse f multple colums of X are measured wth error. As for the classc OV case, a sgle ms-measured varable geerally bases all estmated coeffcets (uless the ms-measured varable s perfectly orthogoal wth the rest of X. The remedy s aga to clease each troublemaker wth oe or more strumets. I ths case, a good strumet wll be correlated wth the troublemaker, but ot wth ether the regresso error ( ε or the measuremet error ( μ. R scrpt mod4sc provdes a eample. 8

Wu-Hausman Test: But if X and ε are independent, βˆ. ECON 324 Page 1

Wu-Hausman Test: But if X and ε are independent, βˆ. ECON 324 Page 1 Wu-Hausma Test: Detectg Falure of E( ε X ) Caot drectly test ths assumpto because lack ubased estmator of ε ad the OLS resduals wll be orthogoal to X, by costructo as ca be see from the momet codto X'