Lecture 1 Review I. RS Lecture 1. CLM - Assumptions. Least Squares Estimation f.o.c. Least Squares Estimation - Assumptions

Transcription

1 RS Lecture CLM - Assuptions Lecture Review I Typicl Assuptions (A) DGP: y = Xβ + ε is correctly specified. (A) E[ε X] = (A3) Vr[ε X] = σ I T (A4) X hs full colun rnk rnk(x)=k-, where T k. Assuption (A) is clled correct specifiction. We know how the DGP. Assuption (A) is clled regression. Fro (A) we get: (i) E[ε X] = => E[y X] = f(x, θ) + E[ε X] = f(x, θ) (ii) Using the Lw of Iterted Expecttions (LIE): E[ε] = E X [E[ε X]] = E X [] = Lest Squres Estition - Assuptions Fro Assuption (A3) we get Vr[ε X] = σ I T => Vr[ε] = σ I T This ssuption iplies (i) hooscedsticity => E[ε i X] = σ for ll i. (ii) no seril/cross correltion => E[ ε i ε j X] = for i j. Fro Assuption (A4) => the k independent vribles in X re linerly independent. Then, the kxk trix X X will lso hve full rnk i.e., rnk(x X) = k. Lest Squres Estition f.o.c. Objective function: S(x i, θ) =Σ i ε i We wnt to iniize w.r.t to θ. The f.o.c. deliver the norl equtions: - Σ i [y i - f(x i, θ LS )] f (x i, θ LS ) = - (y- Xb) X = Solving for b delivers the OLS estitor: b = (X X) - X y Note: (i) b = β OLS. (Ordinry LS. Ordinry=liner) (ii) b is (liner) function of the dt (y i,x i ). (iii) X (y-xb) = X y - X X(X X) - X y = X e = => e X. OLS Estition - Properties Under the typicl ssuptions, we cn estblish properties for b. ) E[b X]= β ) Vr[b X] = E[(b-β) (b-β) X] =(X X) - X E[ε ε X] X(X X) - = σ (X X) - 3) b is BLUE (or MVLUE) => The Guss-Mrkov theore. (4) If (A5) ε X ~N(, σ I T ) => b X ~N(β, σ (X X) - ) => b k X ~N(β k, σ (X X) - kk ) (the rginls of ultivrite norl re lso norl.) Estiting σ Under (A5), E[e e X] = (T-k)σ The unbised estitor of σ is s = e e/(t-k). => there is degrees of freedo correction. Goodness of Fit of the Regression After estiting the odel, we judge the dequcy of the odel. There re two wys to do this: - Visul: plots of fitted vlues nd residuls, histogrs of residuls. - Nuericl esures: R, djusted R, AIC, BIC, etc. Nuericl esures. We cll the goodness-of-fit esures. Most populr: R. R = SSR/TSS = b X M Xb/y M y = - e e/y M y Note: R is bounded by zero nd one only if: () There is constnt ter in X --we need e M X=! (b) The line is coputed by liner lest squres. (c) R never flls when regressors re dded to the regression.

2 RS Lecture Adjusted R-squred R is odified with penlty for nuber of preters: Adjusted R R = - [(T-)/(T-k)]( - R ) = - [(T-)/(T-k)] RSS/TSS = - [RSS/(T-k)] [(T-)/TSS] => xiizing djusted R <=> iniizing [RSS/(T-k)]= s Degrees of freedo --i.e., (T-k)-- djustent ssues soething bout unbisedness. Adjusted-R includes penlty for vribles tht do not dd uch fit. Cn fll when vrible is dded to the eqution. It will rise when vrible, sy z, is dded to the regression if nd only if the t-rtio on z is lrger thn one in bsolute vlue. Other Goodness of Fit Mesures There re other goodness-of-fit esures tht lso incorporte penlties for nuber of preters (degrees of freedo). Infortion Criteri - Aeiy: [e e/(t K)] ( + k/t) - Akike Infortion Criterion (AIC) AIC = -/T(ln L k) L: Likelihood => if norlity AIC = ln(e e/t) + (/T) k (+constnts) - Byes-Schwrz Infortion Criterion (BIC) BIC = -(/T ln L [ln(t)/t] k) => if norlity AIC = ln(e e/t) + [ln(t)/t] k (+constnts) Mxiu Likelihood Estition We ssue the errors, ε, follow distribution. Then, we select the preters of the distribution to xiize the likelihood of the observed sple. Exple: The errors, ε, follow the norl distribution: (A5) ε X ~N(, σ I T ) Then, we cn write the joint pdf of y s / f ( yt ) = ( ) exp[ ( y ' ) ] t xt β πσ σ T / L = f ( y, y,..., y T β, σ ) =Π t = ( ) exp[ ( y ' ) ] exp( ) t xt β = e e T / πσ σ (πσ ) σ Tking logs, we hve the log likelihood function T T ln L = ln π ln σ e e σ Mxiu Likelihood Estition Let θ =(β,σ). Then, we wnt to T T Mxθ ln L( θ y, X ) = ln π σ ( y Xβ) ( y Xβ) σ Then, the f.o.c.: ln L = ( X y X Xβ) = ( X y X Xβ) = β σ σ ln L T = + ( y Xβ) ( y Xβ) = 4 σ σ σ Note: The f.o.c. deliver the norl equtions for β! The solution to the norl eqution, β MLE, is lso the LS estitor, b. Tht is, βˆ MLE = b = ( X X ) X y; σˆ e e = T Nice result for b: ML estitors hve very good properties! MLE Properties of ML Estitors () Efficiency. Under generl conditions, we hve tht θ Vr( θ MLE ) [ ni ( θ)] The right-hnd side is the Crer-Ro lower bound (CR-LB). If n estitor cn chieve this bound, ML will produce it. () Consistency. S n (X; θ) nd ( θˆmle ^ ^ MLE - θ) converge together to zero (i.e., expecttion). Properties of ML Estitors (4) Sufficiency. If single sufficient sttistic exists for θ, the MLE of θ ust be function of it. Tht is, θˆmle depends on the sple observtions only through the vlue of sufficient sttistic. (5) Invrince. The ML estite is invrint under functionl trnsfortions. Tht is, if θˆmle is the MLE of θ nd if g(θ) is function of θ, then g( ) is the MLE of g(θ). θˆmle (3) Theore: Asyptotic Norlity Let the likelihood function be L(X,X, X n θ). Under generl conditions, the MLE of θ is syptoticlly distributed s ˆ θ MLE N ( θ, [ ni ( θ )] )

3 RS Lecture Specifiction Errors: Oitted Vribles Oitting relevnt vribles: Suppose the correct odel is y = X β + X β + ε -i.e., with two sets of vribles. But, we copute OLS oitting X. Tht is, y = X β + ε Soe esily proved results: <= the short regression. () E[b X] = E [(X X ) - X y] = β + (X X ) - X X β β. => Unless X X =, b is bised. The bis cn be huge. () Vr[b X] Vr[b. X] => sller vrince when we oit X. (3) MSE => b y be ore precise. Specifiction Errors: Irrelevnt Vribles Irrelevnt vribles Suppose the correct odel is y = X β + ε But, we estite y = X β + X β + ε Let s copute OLS with X, X. This is clled long regression. Soe esily proved results: () Since the vribles in X re truly irrelevnt, then β =, so E[b. X] = β => No bis () Inefficiency: Bigger vrince Liner Restrictions Q: How do liner restrictions ffect the properties of the lest squres estitor? Model ( DGP): y = Xβ + ε Theory (infortion): Rβ - q = Restricted LS estitor: b* = b - (X X) - R [R(X X) - R ] - (Rb - q). Unbised? YES. E[b* X] = β. Efficiency? NO. Vr[b* X] < Vr[b X] 3. b* y be ore precise. Precision = MSE = vrince + squred bis. 4. Recll: e e = (y -Xb) (y-xb) e* e* = (y Xb*) (y-xb*) => Restrictions cnnot increse R => R R * The Generl Liner Hypothesis: H : Rβ - q = We hve J joint hypotheses. Let R be Jxk trix nd q be Jx vector. Two pproches to testing (unifying point: OLS is unbised): () Is Rb - q close to? Bsing the test on the discrepncy vector: = Rb - q. Using the Wld sttistic: W = (Vr[ X]) - Vr[ X] = R[σ (X X) - ]R. W = (Rb q) {R[σ (X X) - ]R} - (Rb q) Under the usul ssuption nd ssuing σ is known, W ~ χ J In generl, σ is unknown, we use s = e e/(t-k) W* = (Rb - q) {R[s (X X) - ]R} - (Rb - q) = (Rb q) {R[σ (X X) - ]R} - (Rb q)/(s /σ ) F = W/J / [(T-k) (s /σ )/(T-k)] = W*/J ~ F J,T-k. The Generl Liner Hypothesis: H : Rβ - q = () We know tht iposing the restrictions leds to loss of fit. R ust go down. Does it go down lot? -i.e., significntly? Recll (i) e* = y Xb* = e X(b* b) (ii) b*= b (X X) - R [R(X X) - R ] - (Rb q) => e* e* - e e = (Rb q) [R(X X) - R ] - (Rb q) Recll - W = (Rb q) {R[σ (X X) - ]R} - (Rb q) ~ χ J (if σ known) - e e/σ ~ χ T-k. Then, F = (e* e* e e)/j / [e e/(t-k)] ~ F J,T-K. Or F = { (R - R* )/J } / [( - R )/(T-k)] ~ F J,T-K. Exple: Testing H : Rβ - q = In the liner odel y = X β + ε = β + X β + X 3 β 3 + X 4 β 4 + ε We wnt to test if the slopes X 3, X 4 re equl to zero. Tht is, H : β = β = H : β or β 4 or both β 3 nd β 4 Y Y We cn use, F = (e* e* e e)/j / [e e/(t-k)] ~ F J,T-K. Define β + β X + ε = = β + β X + β X + β X + ε RSS R RSS U F (cost in df, unconstr df ) RSS = R-RSS U k U -k R RSS U T-k U 3

4 RS Lecture Functionl For: Chow Test Assuption (A) restricts f(x,β) to be liner function: f(x,β) = X β. But, within the frework of OLS estition, we cn be ore flexible: () We cn ipose non-liner functionl fors, s long s they re liner in the preters (intrinsic liner odel). () We cn use qulittive vribles (duies) to crete non-linerities (splines, chnges in regie, etc.) A Chow test (n F-test) cn be used to check for regies/ctegories or structurl breks. () Run OLS with no distinction between regies. Keep RSS R. (b) Run two seprte OLS, one for ech regie (Unrestricted regression). Keep RSS nd RSS => RSS U = RSS + RSS. (3) Run stndrd F-test (testing Restricted vs. Unrestricted odels): ( RSSR RSSU ) /( k F = ( RSS ) /( T k U U U k ) R ) ( RSSR [ RSS + RSS ]) / k = ( RSS + RSS ) /( T k) 3 Functionl For: Rsey s RESET Test To test the specifiction of the functionl for, we cn use the RESET test. Fro regression, we keep the fitted vlues, ŷ = Xb. Then, we dd ŷ to the regression specifiction. If ŷ is dded to the regression specifiction, it should pick up qudrtic nd interctive nonlinerity: y = X β + ŷ γ + ε We test H (liner functionl for): γ= H ( non liner functionl for): γ => t-test on the OLS estitor of γ. If the t-sttistic for ŷ is significnt => evidence of nonlinerity. 3 Prediction Intervls Prediction: Given x => predict y. () Estite: E[y X, x ] = β x ; () Prediction: y = β x + ε Predictor: ŷ = b x + estite of ε. (Est. ε =, but with vrince) Forecst error. We predict y with ŷ = b x. ŷ - y = b x - β x - ε = (b - β) x - ε => Vr[(ŷ -y ) x ] = E[(ŷ -y ) (ŷ -y ) x ]= x Vr[(b - β) x ]x + σ How do we estite this? Two cses: () If x is vector of constnts => For C.I. s usul. () If x hs to be estited => Coplicted (wht is the vrince of the product?). Use bootstrpping. Forecst Vrince Vrince of the forecst error is σ + x Vr[b x ]x = σ + σ [x (X X) - x ] If the odel contins constnt ter, this is K K jk Vr[ e ] = σ + + ( x j x j )( xk xk )( M ) n j= k= (where is X without x =ί). In ters squres nd cross products of devitions fro ens. Note: Lrge σ, sll n, nd lrge devitions fro the ens, decrese the precision forecsting error. Interprettion: Forecst vrince is sllest in the iddle of our experience nd increses s we ove outside it. Forecsting perfornce of odel: Tests nd esures of perfornce Evlution of odel s predictive ccurcy for individul (insple nd out-of-sple) observtions Evlution of odel s predictive ccurcy for group of (insple nd out-of-sple) observtions Chow prediction test Evlution of forecsts Sury esures of out-of-sple forecst ccurcy Men Error = Men Absolute Error (MAE) = Men Squred Error (MSE) = Root Men Squre Error (RMSE)= ( yˆ i yi ) = ei yˆ i yi = ei ( yˆ i yi ) = ei ( yˆ i yi ) = ei Theil s U-stt = U = ei T yi T

5 RS Lecture CLM: Asyptotics To get exct results for OLS, we rely on (A5) ε X ~iid N(, σ I T ) But, (A5) in ny situtions is unrelistic. Then, we study on the behvior of b (nd the test sttistics) when T i.e., lrge sples. New ssuptions: () {x i,ε i },,..., T is sequence of independent observtions. - X is stochstic, but independent of the process generting ε. - We require tht X hve finite ens nd vrinces. Siilr requireent for ε, but we lso require E[ε]=. () Well behved X: pli (X X/T) = Q (Q pd trix of finite eleents) => (not too uch dependence in X). CLM: New Assuptions Now, we hve new set of ssuptions in the CLM: (A) DGP: y = X β + ε. (A ) X stochstic, but E[X ε]= nd E[ε]=. (A3) Vr[ε X] = σ I T (A4 ) pli (X X/T) = Q (p.d. trix with finite eleents, rnk= k) We wnt to study the lrge sple properties of OLS: Q : Is b consistent? s? YES & YES Q : Wht is the distribution of b? b N(β,(σ /T)Q - ) Q 3: Wht bout the distribution of the tests? => t T =[(z T - µ)/s T ] d N(,) => W = (z T - µ) S - T (z T - µ) d χ rnk(st) => F d χ rnk(vr[]) Asyptotic Tests: Sll sple behvior? The p-vlues fro syptotic tests re pproxite for sll sples. They y be very bd. Their perfornce depends on: () Sple size, T. () Distribution of the error ters, ε. (3) The nuber of regressors, k, nd their properties (4) The reltionship between the error ters nd the regressors. A siultion/bootstrp cn help. Bootstrp tests tend to perfor better thn tests bsed on pproxite syptotic distributions. The errors coitted by both syptotic nd bootstrp tests diinish s T increses. The Delt Method It is used to obtin the syptotic distribution of non-liner function of RV (usully, n estitor). Tools: () A first-order Tylor series expnsion () Slutsky s theore. Let x n be RV, with pli x n =θ nd Vr(x n )=σ <. We use the CLT to obtin n ½ (x n - θ)/σ d N(,) Gol: g(x n )? (g(x n ) is continuous differentible function, independent of n.) Steps: () Tylor series pproxition round θ : g(x n ) = g(θ) + g (θ) (x n - θ) + higher order ters We ssue the higher order ters re o(n) --s n grows, they vnish. The Delt Method () Use Slutsky theore: pli g(x n ) = g(θ) pli g (x n ) = g (θ) Then, s n grows, g(x n ) g(θ) + g (θ) (x n - θ) => n ½ ([g(x n ) - g(θ)]) g (θ) [n ½ (x n - θ)]. => n ½ ([g(x n ) - g(θ)]/σ) g (θ) [n ½ (x n - θ)/σ]. The syptotic distribution of (g(x n ) - g(θ)) is given by tht of [n ½ (x n - θ)/σ], which is stndrd norl. Then, n ½ ([g(x n ) - g(θ)]) N(, [g (θ)] σ ). After soe work ( inversion ), we obtin: g(x n ) N(g(θ), [g (θ)] σ /n). Delt Method: Exple Let x n N(θ, σ /n) Q: g(x n )=δ/x n? (δ is constnt) First, clculte the first two oents of g(x n ): g(x n ) = δ/x n => pli g(x n )=(δ/θ) g (x n ) = -(δ/x n ) => pli g (x n )=-(δ/θ ) Recll delt ethod forul: g(x n ) Then, g(x n ) N(δ/θ, (δ /θ 4 )σ /n) N(g(θ), [g (θ)] σ /n).

6 RS Lecture The IV Proble Wht kes b consistent when X'ε /T p is tht pproxiting (X'ε/T ) by is resonbly ccurte in lrge sples. Now, we chllenge the ssuption tht {x i,ε i } is sequence of independent observtions. Now, we ssue pli (X ε/t) => This is the IV Proble! Q: When ight X be correlted ε? - Correlted shocks cross linked equtions - Siultneous equtions - Errors in vribles - Model hs lgged dependent vrible nd serilly correlted error ter The IV Proble We strt with our liner odel y = Xβ + ε. Now, we ssue pli(x ε/t). pli (X X/T) = Q Then, pli b = pli β + pli (X X/T) - pli (X ε ε/t) = β + Q - pli (X ε ε/t) β => b is not consistent estitor of β. New ssuption: we hve l instruentl vribles, such tht pli( X/T) but pli( ε/t) = Instruentl Vribles: Assuptions To get consistent estitor of β, we lso ssue: {x i, z i, ε i } is sequence of RVs, with: E[X X] = Q xx (pd nd finite) (LLN => pli(x X/T) =Q xx ) E[ ] = Q zz (finite) (LLN => pli( /T) =Q zz ) E[ X] = Q zx (pd nd finite) (LLN => pli( X/T) =Q zx ) E[ ε] = (LLN => pli( ε/t) = ) Following the se ide s in OLS, we get syste of equtions: W' X b IV = W' y Instruentl Vribles: Estition To get the IV estitor, we strt fro the syste of equtions: W' X b IV = W' y Cse : l = k -i.e., nuber of instruents = nuber of regressors. - hs the se diensions s X: Txk => X is kxk trix - In this cse, W is irrelevnt, sy, W=I. - Then, b IV = ( X) - y We hve two cses where estition is possible: - Cse : l = k -i.e., nuber of instruents = nuber of regressors. - Cse : l > k -i.e., nuber of instruents > nuber of regressors. IV Estitors Properties of b IV () Consistent b IV = ( X) - y = ( X) - (Xβ+ε) = ( X/T) - ( X/T) β + ( X/T) - ε/t = β + ( X/T) - ε/t p β (under ssuptions) () Asyptotic norlity T (b IV - β) = T ( X) - ε = ( X/T) - T ( ε/t) Using the Lindberg-Feller CLT T ( ε/t) d N(, σ Q zz ) Then, T (b IV - β) d N(, σ Q - zx Q zz Q - xz ) IV Estitors Properties of σˆ, under IV estition: - We define σˆ : T T ˆ = e IV ( y i x' b IV ) T = T i = i = σ where e IV = y - X b IV = y - X( X) - y = [I - X( X) - ]y = M zx y - Then, σˆ = e IV 'e IV /T = ε'm zx 'M zx ε/t = ε'ε/t ε'x ( X) - ε/t + ε' ('X) - X X( X) - ε/t => pli σˆ = pli(ε'ε/t) - pli[(ε'x/t) ( X/T) - ('ε/t)] + + pli(ε' ( X) - X X( X) - ε/t) = σ Est Asy. Vr[b IV ] = E[('X) - εε εε' ( X) - ]= ˆσ ( X) - '( X) -

7 RS Lecture IV Estitors: SLS (-Stge Lest Squres) Cse : l > k -i.e., nuber of instruents > nuber of regressors. - This is the usul cse. We cn throw l-k instruents, but throwing wy infortion is never optil. - The IV norl equtions re n l x k syste of equtions: y = Xβ+ ε Note: We cnnot pproxite ll the ε by siultenously. There will be t lest l-k non-zero residuls. (Siilr setup to regression!) IV Estitors: SLS (-Stge Lest Squres) We cn esily derive properties for b IV : biv = ( X ' P X ) X ' P y = ( X ' P P X ) Xˆ Xˆ ( ' ) X ˆ ' y ( Xˆ ' X ˆ = = ) Xˆ ' yˆ X ' P P y () b IV is consistent () b IV is syptoticlly norl. - This is estitor is lso clled GIVE (Generlized IV estitor) - Fro the IV norl equtions => W' X b IV = W' y - We define different IV estitor - Let W = ( ) - X = P X = Xˆ - Then, X'P X b IV = X'P y b X P X X P y X P P X X P P y X ˆ X ˆ = ( ' ) ' = ( ' ) ' = ( ' ) X ˆ ' yˆ IV Interprettions of b IV b b IV IV = b X ˆ X ˆ SLS = ( ' ) ( Xˆ = ' X ) X ˆ ' y X ˆ ' y This is the SLS interprettion This is the usul IV = Xˆ Asyptotic Efficiency The vrince is lrger thn tht of LS. (A lrge sple type of Guss-Mrkov result is t work.) () OLS is inconsistent. () Men squred error is uncertin: MSE[estitor β] = Vrince + squre of bis. IV y be better or worse. Depends on the dt: X nd ε. Probles with SLS X/T y not be sufficiently lrge. The covrince trix for the IV estitor is Asy. Cov(b) = σ [( X)( ) - (X )] - If X/T goes to (wek instruents), the vrince explodes. When there re ny instruents, Xˆ is too close to X; SLS becoes OLS. Populr isconception: If only one vrible in X is correlted with ε, the other coefficients re consistently estited. Flse. => The proble is sered over the other coefficients. Wht re the finite sple properties of b IV? We do not hve the condition E[ε X] =, we cnnot conclude tht b IV is unbised, or tht it hs Vr[b SLS ] equl to its syptotic covrince trix. => In fct, b SLS cn hve very bd sll-sple properties. Endogeneity Test (Husn) Exogenous Endogenous OLS Consistent, Efficient Inconsistent SLS Consistent, Inefficient Consistent Bse test on d = b SLS - b OLS - We cn use Wld sttistic: d [Vr(d)] - d Note: Under H (pli (X ε/t) = ) b OLS = b SLS = b Also, under H : Vr[b SLS ]= V SLS > Vr[b OLS ]= V OLS => Under H, one estitor is efficient, the other one is not. Q: Wht to use for Vr(d)? - Husn (978): V = Vr(d) = V SLS - V OLS H = (b SLS - b OLS ) [V SLS - V OLS ] - (b SLS - b OLS ) d χ rnk(v) Endogeneity Test: The Wu Test The Husn test is coplicted to clculte Siplifiction: The Wu test. Consider regression y = Xβ + ε, n rry of proper instruents, nd n rry of instruents W tht includes plus other vribles tht y be either clen or continted. Wu test for H : X is clen. Setup () Regress X on. Keep fitted vlues Xˆ = ( ) - X () Using W s instruents, do SLS regression of y on X, keep RSS. (3) Do SLS regression of y on X nd subset of coluns of Xˆ tht re linerly independent of X. Keep RSS. (4) Do n F-test: F = [(RSS - RSS )/]/[RSS /(T-k)].

8 RS Lecture Endogeneity Test: The Wu Test Under H : X is clen, the F sttistic hs n pproxite F,T-k distribution. Dvidson nd McKinnon (993, 39) point out tht the DWH test relly tests whether possible endogeneity of the right-hnd-side vribles not contined in the instruents kes ny difference to the coefficient estites. These types of exogeneity tests re usully known s DWH (Durbin, Wu, Husn) tests. Endogeneity Test: Augented DWH Test Dvidson nd McKinnon (993) suggest n ugented regression test (DWH test), by including the residuls of ech endogenous righthnd side vrible. Model: y = Xβ+ Uγ + ε, we suspect X is endogenous. Steps for ugented regression DWH test:. Regress x on IV () nd U: x = П + U φ + υ => sve residuls v x. Do n ugented regression: y = Xβ + Uγ + v x δ + ε 3. Do t-test of δ. If the estite of δ, sy d, is significntly different fro zero, then OLS is not consistent. Mesureent Error DGP: y* = βx* + ε - ε ~ iid D(, σ ε ) But, we do not observe or esure correctly x*. We observe x, y: x = x* + u u ~ iid D(, σ u ) -no correltion to ε,v y = y* + v v ~ iid D(, σ v ) -no correltion to ε,u Let s consider two cses: Mesureent Error CASE - Only y* is esured with error. y* = y - v = βx* + ε => y = βx* + ε + v = βx* + (ε + v) Q: Wht hppens when y is regressed on x? A: Nothing! We hve our usul OLS proble since ε nd v re independent of ech other nd x *. CLM ssuptions re not violted! CASE - Only x* is esured with error (y=y*): y = β(x- u) + ε = βx + ε - βu = βx + w E[x w] = E[(x* + u) (ε - βu)] = -βσ u => CLM ssuptions violted => OLS inconsistent! Finding n Instruent: Not Esy The IV proble requires dt on vribles () such tht () Cov(x,) -relevnce condition () Cov(,ε) = -vlid (exogeneity) condition Wek Instruents: Finnce ppliction Finnce exple: The consuption CAPM. In both liner nd nonliner versions of the odel, IVs re wek, -- see Neeley, Roy, nd Whiten (), nd Yogo (4). Then, we do first-stge regression to obtin fitted vlues of X: x = П + Uδ + V -V ~N(, σ V I) Then, using the fitted vlues we estite nd do tests on β. Finding tht eets both requireents is not esy. - The vlid condition is not tht coplicted to eet. - The relevnt condition is ore coplicted: Finding correlted with X. But, the explntory power of y not be enough to llow inference on β. In this cse, we sy is wek instruent. In the liner odel in Yogo (4): X (endogenous vrible): consuption growth (the IVs): twice lgged noinl interest rtes, infltion, consuption growth, nd log dividend-price rtio. But, log consuption is close to rndo wlk, consuption growth is difficult to predict. This leds to the IVs being wek. => Yogo (4) finds F-sttistics for H : П = in the st stge regression tht lie between.7 nd 3.53 for different countries.

9 RS Lecture Wek Instruents: Sury Even if the instruent is good i.e., it eets the relevnt condition--, tters cn be de fr worse with IV s opposed to OLS ( the cure cn be worse... ). Wek correltion between IV nd endogenous regressor cn pose severe finite-sple bis. Even sll Cov(,e) will cuse inconsistency, nd this will be excerbted when Cov(X,) is sll. Lrge T will not help. A&K nd Consuption CAPM tests hve very lrge sples! Wek Instruents: Detection nd Reedies Sypto: The relevnce condition, pli( X/T ) not zero, is close to being violted. Detection of wek IV: Stndrd F test in the st stge regression of x k on. Stiger nd Stock (997) suggest tht F < is sign of probles. Low prtil-r X,. Lrge Vr[b IV ] s well s potentilly severe finite-sple bis. Reedy: Not uch ost of the discussion is bout the condition, not wht to do bout it. Use LIML? Requires norlity ssuption. Probbly, not too restrictive. (Text, ) Wek Instruents: Detection nd Reedies Sypto: The vlid condition, pli( ε/t ) zero, is close to being violted. Detection of instruent exogeneity: Endogenous IV s: Inconsistency of b IV tht kes it no better (nd probbly worse) thn b OLS Durbin-Wu-Husn test: Endogeneity of the proble regressor(s) Reedy: Avoid endogeneous wek instruents. (Also void wek IV!) Generl proble: It is not esy to find good instruents in theory nd in prctice. Find nturl experients. M-Estition An extreu estitor is one obtined s the optiizer of criterion function, q(z,b). Exples: OLS: b = rg x (-e e/t) MLE: b MLE = rg x ln L =,,T ln f (y i,x i,b) GMM: b GMM = rg x - g(y i,x i,b) W g(y i,x i,b) There re two clsses of extreu estitors: - M-estitors: The objective function is sple verge or su. - Miniu distnce estitors: The objective function is esure of distnce. "M" stnds for xiu or iniu estitors --Huber (967). M-Estition The objective function is sple verge or su. For exple, we wnt to iniize popultion (first) oent: in b E[q(z,β)] Using the LLN, we ove fro the popultion first oent to the sple verge: i q(z i,b)/t p E[q(z,β)] We wnt to obtin: b = rgin i q(z i,b) (or divided by T) In generl, we solve the f.o.c. (or zero-score condition): ero-score: i q(z i,b)/ b = M-Estition If s(z,b) = q(z,b)/ b exists (lost everywhere), we solve i s(z i,b M )/T = (*) If, in ddition, E X [s(z,b)] = / b E X [q(z,b)] -i.e., differentition nd integrtion re exchngeble-, then E X [ q(z,β)/ β ] =. Under these ssuptions, the M-estitor is sid to be of ψ-type (ψ= s(z,b)=score). Often, b M is tken to be the solution of (*) without checking whether it is indeed iniu). To check the s.o.c., we define the (pd) Hessin: H = i q(z i,b)/ b b Otherwise, the M-estitor is of ρ-type. (ρ= q(z,β)).

10 RS Lecture M-Estition: LS & ML Lest Squres DGP: y = f(x,β) + ε, z =[y,x] q(z;β) = S(β) = ε ε =,,T (y i - f(x i ;β)) Now, we ove fro popultion to sple oents q(z;b) = S(b) = e e =,,T (y i - f(x i ;b)) b NLLS = rgin S(b) Mxiu Likelihood Let f (x i,β) be the pdf of the dt. L(x,β) = Π,,T f (x i ;β) log L(x,β) =,,T ln f (x i ;β) Now, we ove fro popultion to sple oents q(z,b) = -log L(x,b) b MLE = rgin log L(x;b) M-Estitors: Properties Under generl ssuptions, M-estitors re: - b p M b - b M N(b,Vr[b ]) - Vr[b M ] =(/T) H - V H - - If the odel is correctly specified: -H = V. Then, Vr[b] = V H nd V re evluted t b : - H = i [ q (z i,b)/ b b ] - V = i [ q(z i,b)/ b][ q(z i,b)/ b ] Nonliner Lest Squres: Exple Exple: Min β S(β) ={½ Σ i [y i - f(xβ)] } Fro the f.o.c., we cnnot solve for β explicitly. But, using soe steps, we cn still iniize RSS to obtin estites of β. Nonliner regression lgorith:. Strt by guessing plusible vlues for β, sy β.. Clculte RSS for β => get RSS(β ) 3. Mke sll chnges to β, => get β. 4. Clculte RSS for β => get RSS(β ) 5. If RSS(β ) < RSS(β ) => β becoes your new strting point. 6. Repet steps 3-5 until you RSS(β j ) cnnot be lowered. => get β j. => β j is the (nonliner) lest squres estites. 4 NLLS: Lineriztion We strt with nonliner odel: y i = f(x i,β) + ε i We expnd the regression round soe point, β : f(x i,β) f(x i,β ) + Σ k [ f(x i,β )/ β k ]( β k - β k ) = f(x i,β ) + Σ k x i ( β k - β k ) = [f(x i,β ) - Σ k x i β k ] + Σ k x i β k = f + Σ k x i β k = f + x i β where f i = f(x i,β ) - x i β (f i does not depend on unknowns) Now, f(x i,β) is (pproxitely) liner in the preters. Tht is, y i = f i + x i β + ε i (ε i = ε i + lineriztion error i) => y i = y i f i = x i β + ε i NLLS: Lineriztion We linerized f(x i,β) to get: y = f + X β + ε (ε = ε + lineriztion error) => y = y - f = X β + ε Now, we cn do OLS: b NLLS = (X X ) - X y Note: X re clled pseudo-regressors. NLLS: Lineriztion Copute the syptotic covrince trix for the NLLS estitor s usul: Est. Vr[b NLLS X ] = s NLLS (X X ) - s NLLS = [y - f(x i, b NLLS )] [y - f(x i, b NLLS )]/(T-k). Since the results re syptotic, we do not need degrees of freedo correction. However, df correction is usully included. In generl, we get different b NLLS for different β. An lgorith cn be used to get the best b NLLS. We will resort to nuericl optiiztion to find the b NLLS.

11 RS Lecture Guss-Newton Algorith b NLLS depends on β. Tht is, b NLLS (β ) = (X X ) - X y We use Guss-Newton lgorith to find the b NLLS. Recll GN: β k+ = β k + (J T J) - J T ε -- J: Jcobin = δf(xi;β)/δβ. Given b NLLS t step, b(j), we find the b NLLS for step j+ by: b(j+) = b(j) + [X (j) X (j)] - X (j) e (j) Coluns of X (j) re the derivtives: f(x i,b(j))/ b(j) e (j) = y - f[x,b(j)] The updte vector is the slopes in the regression of the residuls on X. The updte is zero when they re orthogonl. (Just like OLS) Box-Cox Trnsfortion A siple trnsfortion tht llows non-linerities in the CLM. y = f(x i,β) + ε = Σ k x k (λ) β k + ε x k (λ) = (x kλ -)/λ li λ (x kλ -)/λ = ln x k For given λ, OLS cn be used. An itertive process cn be used to estite λ. OLS s.e. hve to be corrected. Not very efficient ethod. NLLS or MLE will work fine. We cn hve ore generl Box-Cox trnsfortion odel: y (λ) = Σ k x k (λ) β k + ε Testing non-liner restrictions Testing liner restrictions s before. Non-liner restrictions chnge he usul tests. We wnt to test: H : R(β) = where R(β) is non-liner function, with rnk[ R(β)/ β=g(β)]=j. Let = R(b NLLS ). Then, W= (Vr[ X]) - = R(b NLLS ) (Vr[R(b NLLS ) X]) - R(b NLLS ) But, we do not know the distribution of R(b NLLS ). We know the distribution of b NLLS. Then, we linerize R(b NLLS ) round β: R(b NLLS ) R(β) + G(b NLLS ) (b NLLS - β) Testing non-liner restrictions Linerize R(b NLLS ) round β (=b ) R(b NLLS ) R(β) + G(b NLLS ) (b NLLS - β) Recll T (b M - b ) d N(, Vr[b ]) where Vr[b ] = H(β) - V (β) H(β) - => T [R(b NLLS ) - R(β)] N(, G(β) Vr[b ] G(β) d ) => Vr[R(b NLLS )] = (/T) G(β) Vr[b ] G(β) Then, W = T R(b NLLS ) {G(b NLLS ) Vr[b NLLS ] G(b NLLS ) } - R(b NLLS ) => W d χ J