7 Semiparametric Methods and Partially Linear Regression

Transcription

1 7 Semiparametric Metods and Partially Linear Regression 7. Overview A model is called semiparametric if it is described by and were is nite-dimensional (e.g. parametric) and is in nite-dimensional (nonparametric). All moment condition models are semiparametric in te sense tat te distribution of te data () is unspeci ed and in nite dimensional. But te settings more typically called semiparametric are tose were tere is explicit estimation of : In many contexts te nonparametric part is a conditional mean, variance, density or distribution function. Often is te parameter of interest, and is a nuisance parameter, but tis is not necessarily te case. In many semiparametric contexts, is estimated rst, and ten ^ is a two-step estimator. But in oter contexts (; ) are jointly estimated. 7.2 Feasible Nonparametric GLS A classic semiparametric model, wic is not in Li-Racine, is feasible GLS wit unknown variance function. Te seminal papers are Carroll (982, Annals of Statistics) and Robinson (987, Econometrica). Te setting is a linear regression y i = X i + e i E (e i j X i ) = E e 2 i j X i = 2 (X i ) were te variance function 2 (x) is unknown but smoot in x; and x 2 R q : (Te idea also applies to non-linear but parametric regression functions). parameter is = 2 () In tis model, te nonparametric nuisance As te model is a regression, te e ciency bound for is attained by GLS regression ~ =! 2 (X i ) X ixi! 2 (X i ) X iy i : Tis of course is infeasible. Carroll and Robinson suggested replacing 2 (X i ) wit ^ 2 (X i ) were ^ 2 (x) is a nonparametric estimator. (Carroll used kernel metods; Robinson used nearest neigbor metods.) estimator Speci cally, letting ^ 2 (x) be te NW estimator of 2 (x); we can de ne te feasible ^ =! ^ 2 (X i ) X ixi! ^ 2 (X i ) X iy i : Tis seems sensible. Te question is nd its asymptotic distribution, and in particular to nd 56

2 if it is asymptotically equivalent to ~ : 7.3 Generated Regressors Te model is y i = (X i ) + e i E (e i j X i ) = were is nite dimensional but is an unknown function. Suppose is identi ed by anoter equation so tat we ave a consistent estimate of ^(x) for (x):(imagine a non-parametric Heckman estimator). Ten we could estimate by least-squares of y i on ^(Z i ): Tis problem is called generated regressors, as te regressor is a (consistent) estimate of a infeasible regressor. In general, ^ is consistent. But wat is its distribution? 7.4 Andrews MINPIN Teory A useful framework to study te type of problem from te previous section is given in Andrews (Econometrica, 994). It is reviewed in section 7.3 of Li-Racine but te discussion is incomplete and tere is at least one important omission. reading Andrews paper. If you really want to learn tis teory I suggest Te setting is wen te estimator ^ MINimies a criterion function wic depends on a Preliminary In nite dimensional Nuisance parameter estimator, ence MINPIN. Let 2 be te parameter of interest, let 2 T denote te in nite-dimensional nuisance parameter. Let and denote te true values. Let ^ be a rst-step estimate of ; and assume tat it is consistent: ^! p Now suppose tat te criterion function for estimation of depends on te rst-step estimate ^: Let te criterion be Q n (; ) and suppose tat Tus ^ is a two-step estimator. Assume tat ^ = argmin Q n Q n (; ) = m n (; ) + o p pn m n (; ) = m i (; ) n 57

3 were m i (; ) is a function of te i t observation. In just-identi ed models, tere is no o p pn error (and we now ignore te presence of tis error). In te FGLS example, () = 2 () ; ^ () = ^ 2 () ; and In te generated regressor problem, m i (; ) = (X i ) X i y i X i : m i (; ) = (X i ) (y i (X i )) : In general, te rst-order condition (FOC) for ^ is = m n ^; ^ Assume ^! p : Implicit to obtain consistency is te requirement tat te population expecation of te FOC is ero, namely Em i ( ; ) = and we assume tat tis is te case. We expand te FOC in te rst argument = p n m n ^; ^ = p n m n ( ; ^) + M n ( ; ^) p n ^ + o p () were It follows tat p n ^ M n (; m n (; ) ' M n ( ; ^) p n m n ( ; ^) : If M n (; ) converges to its expectation M (; ) m i (; ) uniformly in its arguments, ten M n ( ; ^)! p M ( ; ) = M; say. Ten p n ^ ' M p n m n ( ; ^) : We cannot take a Taylor expansion in because it is in nite dimensional. Instead, Andrews uses a stocastic equicontinuity argument. De ne te population version of m n (; ) m (; ) = Em i (; ) : 58

4 Note tat at te true values m ( ; ) = (as discussed above) but te function is non-ero for generic values. De ne te function n () = p n ( m n ( ; ) m ( ; )) = p (m i ( ; ) Em i ( ; )) n Notice tat n () is a normalied sum of mean-ero random variables. Tus for any xed ; n () converges to a normal random vector. Viewed as a function of ; we migt expect n () to vary smootly in te argument : Te stocastic formulation of tis is called stocastic equicontinuity. Rougly, as n! ; n () remains well-beaved as a function of. Andrews rst key assumption is tat n () is stocastically equicontinuous. Te important implication of stocastic equicontinuity is tat ^! p implies Intuitively, if g() is continuous, ten g(^) n (^) n ( )! p : probability uniformly to a continuous function g() ten g n (^) equicontinuity is te most general, but still as te same implication. g( )! p : More generally, if g n () converges in g( )! p : Te case of stocastic Since Em i ( ; ) = ; wen we evaluate te empirical process at te true value we ave a ero-mean normalied sum, wic is asymptotically normal by te CLT: n ( ) = p n = p n (m i ( ; ) Em i ( ; )) m i ( ; )! d N (; ) were It follows tat = Em i ( ; ) m i ( ; ) n (^) = n ( ) + o p ()! d N (; ) and tus p n mn ( ; ^) = p n ( m n ( ; ^) m ( ; ^)) + p nm ( ; ^) = n (^) + p nm ( ; ^) 59

5 Te nal detail is wat to do wit p nm ( ; ^) : Andrews directly assumes tat p nm ( ; ^)! p Tis is te second key assumption, and we discuss it below. Under tis assumption, p n mn ( ; ^)! d N (; ) and combining tis wit our earlier expansion, we obtain: Andrews MINPIN Teorem. Under Assumptions -6 below, p n ^! p N (; V ) were V = M M M m i ( ; ) = Em i ( ; ) m i ( ; ) Assumption As n! ; for m (; ) = Em i (; ). ^! p 2. ^! p 3. p nm ( ; ^)! p 4. m ( ; ) = 5. n () is stocastically equicontinuous at : 6. m n (; m n (; ) satisfy uniform WLLN s over T: (Tey converge in probability to teir expectations, uniformly over te parameter space.) Discussion of result: Te teorem says tat te semiparametric estimator ^ as te same asymptotic distribution as te idealied estimator obtained by replacing te nonparametric estimate ^ wit te true function : Tus te estimator is adaptive. Tis migt seem too good to be true. Te key is assumption, wic olds in some cases, but not in oters. Discussion of assumptions. Assumptions and 2 state tat te estimators are consistent, wic sould be separately veri ed. Assumption 4 states tat te FOC identi es wen evaluated at te true : Assumptions 5 and 6 are regularity conditions, essentially smootness of te underlying functions, plus su cient moments. 6

6 7.5 Ortogonality Assumption Te key assumption 3 for Andrews MINPIN teorey was someow missed in te write-up in Li-Racine. Assumption 3 is not always true, and is not just a regularity condition. It requires a sort of ortogonality between te estimation of and : Suppose tat is nite-dimensional. Ten by a Taylor expansion p nm ( ; ^) ' p nm ( ; m ( ; ) p n (^ m ( ; ) p n (^ ) since m ( ; ) = by Assumption 4. Since is parametric, we sould expect p n (^ ) to converge to a normal vector. Tus tis expression will converge in probability to ero m ( ; ) = Recall tat m (; ) is te expectation of te FOC, wic is te derivative of te criterion : Tus m (; ) is te cross-derivative of te criterion, and te above statement is tat cross-derivative is ero, wic is an ortogonality condition (e.g. block diagonality of te Hessian.) Now wen is in nite-dimensional, te above argument does not work, but it lends intuition. An analog of te derivative condition, wic is su cient for Assumption 3, is tat for all in a neigborood of : In te FGLS example, m ( ; ) = m ( ; ) = E (X i ) X ie i = for all ; so tis is Assumption 3 is satis ed in tis example. We ave te implication: Teorem. Under regularity conditions te Feasible nonparametric GLS estimator of te previous section is asymptotically equivalent to infeasible GLS. In te generated regressor example m ( ; ) = E ((X i ) (y i (X i ))) = E ((X i ) (e i + ( (X i ) (X i )))) = E ((X i ) ( (X i ) (X i ))) Z = (x) ( (x) (x)) f(x)dx Assumption 3 requires p nm ( ; ^)! p. But note tat p nm ( ; ^) ' p n ( (x) ^(x)) wic certainly does not converge to ero. Assumption 3 is generically violated wen tere are generated regressors. 6

7 Tere is one interesting exception. Wen = ten m ( ; ) = and tus p nm ( ; ^) = so Assumption 3 is satisi ed. We see tat Andrews MINPIN assumption do not apply in all semiparametric models. Only tose wic satisfy Assumption 3, wic needs to be veri ed. Te oter key condition is stocastic equicontinuity, wic is di cult to verify but is genearly satis ed for well-beaved estimators. Te remaining assumptions are smootness and regularity conditions, and typically are not of concern in applications. 7.6 Partially Linear Regression Model Te semiparametric partially linear regression model is E (e i j X i ; Z i ) = y i = X i + g (Z i ) + e i E e 2 i j X i = x; Z i = = 2 (x; ) Tat is, te regressors are (X i ; Z i ); and te model speci es te conditional mean as linear in X i but possibly non-linear in Z i 2 R q : Tis is a very useful compromise between fully nonparametric and fully parametric. Often te binary (dummy) variables are put in X i : Often tere is just one nonlinear variable: q = ; to keep tings simple. Te goal is to estimate and g; and to obtain con dence intervals. Te rst issue to consider is identi cation. Since g is unconstrained, te elements of X i cannot be collinear wit any function of Z i : Tis means tat we must exclude from X i intercepts and any deterministic function of Z i : Te function g includes tese components. 7.7 Robinson s Transformation Robinson (Econometrica, 988) is te seminal treatment of te partially linear model. He rst contribution is to sow tat we can concentrate out te unknown g by using a genearliation of residual regression. Take te equation y i = Xi + g (Z i ) + e i and apply te conditional expectation operator E ( j Z i ) : We obtain E (y i j Z i ) = E X i j Z i + E (g (Zi ) j Z i ) + E (e i j Z i ) = E (X i j Z i ) + g (Z i ) 62

8 (using te law of iterated expectations). De ning te conditional expectations g y () = E (y i j Z i = ) g x () = E (X i j Z i = ) We can write tis expression as g y () = g x () + g() Subtracting from te original equation, te function g disappears: We can write tis as y i g y (Z i ) = (X i g x (Z i )) + e i e yi = e xi + e i y i = g y (Z i ) + e yi X i = g x (Z i ) + e xi Tat is, is te coe cient of te regression of e yi on e xi ; were tese are te conditional expectations errors from te regression of y i (and X i ) on Z i only. Tis is a conditional expecation generaliation of te idea of residual regression. Tis transformed equation immediately suggests an infeasible estimator for ; by LS of e yi on e xi : ~ = 7.8 Robinson s Estimator! e xi e xi e xi e yi Robinson suggested rst estimating g y and g x by NW regression, using tese to obtain residuals ^e xi and ^e xi ; and replacing tese in te formula for : ~ Speci cally, let ^g y () and ^g x () denote te NW estimates of te conditional mean of y i and X i given Z i = : Assuming q = ; P n k Zi ^g y () = P n k Zi P n k Zi ^g x () = P n k Zi Te estimator ^g x () is a vector of te same dimension as X i. 63 y i X i

9 Notice tat you are regressing eac variable (te y i and te Xi s) separately on te continuous variable Z i : You sould view eac of tese regressions as a separate NW regression. You sould probably use a di erent bandwidt for eac of tese regressions, as te depdendence on Z i will depend on te variable. (For example, some regressors X i migt be independent of Z i so you would want to use an in nite bandwidt for tose cases.) Wile Robinson discussed NW regression, tis is not essential. You could substitute LL or WNW instead. Given te regression functions, we obtain te regression residuals Our rst attempt at an estimator for is ten ^e yi = y i ^g y (Z i ) ^e xi = X i ^g x (Z i ) ^ =! ^e xi^e xi ^e xi^e yi 7.9 Trimming Te asymptotic teory for semiparametric estimators, typically requires tat te rst step estimator converges uniformly at some rate. Te di culty is tat ^g y () and ^g x () do not converge uniformly over unbounded sets. Equivalently, te problem is due to te estimated density of Z i in te denominator. Anoter way of viewing te problem is tat tese estimates are quite noisy in sparse regions of te sample space, so residuals in suc regions are noisy, and tis could unduely in uence te estimate of : su er a problem tat tey can be unduly in uence by unstable residuals from observations in sparse regions of te sample space. Te nonparametric regression estimates depend inversely on te density estimate ^f () = n Zi k For values of were f () is close to ero, ^f () is not bounded away from ero, so te NW estimates at tis point can be poor. Consequently te residuals for observations i suc tat f (Z i ) will be quite unreliable, and can ave an undue in uence on ^: A standard solution is to introduce trimming. Let ^f () = n Zi k let b > be a trimming constant and let i (b) = ^f (Z i ) b tose observations for wic te estimated density of Z i is above b: : ; denote a indicator variable for 64

10 Te trimmed version of ^ is ^ =! ^e xi^e xi i (b) ^e xi^e yi i (b) Tis is a trimmed LS residual regression. Te asymptotic teory requires tat b = b n! but unfortunately tere is not good guidance about ow to select b in practice. Often trimming is ignored in applications. One practical suggestion is to estimate wit and witout trimming to assess robustness. 7. Asymptotic Distribution Robinson (988), Andrews (994) and Li (996) are references. Te needed regularity conditions are tat te data are iid, Z i as a density, and te regression functions, density, and conditional variance function are su ciently smoot wit respect to teir arguments. Assuming a second-order kernel, and for simplicity writing = = = q, te important condition on te bandwidts sequence is p n 4 + n q! Tecnically, tis is not quite enoug, as tis ignores te interaction wit te trimming parameter b: But since tis can be set b n! at an extremely slow rate, it can be safely ignored. Te above condition is similar to te standard convergence rates for nonparametric estimation, multiplied by p n: Equivalently, wat is essential is tat te uniform MSE of te nonparametric estimators converge faster tan p n; or tat te estimators temselves converges faster tan n =4 : Tat is, wat we need is n =4 sup j^g y () g y ()j! p n =4 sup j^g x () g x ()j! p From te teory for nonparametric regression, tese rates old wen bandwidts are picked optimally and q 3: In practice, q 3 is probably su cient. If q > 3 is desired, ten iger-order kernels can be used to improve te rate of convergence. So long as te rate is faster tan n =4 ; te following result applies. Teorem (Robinson). Under regularity conditions, including q 3; te trimmed estimator satis es p n ^! d N (; V ) V = E e xi e xi E exi e xi 2 (X i ; Z i ) E exi e xi Tat is, ^ is asymptotically equivalent to te infeasible estimator ~ : Te variance matrix may be estimated using conventional LS metods. 65

11 7. Veri cation of Andrews MINPIN Condition Tis Teorem states tat Robinson s two-step estimator for is asymptotically equivalent to te infeasible one-step estimator. Tis is an example of te application of Andrews MINPIN teory. Andrews speci cally mentions tat te n =4 convergence rates for ^g y () and ^g x () are essential to obtain tis result. To see tis, note tat te estimator ^ solves te FOC n (X i ^g x (Z i )) y i ^g y (Z i ) ^ (Xi ^g x (Z i )) = In Andrews MINPIN notation, let x = ^g x and y regression estimates, ten Since = ^g y denote xed (function) values of te m i ( ; ) = (X i x (Z i )) y i y (Z i ) (X i x (Z i )) y i = g y (Z i ) + (X i g x (Z i )) + e i ten E y i y (Z i ) (X i x (Z i )) j X i ; Z i = gy (Z i ) y (Z i ) (g x (Z i ) x (Z i ) +) and m ( ; ) = Em i ( ; ) = EE (m i ( ; ) j X i ; Z i ) = E (X i x (Z i )) g y (Z i ) y (Z i ) (g x (Z i ) x (Z i )) = E (g x (Z i ) x (Z i )) g y (Z i ) y (Z i ) (g x (Z i ) x (Z i )) Z = (g x () x ()) g y () y () (g x () x ()) f ()d were te second-to-last line uses conditional expecations given X i. Ten replacing x wit ^g x and y wit ^g y Z p nm ( ; ^) = (g x () ^g x ()) g y () ^g y () (g x () ^g x ()) f ()d Taking bounds p nm ( ; ^) sup jg x () ^g x ()j sup jg y () ^g y ()j + sup jg x () ^g x ()j 2 sup jf y ()j! p wen te nonparametric regression estimates converge faster tan n =4 : 66

12 Indeed, we see tat te o(n =4 ) convergence rates imply te key condition p nm ( ; ^)! p : 7.2 Estimation of Nonparametric Component Recall tat te model is y i = Xi + g (Z i ) + e i and te goal is to estimate and g: We ave described Robinson s estimator for : We now discuss estimation of g: Since ^ converges at rate n =2 wic is faster tan a nonparametric rate, we can simply pretend tat is known, and do nonparametric regression of y i Xi ^ on Z i ^g () = P n k Zi y i Xi ^ P n k Zi Te bandwidt = ( ; :::; q ) is distinct from tose for te rst-stage regressions. It is not ard to see tat tis estimator is asymptotically equivalent to te infeasible regressor wen ^ is replaced wit te true : Standard errors for ^g(x) may be computed as for standard nonparametric regression. 7.3 Bandwidt Selection In a semiparametric context, it is important to study te e ect a bandwidt as on te performance of te estimator of interest before determining te bandwidt. In many cases, tis requires a nonconventional bandwidt rate. However, tis problem does not occur in te partially linear model. Te rst-step bandwidts used for ^g y () and ^g x () are inputs for calculation of ^: Te goal is presumably accurate estimation of : Te bandwit impacts te teory for ^ troug te uniform convergence rates for ^g y () and ^g x () suggesting tat we use conventional bandwidt rules, e.g. cross-validation. 67