Lecure 0 Esimaing Nonlinear Regression Models References: Greene, Economeric Analysis, Chaper 0
Consider he following regression model: y = f(x, β) + ε =,, x is kx for each, β is an rxconsan vecor, ε is an unobservable error process and f is a ( sufficienly well-behaved ) funcion - f: R k xr r R. So, each y is a (fixed) funcion of x and β plus an addiive error erm, ε. Example: y β x β 2 = + ε he esimaion problem: given f, y,,y, and x,,x, esimae β. he soluion: esimae β by LS (NLS), ML, or GMM. he problem? In conras o he linear regression case, he FOCs are nonlinear and so, in general, numerical mehods mus be applied o obain (consisen) poin esimaes. Also, he avar marice of β-ha will have a slighly more complicaed form. Nonlinear models are commonly encounered in applied economics largely because advances in compuaional mahemaics and deskop/lapop compuer echnology have made solving nonlinear opimizaion problems more feasible and more reliable.
Nonlinear Leas Squares (NLS) Choose β-ha o minimize he SSR FOCs SSR( ˆ) β = ( y f (, ˆ)) β = g ( ˆ) β = ( y x 2 f ( x, ˆ)) β f ( x = β, ˆ) β = 0 which form a se of r nonlinear equaions in he r unknowns, ˆ,..., β ˆ β r. [In he case where f is linear in he β s, he derivaive vecor df/dβ = [ x x r ], r = k.] Example: y = 2 β β + ε x g ( ˆ) β = = ( y ˆ β x ˆ β ˆ ˆ 2 β2 )[ ˆ ˆ β2 x β β x ]' = 2 0
Compuing he NLS Esimaor In general, hese FOCs mus be solved numerically o find he NLS esimaor of β, βˆnls. (E.g., he Gauss- Newon procedure described in Greene, 0.2.3.) Some issues - choice of algorihm - selecing an iniial value for β-ha - convergence crieria - local vs. global min
Asympoic Properies of he NLS Esimaor If he x s are weakly exogenous he errors are serially uncorrelaed and homoskedasic he funcion f is sufficienly smooh he {x,ε } process is sufficienly well-behaved hen ( ˆ β 2 β ) N(0, σ Q / 2, NLS D where σ 2 = var(ε ) Q = plim(/ ) Q = [ f ( x = Q, ˆ β ) / β ][ f ( x, ), ˆ β ) / β '] he NLS esimaor is (under appropriae condiions), consisen, asympoically normal and asympoically efficien. Inference: For large samples ac as hough ˆ 2 ~ (, ˆ NLS N β σ Q β 2 2 σˆ = (/ ) ˆ ε )
If he disurbances are heeroskedasic and/or serially correlaed he NLS esimaor will be consisen bu no asympoically efficien. Also, he correc form of he asympoic variance marix of he NLS esimaor requires a heeroskedasiciy and/or auocorrelaion correcion. Heeroskedasiciy and HAC esimaors of he variance-covariance marix of ε can be used if he exac forms of he heeroskedasiciy and auorcorrelaion are no know. If he form of he heeroskedasiciy and/or serial correcion is known up o a small number of parameers (e.g., ε is known o be an AR() process wih unknown ρ) hen nonlinear GLS or (quasi)- maximum likelihood will be asympoically efficien esimaors. Example GNLS Suppose E(εε ) = Σ. hen he GNLS esimaor of β is he value of βˆ ha minimizes he weighed SSR: [y-f(x, βˆ )] Σ - [y-f(x, βˆ )] If Σ hen i can be replaced wih a consisen esimaor o obain he FGNLS esimaor. (Wha consisen esimaor of Σ?)
If he regressors are correlaed wih he errors, none of hese esimaors is consisen (even if he errors are homoskedasic and serially uncorrelaed). A consisen, semi-paramerically efficien esimaor ha does no rely on knowledge of he form/exisence of heeroskedasiciy/auocorrelaion and allows for endogenous regressors: Nonlinear GMM In addiion, GMM provides a semi-parameric alernaive o MLE for nonlinear models ha do no fi he nonlinear regression forma.
GMM in he nonlinear regression model Consider he populaion momen condiions: E[w (y f(x,β))] = 0 for all where w is an insrumen vecor. he GMM esimaor: choose βˆ corresponding sample momens o make he w ( y f ( x ˆ, β )) close o zero. As in he linear case, his will involve minimizing an opimally weighed quadraic form in hese momens.
GMM in a more general nonlinear seing - Hansen and Singleon s (Economerica, 982) Consumpion-Based Asse Pricing Model A he sar of each ime period, a represenaive agen chooses consumpion and saving o maximize expeced discouned uiliy: i c E[ δ U ( c+ i ) Ω ], ( c ) = γ i= 0 γ U, 0 < γ < A he sar of, he agen can allocae income o purchase he consumpion good or N asses wih mauriies,2,,n according o he sequence of budge consrains c N N + = p, q, = = r, q +, where p, = price of a uni of asse (i.e., an asse ha maures in +) in period q, = unis of asse purchased in period r, = payoff in period of asse purchased in - w = labor income in period Unknown parameers in his model- δ,γ w
he opimal consumpion pah mus saisfy he sequence of Euler equaions: E[ δ ( r γ, + / p, )( c+ / c ) Ω ] = 0 Le z be any vecor in Ω. hen he Euler equaions imply he following se of momen condiions which form he basis for esimaing δ and γ by GMM E[ δ ( r γ, + / p, )( c + / c ) z ] = 0 for all and =,...N GMM: Choose δ and γ o make he sample momens δ ( r, + / p, )( c + / c ) γ z, =,,N close o zero. he alernaive MLE: specify he oin disribuion for {(r,+ /p,+,c + /c ), =,2,,N} hen maximize he corresponding likelihood funcion subec o he Euler equaions. (See, e.g., Hansen and Singleon, JPE, 983).