5. NONLINEAR MODELS [1] Nonlinear (NL) Regression Models

Transcription

1 5. NONLINEAR MODELS [1] Nonlinear (NL) Regression Models General form of nonlinear or linear regression models: y = h(x,β) + ε, ε iid N(0,σ ). Assume ha he x and ε sochasically independen. his assumpion implies ha: hx (, β ) E ε = 0 β i ; k 1 hx (, β ) E ε = 0 β β i k k. Example 1: y x = β + ε. Example : = + +ε, where x is a scalar. 3x y β β e β 1 Example 3: y Ax x β β 3 = + ε. (*) 3 NLLS-1

2 [] Esimaion of NL models Definiion: Le S( β ) =Σ ( y h( x, β )). he NLLS (Nonlinear leas squares) esimaor, ˆNL β, minimizes S(β). Assumpions: See Greene, Chaper 9. Facs: 1) p ˆ β β lim NL o = (consisen). [No guaranee ha E ( ) ˆNL β = β.] o ) ˆ 1 β β N p σ Σ H β H β ( NL o) d 0 k 1, lim ( o) ( o) where H ( β ) = h( x, β)/ β., If we define H ( β ) H1( β ) H ( β ), H( β ) H( β) = Σ H ( β) H ( β). = : H 3) Le s = S( ˆ β )/, hen, NL NL ( β ) ( ) 4) ˆ β, ( ˆ NL N β s NL H βnl ) H( ˆ βnl ). plim s NL = σ. 5) R = 1 - SSE/SS, where SSE = S( ˆ β NL ). here is no guaranee ha 0 R 1. NLLS-

3 Example: y hen, 3x = β + β e β + ε. 1 hx (, β ) hx (, β) β x hx (, β) = 1; = e ; = βxe β β β 1 β3x hus, H ( β ) = e. β3x βxe 1 3 β x 3 3. <Skechy Proof of Consisency> From S( ˆ βnl)/ β = 0k 1, Σ H ( ˆ β )( y h( ˆ β )) = 0. = 1 NL NL k 1 aylor expansiion around β = β o : 0 Σ k 1 H( βo)( y h( βo)) h( βo) +Σ ˆ y h o H o H o NL o. β β h( βo) ( ) ( ) ( ( β )) ( β ) ( β ) ( β β ) ˆ βnl βo Σ y h ( βo) H ( βo) H( βo) β β Σ H ( β )( y h( β )). o o NLLS-3

4 1 h( βo) ( ) ( ) ˆ βnl βo Σ y h ( βo) H ( βo) H ( βo) β β 1 ΣH( βo)( y h( βo)) 1 h( βo) 1 ε H βo H β o H βo Σ ( ) ( ) Σ ( ) ε β β 1 Σ H( βo) H( βo) 0k 1 = 0 k 1. <Skechy Proof of Asympoic Normaliy> Noe ha 1 h( βo) ( ) ( ) ˆ βnl βo Σ y h ( βo) H ( βo) H ( βo) β β 1 ΣH( βo)( y h( βo)) 1 1 Σ H( βo) H( βo) Σ H( βo)( y h( βo)). Bu 1 1 ΣH( βo)( y h( βo)) = ΣH( βo) ε 1 d N 0 k 1, plim σ Σ H( βo) H( βo). NLLS-4

5 [3] SPECIFICAION ESS CASE A: Same dependen variables under boh he null and he alernaives H o : y = h o (x,β) + ε. H a : y = h a (w,γ) + ε. (A) (B) Example: h o (x,β) = β 1 + β x ; h a (w,γ) = γ 1 + γ ln(x ). [Even if he models are linear, we sill can perform J or P ess.] (1) J es: Davidson and Mackinnon (1981, Economerica) Consruc he following auxiliary model: y = (1-α)h o (x,β) + αh a (w,γ) + ε. If H o is correc, α = 0. Le ˆNL γ be he NLLS esimaor of γ from (B). Replace h a (w,γ) (C) by ˆ a a h = h ( w, ˆ ) (fied value of y from (B)): γ NL o ˆa y = (1 α) h ( x, β) + αh + error. (D) Do NLLS on (D), and esimae β and α joinly. Using he esimaes, we can perform a -es for H o : α = 0. [In he sense ha we esimae β and α joinly, we call he es J-es.] NLLS-5

6 () P-es: An alernaive o J-es. Ge ˆNL β and ˆNL γ by NLLS on boh (A) and (B). Consider he following auxiliary regression: ˆo ˆ o ( ˆa ˆo y h = H b + h h ) α + error, (E) where hˆo o h (, ˆ = x β NL) ˆa a, h = h ( w, ˆ γ NL) and Hˆ = H ( x, ˆ β. o o ) NL Do OLS on (E) and esimae b and α. hen, perform -es for H o : α = 0. CASE B: Differen dependen variables H o : y = h o (x,β) + ε. H a : g(y ) = h a (w,γ) + ε, where g(y ) is a funcion of y (e.g., g(y ) = ln(y )). Example: H o : y = β 1 + β x + ε H a : ln(y ) = γ 1 + γ ln(x ) + ε NLLS-6

7 P es: Esimae boh β and γ by NLLS (or OLS). Consruc he following auxiliary model: ˆo ˆ o ˆa ( ( ˆo y h = H b+ h g h )) α + error. (F) Do OLS on (F), and es H o : α = 0. Example: H o : y = x β + ε H a : ln(y ) = w γ + ε, where x = [1,x,...,x k ] and w = [1,ln(x ),...,ln(x k )]. Explain in deail how you would es H o. NLLS-7

8 [EXAMPLE] Daa: (WAGE.WF1 or WAGE.X from Wooldridge s websie) # of observaions (): wage monhly earnings. hours average weekly hours 3. IQ IQ score 4. KWW knowledge of world work score 5. educ years of educaion 6. exper years of work experience 7. enure years wih curren employer 8. age age in years 9. married =1 if married 10. black =1 if black 11. souh =1 if live in souh 1. urban =1 if live in SMSA 13. sibs number of siblings 14. brhord birh order 15. meduc moher's educaion 16. feduc faher's educaion 17. lwage naural log of wage H o : lwage = β exp 1 + βeduc + β3 er +ε. γ 3 γ 4 H a : lwage 1 ( educ exper ) = γ + γ + + ε. NLLS-8

9 <J-es> = SEP 1 = Esimae he model under H a : Dependen Variable: LWAGE Mehod: Leas Squares Sample: Included observaions: 935 Esimaion seings: ol= , derivs=analyic Iniial Values: C(1)= , C()= , C(3)= , C(4)= Convergence achieved afer 80 ieraions LWAGE=C(1)+C()*(EDUC^C(3)+EXPER^C(4)) Coefficien Sd. Error -Saisic Prob. C(1) C() C(3) C(4) R-squared Mean dependen var Adjused R-squared S.D. dependen var S.E. of regression Akaike info crierion Sum squared resid Schwarz crierion Log likelihood Durbin-Wason sa Ge fiya = lwage - resid NLLS-9

10 = SEP = Esimae lwage = (1 α)( β1+ βeduc + β3exer) + α fiya + error. Dependen Variable: LWAGE Mehod: Leas Squares Dae: 04/09/0 ime: 1:6 Sample: Included observaions: 935 Esimaion seings: ol= , derivs=analyic Iniial Values: C(1)= , C()= , C(3)=1.141, C(4)= Convergence achieved afer ieraions LWAGE=(1-C(1))*(C()+C(3)*EDUC+C(4)*EXPER)+C(1)*FIYA Coefficien Sd. Error -Saisic Prob. C(1) C() C(3) C(4) R-squared Mean dependen var Adjused R-squared S.D. dependen var S.E. of regression Akaike info crierion Sum squared resid Schwarz crierion Log likelihood Durbin-Wason sa Do no rejec H o : C(1) = 0 (α = 0). NLLS-10

11 <P-es> = SEP 1 = Esimae he model under H o : Dependen Variable: LWAGE Mehod: Leas Squares Sample: Included observaions: 935 LWAGE=C(1)+C()*EDUC+C(3)*EXPER Coefficien Sd. Error -Saisic Prob. C(1) C() C(3) R-squared Mean dependen var Adjused R-squared S.D. dependen var S.E. of regression Akaike info crierion Sum squared resid Schwarz crierion Log likelihood Durbin-Wason sa Ge fiy0 = lwage resid and res0 = resid. NLLS-11

12 = SEP = Esimae he model under H a : Dependen Variable: LWAGE Mehod: Leas Squares Sample: Included observaions: 935 Esimaion seings: ol= , derivs=analyic Iniial Values: C(1)= , C()= , C(3)= , C(4)= Convergence achieved afer 80 ieraions LWAGE=C(1)+C()*(EDUC^C(3)+EXPER^C(4)) Coefficien Sd. Error -Saisic Prob. C(1) C() C(3) C(4) R-squared Mean dependen var Adjused R-squared S.D. dependen var S.E. of regression Akaike info crierion Sum squared resid Schwarz crierion Log likelihood Durbin-Wason sa Ge fiya = lwage resid. NLLS-1

13 = SEP 3 = Esimae res0= β1 + βeduc + β3exp er + α( fiya fiy0). Dependen Variable: RES0 Mehod: Leas Squares Sample: Included observaions: 935 RES0=C(1)+C()*EDUC+C(3)*EXPER+C(4)*(FIYA-FIY0) Coefficien Sd. Error -Saisic Prob. C(1) C() C(3) 9.34E C(4) R-squared Mean dependen var 3.38E-15 Adjused R-squared S.D. dependen var S.E. of regression Akaike info crierion Sum squared resid Schwarz crierion Log likelihood Durbin-Wason sa Do no rejec H o a 5% of significance level. NLLS-13

14 [4] NL models wih unknown parameers in LHS General form: g(y,δ) = h(x,β) + ε, ε iid N(0,σ ). Example: Generalized Cobb-Douglas Funcion ln(y ) + δy = β 1 + β (1-β 3 )ln(k ) + β β 3 ln(l ) + ε. If δ = 0, he funcion becomes Cobb-Douglas. Esimaion: 1) NLLS could be inconsisen. Even if i is consisen, compuaion of he covariance marix of he NLLS esimaor could be complicaed. ) Do MLE. (See Greene.) Example: Box-Cox ransformaion Define: y (δ) = [y δ - 1]/δ ; x j (λ) = [x λ j - 1]/λ, j =,..., k. Assume ha λ is all he same for j, we can allow λ o vary over differen j. If λ = 1, x j (1) = x j - 1 (linear). If λ 0, x j (λ) ln(x j ) (log). NLLS-14

15 Box-Cox Model: k y 1 j= xj ( δ ) = β +Σ ( λ) β +ε. j y (δ) = x (λ) β + ε, where x (λ) = [1,x (λ),...,x k (λ)] and β = [β 1,...,β k ]. Use MLE o esimae δ, λ and β. NLLS-15