Lecture notes to Stock and Watson chapter 8

Transcription

1 Lecture notes to Stock and Watson chapter 8 Nonlinear regression Tore Schweder September 29 TS () LN7 9/9 1 / 2

2 Example: TestScore Income relation, linear or nonlinear? TS () LN7 9/9 2 / 2

3 General problem Fully nonlinear regression: Y = f (X 1, X 2,, X k ; β) + u for given parameter vector β f (x 1,, x k ; β) is a speci c (nonlinear) function of (x 1,, x k ). Estimate β from data (Y i, X i1,, X ik ) i = 1,, n (assuming the sample being random, the random terms u i having mean zero, and having nite fourth moment) E ect calculation: Y = f (X 1 + X 1, X 2,, X k ; β) f (X 1, X 2,, X k ; β) d Y = f (X 1 + X 1, X 2,, X k ; bβ) f (X 1, X 2,, X k ; bβ) TS () LN7 9/9 3 / 2

4 Simpli ed method, only one X linear regression of one or more nonlinear functions of X on Y Y = β + β 1 X + β 2 f 2 (X ) + + β k f k (X ) + u Standard assumptions for a sample (Y i, X i ) i = 1,, n Estimate β, β 1, β k by linear regression of (X 1 = X, X 2 = f 2 (X ),, X k = f k (X )) on Y What is required of the functions f j j = 2,, k to avoid perfect collinearity? TS () LN7 9/9 4 / 2

5 Polynomial regression, Example TestScore = β + β 1 Income + β 2 (Income) 2 + u generate avginc2 = avginc*avginc; reg testscr avginc avginc2, r; Create a new regressor Regression with robust standard errors Number of obs = 42 F( 2, 417) = Prob > F =. R squared =.5562 Root MSE = Robust testscr Coef. Std. Err. t P> t [95% Conf. Interval] + avginc avginc _cons TS () LN7 9/9 5 / 2

6 Reasonable to expect an optimal District income, and declining Test scores for richer districts? Cubic regression?, linear-log regression? TS () LN7 9/9 6 / 2

7 Example: linear-log regression. gen lnincome=ln(avginc). reg testscr lnincome,r Linear regression Number of obs = 42 F( 1, 418) = Prob > F =. R squared =.5625 Root MSE = Robust testscr Coef. Std. Err. t P>t [95% Conf. Interval] lnincome _cons TS () LN7 9/9 7 / 2

8 TS () LN7 9/9 8 / 2

9 Log-log regression ln(y ) = β + β 1 ln(x ) + u Y = e β X β 1e u β 1 is the expected elasticity, cβ 1 is the estimated elasticity E ect estimates on log scale: cβ 1 = d dy ln(y )/ d dx ln(x ) E ect estimates on original scale: Y = cβ 1 Y X X The estimated elasticity of Test score on District income is cβ 1 =.55 TS () LN7 9/9 9 / 2

10 E ect calculations for Test score by District income Expected increase in Test score by one unit increase in average district income (1$) quadratic cubic linear-log log-log TS () LN7 9/9 1 / 2

11 Which model to choose? The model must allow estimation of the desired quantity log-log is good if you are desperate for an invariant elasticity It must t the data It should not be more complex than necessary If only t and complexity matters, use AIC or BIC as information criteria STAT command estat ic TS () LN7 9/9 11 / 2

12 Example Test score District income After each regression, ask for estat ic. Low value of AIC and BIC is good! Only models with the response testscr on the same scale can be compared. Response variable Y = testscr Model AIC BIC linear quadratic cubic log The linear-log model is preferred! BIC penalize complexity more than AIC! TS () LN7 9/9 12 / 2

13 Dummy variables and interactions. gen HiEL= el_pct>1. gen lninc_hiel= lnincome*hiel. reg testscr lnincome HiEL lninc_hiel,r Linear regression Number of obs = 42 F( 3, 416) = Prob > F =. R squared =.6918 Root MSE = Robust testscr Coef. Std. Err. t P>t [95% Conf. Interval] lnincome HiEL lninc_hiel _cons AIC=318.2 TS () LN7 9/9 13 / 2

14 Residuals Residual plot predict r, residuals twoway (scatter r avginc), by(hiel) Graphs by HiEL avginc Figure: Residuals from the linear-log regression with interaction: testscr = β + β 1 log(avginc) + β 2 HiEL + β 3 HiEL log(avginc) TS () LN7 9/9 14 / 2

15 Problem in polynomial regression - near collinearity? Example polynomial regression of degree 4: X i = i/25 i = 1,, 25 = n. k = X^ 1 X^ 1 X^ 1 X^ X^ 2 X^ 2 X^ 2 X^ X^ 3 X^ 3 X^ 3 X^ X^ 4 X^ 4 X^ 4 X^ 4 Figure: plots of X i vr. X j Correlation given on top of each plot. TS () LN7 9/9 15 / 2

16 Remedy: work with orthogonal polynomials! Example cont Let 2 = X 2 c X 2 where c X 2 is the tted values from the regression X 2 = α 2 + α 12 X + u 2, 3 = X 3 X c3 where X c3 is the tted values from the regression X 3 = α 3 + α 13 X + α 13 X 2 + u 3 etc. TS () LN7 9/9 16 / 2

17 Figure: Orthogonal covariates plotted against each other, correlations over each graph. TS () LN7 9/9 17 / 2

18 Making the orthogonal variables for District income. reg avginc2 avginc,r. predict OInc2, residuals. reg avginc3 avginc avginc2,r. predict OInc3, residuals. corr avginc avginc2 avginc3 (obs=42) avginc avginc2 avginc3 avginc 1. avginc avginc corr avginc OInc2 OInc3 (obs=42) avginc OInc2 OInc3 avginc 1. OInc2. 1. OInc TS () LN7 9/9 18 / 2

19 Comparing direct polynomial regression and equivalent polynomial regression on orthogonal variables. reg testscr avginc avginc2 avginc3,r Linear regression Number of obs = 42 R squared =.5584 Root MSE = Robust testscr Coef. Std. Err. t P>t [95% Conf. Interval] avginc avginc avginc e _cons reg testscr avginc OInc2 OInc3,r Linear regression Number of obs = 42 R squared =.5584 Root MSE = Robust testscr Coef. Std. Err. t P>t [95% Conf. Interval] avginc OInc OInc e _cons TS () LN7 9/9 19 / 2

20 Do SW: 7.8 TS () LN7 9/9 2 / 2