4. Nonlinear regression functions

Transcription

1 4. Nonlinear regression functions Up to now: Population regression function was assumed to be linear The slope(s) of the population regression function is (are) constant The effect on Y of a unit-change in the regressor X j (j = 1,..., k) does not depend on the value of X j Now: Two groups of methods for detecting and modeling nonlinear population regression functions 72

2 Group #1: Effect on Y of a change in one regressor, say X 1, depends on the value of X 1 itself Example: reducing class size by one student per teacher (that is a change in STR) might have a larger effect on TEST SCORE when class sizes are already managebly small Group #2: Effect on Y of a change in one regressor, say X 1, depends on the value of another regressor, say X 2 Example: students still learning English might benefit from having more one-on-one attention Effect on TEST SCORE of reducing STR is greater in districts with higher values of PCTEL 73

3 Population regression functions with different slopes Constant slope Slope depends on the value of X1 Y Y X1 X1 Slope depends on the value of X2 Population regression function when X2=1 Y Population regression function when X2=0 X1 74

4 4.1. A general strategy for modeling nonlinear regression functions Empirical example: Consider the student-performance dataset Generally, we would expect that the economic background of the students might have an impact on TEST SCORES ( rich students perform better than poor students) The economic background is measured by the variable AVGINC (average per capita income in the school district in thousands 1998 US-dollars) 75

5 Test score vs. district income with a linear OLS regression function Test score District income (thousands of dollars) 76

6 Scatterplot characteristics: The variable AVGINC and TEST SCORE are highly correlated (correlation coefficient: 0.71) For incomes below US-$ or above US-$ the points are below the OLS-line For incomes between US-$ and US-$ the points are above the OLS-line nonlinear relationship between TEST SCORE and AVGINC Possibly: Quadratic relationship between both variables: TEST SCORE i = β 0 + β 1 AVGINC i + β 2 AVGINC 2 i + u i (4.1) (quadratic regression model) 77

7 Estimation of model (4.1): Eq. (4.1) is a variant of the multiple regression model with k = 2 and Y i = β 0 + β 1 X 1i β k X ki + u i X 1i = AVGINC i X 2i = AVGINC 2 i OLS estimation technique is applicable We can test the null hypothesis that the population regression function is linear versus the alternative that it is quadratic by conducting the test H 0 : β 2 = 0 vs. H 1 : β 2 0 on the basis of the conventional t-statictic 78

8 OLS estimation results of the quadratic model (4.1) Dependent Variable: TEST_SCORE Method: Least Squares Date: 10/04/12 Time: 18:31 Sample: Included observations: 420 White heteroskedasticity-consistent standard errors & covariance Variable Coefficient Std. Error t-statistic Prob. C AVGINC AVGINC_SQ R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid Schwarz criterion Log likelihood Hannan-Quinn criter F-statistic Durbin-Watson stat Prob(F-statistic)

9 Test scores vs. district income with a quadratic OLS regression function Test score District income (thousands of dollars) 80

10 Obviously: β 2 is significantly different from zero at all conventional levels quadratic model fits the data better than the linear model Next: Consider the general nonlinear regression model Y i = f(x 1i, X 2i,..., X ki ) + u i (i = 1,..., n) (4.2) where f(x 1i, X 2i,..., X ki ) is a general nonlinear population regression function Under the OLS assumptions on Slide 18 we have E(Y i X 1i, X 2i,..., X ki ) = f(x 1i, X 2i,..., X ki ) 81

11 Question: What is the expected effect on Y of a change in one regressor, say of a change X j in X j (j = 1,..., k)? Answer: The expected change in Y, Y, associated with a change in the regressor X j, X j, holding all other regressors constant, is the difference between the value of the population regression function before and after changing X j, holding all other regressors constant: Y = f(x 1,..., X j + X j,..., X k ) f(x 1,..., X j,..., X k ) (4.3) 82

12 Remarks: Note that the specific parametric form of f(x 1, X 2,..., X k ) is unknown f(x 1, X 2,..., X k ) contains unknown parameters that have to estimated from the data Let ˆf(X 1, X 2,..., X k ) denote the predicted value of Y based on the estimator ˆf of the population regression function Then, the predicted change in Y is Ŷ = ˆf(X 1,..., X j + X j,..., X k ) ˆf(X 1,..., X j,..., X k ) (4.4) 83

13 Example: Consider the quadratic OLS regression of TEST SCORE on AVG- INC and AVGINC 2 on Slide 80 with the estimated coefficients ˆβ 0 = , ˆβ 1 = , ˆβ 2 = An increase in district income from 10 to 11 (i.e. from US-$ per capita to US-$) yields the estimated effect Ŷ = (ˆβ 0 + ˆβ ˆβ ) (ˆβ 0 + ˆβ ˆβ ) = An increase in district income from 40 to 41 (i.e. from US-$ per capita to US-$) yields the estimated effect Ŷ = (ˆβ 0 + ˆβ ˆβ ) (ˆβ 0 + ˆβ ˆβ ) =

14 Example: [continued] Obviously, a change of income of 1000 US-$ is associated with a larger change in predicted test scores if the initial income is low (10000 US-$) than if it is high (40000 US-$) The predicted changes are points versus point Remarks: The estimator Ŷ of the effect on Y of changing the regressor X j depends on the estimator of the population regression function, ˆf, which varies from one sample to the next Ŷ contains sampling error There are several techniques for computing the standard error SE( Ŷ ) (see Stock & Watson, 2011, pp. 302, 303) 85

15 Strategy for modeling nonlinear regressions: 1. Identify a possible nonlinear relationship (use economic theory and general knowledge) 2. Specify a nonlinear function and estimate parameters by OLS (see next section for various nonlinear functions) 3. Check if the nonlinear model improves upon a linear model (use t- and F -statistics) 4. Plot the estimated nonlinear regression function 5. Estimate the effect on Y of a change in the regressor X j 86

16 4.2. Nonlinear functions of a single regressor Outline: Description of most important nonlinear regression functions (polynomials and logarithms) We restrict attention to regressions with a single regressor Extensions to multiple regressors are straightforward We treat the alternative nonlinear functions separately, although it is unproblematic to combine them in one regression function 87

17 Polynomials Definition 4.1: (Polynomial regression model) We define the general polynomial regression model of degree r as Y i = β 0 + β 1 X i + β 2 X 2 i β r X r i + u i. (4.5) When r = 2 or r = 3, we call Eq. (4.5) the quadratic or the cubic regression model, respectively. Remarks: We interpret X i, X 2 i,..., Xr i as the r distinct regressors X 1i, X 2i,..., X ri We estimate the parameters β 0, β 1,..., β r via OLS by regressing Y i against X i, X 2 i,..., Xr i 88

18 Test of linear versus polynomial specification: If the true regression is linear, then the terms X 2 i, X3 i,..., Xr i do not enter the population regression function (4.5) Hypothesis test: H 0 : Regression is linear vs. H 1 : Regression is polynomial of degree r In probabilistic terms: H 0 : β 2 = 0, β 3 = 0,..., β r = 0 vs. H 1 : at least one β j 0 (j = 2..., r) Use the F -testing strategy as decribed in Section 3.4 to test this specification issue 89

19 Which polynomial degree? Trade-off between (1) flexibility in the shape of the regression function and (2) the precision of estimated coefficients Include just enough polynomial terms to model the nonlinear regression function adequately, but no more Sequential testing procedure: 1. Pick a maximum value of r and perform OLS estimation (try a maximal r = 4) 2. Use the t-statistic to test H 0 : β r = 0 vs. H 1 : β r 0 3. If you reject H 0 in Step 2 use the polynomial of degree r and stop the procedure 90

20 Sequential testing procedure: [continued] 4. If you do not reject H 0 in Step 2, eliminate the Xi r -term from the regression and estimate a polynomial of degree r 1. Use the t-statistic to test H 0 : β r 1 = 0 vs. H 1 : β r 1 = 0 5. If you reject H 0 in Step 4, use the polynomial of degree r 1 and stop the procedure 6. If you do not reject H 0 in Step 4, continue this procedure until the coefficient on the highest power in your polynomial is statistically significant 91

21 Logarithms Natural logarithm [ln(x)]: ln(x) is the most important nonlinear function in economics Logs convert changes in variables into percentage changes Logs and pecentages: Consider a variable x and a small change in x, x The percentage change in x is given by 100 ( x/x) The following approximation holds: ln(x + x) ln(x) x x (difference in logs approximates the percentage change devided by 100) 92

22 Definition 4.2: (Logarithmic regression models) We consider the following three types of regression models: Y i = β 0 + β 1 ln(x i ) + u i, (4.6) ln(y i ) = β 0 + β 1 X i + u i, (4.7) ln(y i ) = β 0 + β 1 ln(x i ) + u i. (4.8) We refer to the models (4.6) (4.8) as the linear-log, the loglinear and the log-log model, respectively. Remarks: The regression models (4.6) (4.8) are conventional regression models with a single regressor OLS estimation technique applies (provided that the OLS assumptions are satisfied) 93

23 Remarks: [continued] The three models (4.6) (4.8) differ in their interpretation of the coefficient β 1 Interpretation of β 1 : In the linear-log model (4.6) a 1%-change in X is associated with a change in Y of 0.01β 1 In the log-linear model (4.7) a change in X by one unit ( X = 1) is associated with a 100β 1 %-change in Y In the log-log model (4.8) a 1%-change in X is associated with a β 1 % change in Y, that is β 1 is the elasticity of Y with respect to X (see class for details) 94

24 Remarks: Which of the log regression models best fits the data? Only the log-linear and the log-log models (4.7) and (4.8) can be compared via their R 2 values The linear-log model (4.6) cannot be compared with the other log models via the R 2 values since the dependent variables are different (Y i vs. ln(y i )) Use economic theory and other expert knowledge of the specific data problem at hand to decide whether it makes sense to specify Y in logarithms 95

25 4.3. Interactions between regressors Up to now: Nonlinear relationship between Y and the regressor X depends on the values of the regressor X itself Now: The effect on Y of a change in one regressor, say X 1, depends on the value of another regressor, say X 2 Interactions between the regressors 96

26 Interactions between two dummy regressors Definition 4.3: (Dummy variables) We consider a potential regressor that may indicate the presence or absence of a qualitative characteristic or an attribute (such as male or female, catholic or non-catholic and so forth). We quantify such attributes by constructing artificial variables of the form D i = { 1 if the attribute is present for the i th observation 0 if the attribute is not present for the i th observation for i = 1,..., n. We call variables like D i dummy variables (or binary or indicator variables). 97

27 Remarks: We have already made use of dummy variables on the Slides Dummies are essentially nominal scale (qualitative) variables that have been quantified Note that a dummy variable can only assume the two values 0 and 1 Dummy regressors as specified in Definition 4.3 can be incorporated in regression models just as easily as any other quantitative (continuous) regressor 98

28 Consider the following empirical problem: Assume you have a data set containing the dependent variable (log) earnings, that is Y i = ln(earnings i ) (i = 1,..., n) and the two dummy variables D 1i = D 2i = { 1 if the i th worker has a college degree 0 otherwise { 1 if the i th worker is female 0 otherwise You aim at analyzing the effects of a worker s schooling (college degree or not) and the worker s gender (female or male) on the worker s earnings 99

29 Empirical problem: [continued] Consider the intuitive regression model Interpretation of parameters: Y i = β 0 + β 1 D 1i + β 2 D 2i + u i (4.9) β 1 is the effect on (log) earnings of having a college degree holding gender constant β 2 is the effect on (log) earnings of being female holding schooling constant The limitation of this model is that the effect on earnings of having a college degree is the same for men and women 100

30 Removal of this limitation: Augmenting the regression model (4.9) by the interaction term (D 1i D 2i ): Y i = β 0 + β 1 D 1i + β 2 D 2i + β 3 (D 1i D 2i ) + u i (4.10) The interaction term (D 1i D 2i ) in (4.10) allows the population effect on log earnings (Y i ) of having a college degree (that is changing D 1i from D 1i = 0 to D 1i = 1) to depend on gender D 2i Mathematical background: Use Formula (4.3) on Slide 82 to compute the expected effect on Y, Y, resulting from a change in D 1i from 0 to 1 given the fixed value d 2 for D 2i 101

31 Mathematical background: [continued] We have E(Y i D 1i = 0, D 2i = d 2 ) = β 0 + β β 2 d 2 + β 3 (0 d 2 ) and = β 0 + β 2 d 2 E(Y i D 1i = 1, D 2i = d 2 ) = β 0 + β β 2 d 2 + β 3 (1 d 2 ) This yields the expected effect on Y : = β 0 + β 1 + β 2 d 2 + β 3 d 2 Y = E(Y i D 1i = 1, D 2i = d 2 ) E(Y i D 1i = 0, D 2i = d 2 ) = β 0 + β 1 + β 2 d 2 + β 3 d 2 β 0 β 2 d 2 = β 1 + β 3 d 2 (4.11) 102

32 Interpretation of (4.11): The expected effect of acquiring a college degree (that is a unit change in D 1i ) depends on the person s gender: Y = { β 1 if the worker is male (d 2 = 0) β 1 + β 3 if the worker is female (d 2 = 1) The coefficient β 3 on the interaction term (D 1i D 2i ) in regression (4.10) is the difference in the effect of acquiring a college degree for women versus men Empirical exercise: Interaction between the student-teacher ratio and the percentage of English learners in a dummy regression model of the form (4.10) (see class) 103

33 Interactions between a continuous and a dummy regressor Consider the following data set: The dependent variable is (log) earnings, that is Y i = ln(earnings i ) (i = 1,..., n) We consider the dummy regressor D i = { 1 if the i th worker has a college degree 0 otherwise We consider the continuous regressor X i = individual s years of work experience 104

34 Three alternative regression models: Specification with both regressors, no interaction term Y i = β 0 + β 1 X i + β 2 D i + u i (4.12) Specification with both regressors plus interaction term Y i = β 0 + β 1 X i + β 2 D i + β 3 (X i D i ) + u i (4.13) Specification with continuous regressor plus interaction term Y i = β 0 + β 1 X i + β 2 (X i D i ) + u i (4.14) Question: For each of the specifications (4.12) (4.14), what are the expected effects on (log) earnings (Y i ) of having a college degree (that is from changing D i from D i = 0 to D i = 1)? 105

35 Expected effects: For specification (4.12) we have Y = E(Y i D i = 1, X i ) E(Y i D i = 0, X i ) = β 0 + β 1 X i + β 2 β 0 β 1 X i = β 2 (4.15) By analogous calculations, we find for the specifications (4.13) and (4.14) and Y = β 2 + β 3 X i (4.16) Y = β 2 X i (4.17) 106

36 Remarks: The effects Y = E(Y i D i = 1, X i ) E(Y i D i = 0, X i ) computed in (4.15) (4.17) can be interpreted as differences in the two population regression functions associated with the two values of the dummy regressor D i = 1 and D i = 0 For specification (4.12) this difference is constantly equal to β 2 producing two population regression lines with different intercepts and the same slope By analogous reasoning, the specifications (4.13) and (4.14) produce population regression lines with (a) different intercepts and different slopes and with (b) the same intercept and different slopes (see the figure on Slide 108) 107

37 Regression functions using dummy and continuous regressors 108

38 Interpretation of coefficients: How can we interpret the specific regression coefficients involved in the specifications (4.12) (4.14) See class Empirical exercise: Application to the student-teacher ratio and the percentage of English learners (see class) 109

39 Interactions between two continuous regressors Consider the following data set: The dependent variable is (log) earnings, that is Y i = ln(earnings i ) (i = 1,..., n) We consider the two continuous regressors X 1i = individual s years of work experience X 2i = individual s years he or she went to school Regression specification: Y i = β 0 + β 1 X 1i + β 2 X 2i + β 3 (X 1i X 2i ) + u i (4.18) 110

40 Expected effects on Y : In (4.18), a change in X 1 by X 1 (holding X 2 constant) leads to Y = (β 1 + β 3 X 2 ) X 1 (4.19) The effect on Y of a change in X 1 by X 1 depends on the value of X 2 Analogously, we find that the effect on Y of a change in X 2 by X 2 (holding X 1 constant) depends on the value of X 1 : Y = (β 2 + β 3 X 1 ) X 2 (4.20) 111

41 Expected effects on Y : [continued] We now consider a simultaneous change in X 1 by X 1 and, at the same time, in X 2 by X 2 We then find that the expected change in Y is given by Y = (β 1 + β 3 X 2 ) X 1 + (β 2 + β 3 X 1 ) X 2 + β 3 X 1 X 2 (4.21) The first term in (4.21) is the effect from changing X 1 holding X 2 constant The second term in (4.21) is the effect from changing X 2 holding X 1 constant The final term, β 3 X 1 X 2, in (4.21) is the extra effect from changing both X 1 and X 2 112

42 Interactions in multiple regression: [Summary] The interaction term between the regressors X 1 and X 2 is their product X 1 X 2 Including the interaction term allows the effect on Y of a change in X 1 to depend on the value of X 2 and, conversely, allows the effect of a change in X 2 to depend on the value of X 1 The coefficient on X 1 X 2 is the effect of a one-unit increase in X 1 and X 2, above and beyond the sum of the individual effects of a unit increase in X 1 alone and a unit increase in X 2 alone This is true irrespective of whether X 1 and/or, X 2 are continuous or dummy regressors 113