Lecture notes to Stock and Watson chapter 4

Size: px

Start display at page:

Download "Lecture notes to Stock and Watson chapter 4"

Samson Stafford
6 years ago
Views:

1 Lecture notes to Stock and Watson chapter 4 Introductory linear regression Tore Schweder Sept 2009 TS () LN3 03/09 1 / 15

Regression "Regression" is due to Francis Galton (1822-1911): how is

2 Regression "Regression" is due to Francis Galton ( ): how is a son s height related to his father s height? TS () LN3 03/09 2 / 15

3 Regression towards the mean Figure: Galton s original diagram. Parents-child pairs by heigth. TS () LN3 03/09 3 / 15

4 The geometry of linear regression Figure: The regression curve is y = f (x) = E [Y jx = x]. TS () LN3 03/09 4 / 15

5 The linear regression model Question: how is a response variable Y related to a stimulus variable X (explanatory/control/explanatory)? Assuming a linear relation, 1 what is the slope? TS () LN3 03/09 5 / 15

6 The linear regression model Question: how is a response variable Y related to a stimulus variable X (explanatory/control/explanatory)? Assuming a linear relation, 1 what is the slope? 2 is the slope positive? TS () LN3 03/09 5 / 15

7 The linear regression model Question: how is a response variable Y related to a stimulus variable X (explanatory/control/explanatory)? Assuming a linear relation, 1 what is the slope? 2 is the slope positive? 3 how good does the estimated line t the observed data? TS () LN3 03/09 5 / 15

8 The linear regression model Question: how is a response variable Y related to a stimulus variable X (explanatory/control/explanatory)? Assuming a linear relation, 1 what is the slope? 2 is the slope positive? 3 how good does the estimated line t the observed data? Example: Y = growth in BNP, X = in ation previous year. TS () LN3 03/09 5 / 15

9 The linear regression model Question: how is a response variable Y related to a stimulus variable X (explanatory/control/explanatory)? Assuming a linear relation, 1 what is the slope? 2 is the slope positive? 3 how good does the estimated line t the observed data? Example: Y = growth in BNP, X = in ation previous year. D : A sample of size n of pairs explanatory variables X and response variable Y. TS () LN3 03/09 5 / 15

10 The linear regression model Question: how is a response variable Y related to a stimulus variable X (explanatory/control/explanatory)? Assuming a linear relation, 1 what is the slope? 2 is the slope positive? 3 how good does the estimated line t the observed data? Example: Y = growth in BNP, X = in ation previous year. D : A sample of size n of pairs explanatory variables X and response variable Y. M : (X 1, Y 1 ),, (X n, Y n ) is an iid random sample from an in nite population. Y = β 0 + β 1 X + u; E [Y jx = x] = β 0 + β 1 x, Eu = 0, cov(u, X ) = 0. β 0, β 1 are parameters. TS () LN3 03/09 5 / 15

11 OLS Least squares and regression was known by Laplace and Gauss (80 years before Galton). b β 0, bβ 1 = arg min b 0,b 1 bβ 1 = s XY s 2 X n i=1 (Y i (b 0 + b 1 X i )) 2 = r s Y s X, bβ 0 = Y bβ 1 X Predicted response given the stimuli: by i = bβ 0 + bβ 1 X i is the tted value. TS () LN3 03/09 6 / 15

12 OLS Least squares and regression was known by Laplace and Gauss (80 years before Galton). b β 0, bβ 1 = arg min b 0,b 1 bβ 1 = s XY s 2 X n i=1 (Y i (b 0 + b 1 X i )) 2 = r s Y s X, bβ 0 = Y bβ 1 X Predicted response given the stimuli: by i = bβ 0 + bβ 1 X i is the tted value. (Empirical) residual: bu i = Y i by i is the vertical distance from the tted value to the observed value of Y. bu = 0, r bux = 0 TS () LN3 03/09 6 / 15

13 Measures of t Homoscedasticity when var(y jx = x) = σ 2 is independent of x. TS () LN3 03/09 7 / 15

14 Measures of t Homoscedasticity when var(y jx = x) = σ 2 is independent of x. 2 Then bσ 2 = 1 n 2 n i=1 Y i by i = 1 n 2 RSS is unbiased. TS () LN3 03/09 7 / 15

15 Measures of t Homoscedasticity when var(y jx = x) = σ 2 is independent of x. 2 Then bσ 2 = 1 n 2 n i=1 Y i by i = 1 n 2 RSS is unbiased. TSS = n i=1 Y i Y 2 = n i=1 Y i 2 by i + n i=1 Y bi 2 Y = RSS + ESS TS () LN3 03/09 7 / 15

16 Measures of t Homoscedasticity when var(y jx = x) = σ 2 is independent of x. 2 Then bσ 2 = 1 n 2 n i=1 Y i by i = 1 n 2 RSS is unbiased. TSS = n i=1 Y i Y 2 = n i=1 Y i by i 2 + n i=1 b Y i Y 2 = RSS + ESS R 2 = ESS TSS = 1 RSS TSS = r 2 XY TS () LN3 03/09 7 / 15

17 Be aware of extreme observations! Figure: The OLS line hinges on one point. Figure: Heavy tails in the conditional distribution of Y jx distorts the OLS line. TS () LN3 03/09 8 / 15

18 Desired number of children and age for beginning master students n Children Age OLS ρ R C = A C = A Children + rnorm(30, 0, 0.1) Children A ge A ge 2008 TS () LN3 03/09 9 / 15

19 Desired number of children and number of siblings, 2009 TS () LN3 03/09 10 / 15

20 Siblings Children Fitted values Figure: STATA scatter diagram of Children vs. Siblings, with OLS line. Some points represent several observations. Is the (slightly) positive relation between Age and Children due to Siblings being an intermediate variable? Age!Siblings!Children, but controling for siblings, no e ect of Age on Siblings? TS () LN3 03/09 11 / 15

21 Constructed model and simulated data y = number of children x = age (and probability) y x = age (and probability) y = number of children var(y1x=x) x x = age Figure: UL: E [Y = yjx = x] = x, and P [Y = yjx = x] shown by horizontal thick lines for some values of x; UR: the same, but axes interchanged; LL the scatter plot of a simulated sample of size n = 43, the points are slightly jittered, with the OLS line y = x; LR: var [Y jx = x]. TS () LN3 03/09 12 / 15

22 Constructed model - repeated simulations density density beta beta1 density density T0 Figure: Approximate densities of bβ 0 (UL), bβ 1 (UR), T 0 = b β 0 β 0 /SE b β 0 (LL), and T 1 = b β 1 β 1 /SE b β 1 (LL), based on 1000 replicates of simulated data of size n = 43 in constructed model for (X, Y ).Dotted curve is the normal density. TS () LN3 03/09 13 / 15 T1

23 Constructed model - repeated simulations (cont) beta0.hat beta1.hat Quantiles of Standard Normal Quantiles of Standard Normal T T Quantiles of Standard Normal Quantiles of Standard Normal Figure: Normal probability plots (QQ-plots against N(0,1)) for the same simulated material as in previous gure. TS () LN3 03/09 14 / 15

24 Problems to be done in class SW: 4.3 TS () LN3 03/09 15 / 15

Environmental Econometrics

Environmental Econometrics Syngjoo Choi Fall 2008 Environmental Econometrics (GR03) Fall 2008 1 / 37 Syllabus I This is an introductory econometrics course which assumes no prior knowledge on econometrics;