ECON Introductory Econometrics. Lecture 4: Linear Regression with One Regressor

Transcription

1 ECON Introductory Econometrics Lecture 4: Linear Regression with One Regressor Monique de Haan (moniqued@econ.uio.no) Stock and Watson Chapter 4

2 Lecture outline 2 The OLS estimators The effect of class size on test scores The Least Squares Assumptions E (u i X i ) = 0 (X i, Y i ) are i.i.d Large outliers are unlikely Properties of the OLS estimators unbiasedness consistency large sample distribution The compulsory term paper

3 The OLS estimators 3 Question of interest: What is the effect of a change in X i on Y i? Y i = β 0 + β 1 X i + u i Last week we derived the OLS estimators of β 0 and β 1 : β 0 = Y β 1 X β 1 = 1 n n 1 i=1(x i X)(Y i Y) n i=1(x i X)(X i X) = sxy 1 n 1 s 2 x,

4 OLS estimates: The effect of class size on test scores 4 Friday January 13 14:48: Page 1 Question of interest: What is the effect of a change in class size on test scores? / / / / / / / TestScore i = β 0 + β 1 ClassSize i + u i 1. regress test_score class_size, robust (R) / / / / / Statistics/Data Analysis Linear regression Number of obs = 420 F(1, 418) = Prob > F = R-squared = Root MSE = Robust test_score Coef. Std. Err. t P> t [95% Conf. Interval] class_size _cons TestScore i = ClassSize i

5 The Least Squares assumptions 5 Y i = β 0 + β 1 X i + u i Under what assumptions does the method of ordinary least squares provide appropriate estimators of β 0 and β 0? Under what assumptions does the method of ordinary least squares provide an appropriate estimator of the effect of class size on test scores? The Least Squares assumptions: Assumption 1: The conditional mean of u i given X i is zero E (u i X i ) = 0 Assumption 2: (Y i, X i ) for i = 1,..., n are independently and identically distributed (i.i.d) Assumption 3: Large outliers are unlikely ( ) ( 0 < E < & 0 < E X 4 i Y 4 i ) <

6 The Least Squares assumptions: Assumption 1 6 The first OLS assumption states that: E (u i X i ) = 0 All other factors that affect the dependent variable Y i (contained in u i ) are unrelated to X i in the sense that, given a value of X i, the mean of these other factors equals zero. In the class size example: All the other factors affecting test scores should be unrelated to class size in the sense that, given a value of class size, the mean of these other factors equals zero.

7 The Least Squares assumptions: Assumption 1 7 The first OLS assumption can also be written as: E (Y i X i ) = E (β 0 + β 1 X i + u i X i ) Expectation rules = β 0 + β 1 E (X i X i ) + E (u i X i ) ASS#1 : E (u i X i ) = 0 = β 0 + β 1 X i

8 The Least Squares assumptions: Assumption 1 8 E (Y i X i ) = β 0 + β 1

9 The Least Squares assumptions: Assumption 1 9 Example of a violation of assumption 1: Suppose that districts which wealthy inhabitants have small classes and good teachers these districts have a lot of money which they can use to hire more and better teachers districts with poor inhabitants have large classes and bad teachers. These districts have little money and can hire only few and not very good teachers In this case class size is related to teacher quality. Since teacher quality likely affects test scores it is contained in u i. This implies a violation of assumption 1: E (u i ClassSize i = small) E (u i ClassSize i = large) 0

10 The Least Squares assumptions: Assumption 2 10 (Y i, X i ) for i = 1,..., n are i.i.d If the sample is drawn by simple random sampling assumption 2 will hold Example: What is effect of mother s education (X i ) on child s education (Y i ) Example of simple random sampling: randomly draw sample of mother s with information on her education and the education of one randomly selected child (Y i, X i ) for i = 1,..., n are i.i.d Example of a violation of simple random sampling randomly draw sample of mothers with information on her education and the education of all of her children. (Y i, X i ) for i = 1,..., n are NOT i.i.d Observations on children from the same mother are not independent!

11 The Least Squares assumptions: Assumption 3 11 Large outliers are unlikely ( ) ( ) 0 < E < & 0 < E < X 4 i Y 4 i Outliers are observations that have values far outside the usual range of the data Large outliers can make OLS regression results misleading Another way to state assumption is that X and Y have finite kurtosis. Assumption is necessary to justify the large sample approximation to the sampling distribution of the OLS estimators

12 The Least Squares assumptions: Assumption 3 12

13 Use of the Least Squares assumptions 13 Y i = β 0 + β 1 X i + u i Assumption 1: E (u i X i ) = 0 Assumption 2: (Y i, X i ) for i = 1,..., n are i.i.d Assumption 3: Large outliers are unlikely If the 3 least squares assumptions hold the OLS estimators β 0 and β 1 Are unbiased estimators of β 0 and β 1 Are consistent estimators of β 0 and β 1 Have a jointly normal sampling distribution

14 Properties of the OLS estimator: unbiasedness 14 Y i = β 0 + β 1 X i + u i Y = β 0 + β 1 X i + u ] E [ β1 = E ] i=1(x i X)(Y i Y) n i=1(x i X)(X i X) [ n substitute for Y i, Y = E [ n i=1(x i X)(β 0 +β 1 X i +u i (β 0 +β 1 X+u)) n i=1(x i X)(X i X) rewrite (β 0 drops out) = E [ n i=1(x i X)(β 1 (X i X)+(u i u)) n i=1(x i X)(X i X) rewrite & use expectation rules [ β n ] [ n ] 1 i=1(x i X)(X i X) i=1(x i X)(u i u) = E n + E i=1(x i X)(X i X) n i=1(x i X)(X i X) ] ]

15 Properties of the OLS estimator: unbiasedness 15. ] E [ β1 [ β ni=1 ] [ ni=1 ] = E 1 (X i X)(X i X) (X ni=1 + E i X)(u i u) (X i X)(X i X) ni=1 (X i X)(X i X) take β 1 out of 1st expectation Algebra trick [ ni=1 ] (X = β 1 + E i X)u i ni=1 (X i X)(X i X) Law of iterated expectations [ ni=1 ] (X = β 1 + E i X)E[u i X i ] ni=1 (X i X)(X i X) ] E [ β1 = β 1 if E [u i X i ] = 0

16 Algebra trick 16 ( ) n i=1 X i X (u i u) = n i=1 X iu i n i=1 X iu n i=1 Xu i + n i=1 Xu = n i=1 X iu i n ( 1 n n i=1 X i) u n i=1 Xu i + nxu = n i=1 X iu i nxu + n i=1 Xu i+nxu = n i=1 X iu i n i=1 Xu i = n i=1 ( X i X ) u i

17 Consistency 17 Consistency: β 1 p β 1 or plim β 1 = β 1 Plim β 1 = plim ( ni=1 ) (X i X)(Y i Y) ni=1 (X i X)(X i X) = Plim 1 Plim 1 ni=1 n 1 (X i X)(Y i Y) ni=1 n 1 (X i X)(X i X) law of large numbers OLS assumptions 2 and 3 = s XY s 2 X = Cov(X i,y i ) Var(X i ) substitute for Y i = Cov(X i,β 0 +β 1 X i +u i ) Var(X i ) see Key Concept 2.3 = β 1Var(X i )+Cov(X i,u i ) Var(X i )

18 Consistency 18 Plim β 1 = β 1Var(X i )+Cov(X i,u i ) Var(X i ) Var(X = β i ) 1 + Cov(X i,u i ) Var(X i ) Var(X i ) substitute covariance expression = β 1 + E[(X i µ x )(u i µ u)] Var(X i ) algebra trick = β 1 + E[(X i µ x )u i ] Var(X i ) Law of iterated expectations so = β 1 + E[(X i µ x )E[u i X i ]] Var(X i ) Plim β 1 = β 1 if E [u i X i ] = 0

19 Unbiasedness vs Consistency 19 Unbiasedness & consistency both rely on E [u i X i ] = 0 [ ] Unbiasedness implies that E β1 = β 1 for a given sample size n Consistency implies that the sampling distribution becomes more and more tightly distributed around β 1 if the sample size n becomes larger and larger.

20 Consistency: A simulation example 20 Lets create a data set with 100 observations X i N(0, 1) u i N(0, 1) We define Y to depend on X as: Y i = 1 + 2X i + u i Thursday January 19 12:00: Page 1 1. sum y x set obs 1000 (R) gen x=invnorm(uniform()) / / / / / / / / / / / / gen y=1+2*x+invnorm(uniform()) Statistics/Data Analysis Variable Obs Mean Std. Dev. Min Max y x

21 A simulation example 21 5 Y 0 Thursday January 19 12:01: Page (R) X / / / / / / / / / / / / Statistics/Data Analysis 1. regress y x Source SS df MS Number of obs = 100 F( 1, 98) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = y Coef. Std. Err. t P> t [95% Conf. Interval] x _cons

22 A simulation example n= Monday February 16 16:57: Page 1 We can create 999 of these data sets with 100 observations and use OLS to estimate Y i = β 0 + β 1 + u i 1. program define ols, rclass 1. drop _all 2. set obs gen x=invnorm(uniform()) 4. gen y=1+2*x+invnorm(uniform()) 5. regress y x 6. end simulate _b, reps(999) nodots : ols command: ols (R) / / / / / / / / / / / / Statistics/Data Analysis 4. sum Variable Obs Mean Std. Dev. Min Max _b_x _b_cons

23 A simulation example n= OLS estimates of B 1 in 999 samples with n= OLS estimates of B 1

24 A simulation example n=1000 Tuesday February 17 13:03: Page 1 24 (R) / / / / / / / / / / / / Statistics/Data Analysis 1. program define ols, rclass 1. drop _all 2. set obs gen x=invnorm(uniform()) 4. gen y=1+2*x+invnorm(uniform()) 5. regress y x 6. end simulate _b, reps(999) nodots : ols command: ols 4. sum Variable Obs Mean Std. Dev. Min Max _b_x _b_cons

25 A simulation example n= OLS estimates of B 1 in 999 samples with n= OLS estimates of B 1

26 A simulation Friday January example 20 12:01:22 n= Page 1 26 (R) / / / / / / / / / / / / Statistics/Data Analysis 1. program define ols, rclass 1. drop _all 2. set obs gen x=invnorm(uniform()) 4. gen y=1+2*x+invnorm(uniform()) 5. regress y x 6. end simulate _b, reps(999) nodots: ols command: ols 4. sum Variable Obs Mean Std. Dev. Min Max _b_x _b_cons

27 A simulation example n= OLS estimates of B 1 in 999 samples with n= OLS estimates of B 1

28 . 28 Consistency of the OLS estimator of β 1 True model : Y i = 1 + 2X i + u i, Estimated model : Y i = β 0 + β 1 X i + u i OLS estimates of B 1 in 999 samples with n=100; n=1000 and n=10000 n=100 n=1000 n= OLS estimates of B 1

29 29 Sampling distribution of β 0 and β 1 We discussed the sampling distribution of the sample average Y : sampling distribution is complicated for small n, but if Y 1,..., Y n are i.i.d. we know that ( ) E Y = µ Y By the Central Limit theorem the large sample distribution can be approximated by the normal distribution: ( ) Y N µ Y, σ2 Y n If the 3 least squares assumptions hold we can make similar statements about the OLS estimators β 0 and β 1

30 30 Large-sample distribution of β 0 and β 1 Technically the Central Limit theorem concerns the large sample distribution of averages (like Y ) Examining the formulas of the OLS estimators shows that these are functions of sample averages: β 0 = Y β 1 X n β 1 = 1 ni=1 (X i X)(Y i Y) 1 ni=1 n (X i X)(X i X) It turns out that the Central Limit theorem also applies to these functions of sample averages.

31 31 Sampling distribution of β 0 and β 1 If the first least squares assumption holds: The OLS estimators are unbiased which implies that (for any sample size n) ) ) E ( β0 = β 0 and E ( β1 = β 1 In addition, if all 3 least squares assumptions hold The Central Limit theorem implies that β 0 and β 1 are approximately jointly normally distributed in large samples: ) β 0 N (β 0, σ 2 β0 ) β 1 N (β 1, σ 2 β1

32 32 Large-sample distribution of β 0 and β 1 In large samples ) β 0 N (β 0, σ 2 β0 ) β 1 N (β 1, σ 2 β1 where it can be shown that σ 2 β0 = 1 Var(H i u i ) n [E(H 2 i )] 2 [ with H i = 1 µ X E(X 2 i ) ] X i σ 2 β1 = 1 Var[(X i µ X )u i ] n [Var(X i )] 2 Expression for σ 2 β1 shows that the larger the variation in the regressor X i the smaller the variance of β 1

33 33 Large-sample distribution of β 0 and β 1 When Var(X i ) is low, it is difficult to obtain an ) accurate estimate of the effect of X on Y which implies that Var ( β1 = σ 2 β1 is high. If there is more variation in X, then there is more information in the data that you can use to fit the regression line.

34 Compulsory term paper 34 Traffic fatalities are the leading cause of death for Americans between the ages of 5 and 32. The government wants to decrease the number of traffic fatalities by increasing seat belt usage. If many people wear seat belts the chance that people die in a car crash is likely smaller. People who wear seat belts might however be more careful drivers. Regions with many seat belt users might have fewer traffic fatalities not because of the seat belt usage but because the drivers are more careful.

35 Compulsory term paper 35 In the term paper you are going to investigate the following research question. What is the causal effect of seat belt usage on traffic fatalities? This research question can be addressed by using the data set seatbelts.dta. Data consists of a panel of 50 U.S. States, plus the District of Columbia, for the years The data sets can be downloaded from the course website site. In analyzing this data you may consider the use of panel data methods on top of a pure cross-section analysis.

36 Compulsory term paper 36 The term paper should consist of the following sections: Introduction Empirical approach Data Results Conclusion References Appendix with Stata code & output The term paper should be at most 10 pages including tables and figures (but excluding the stata code and output). The quality (and not the quantity) of the content of the term paper will determine your grade.

37 Compulsory term paper 37 You are expected to work in a group of two students. You can form a group of two students yourself Register this group before 29 January :00, by using link in you will receive today. If you are unable to form a group, please let me know before 29 January you will be randomly assigned to another student. Important dates: 25 January 2017 Hand-out of term paper 22 March 2017 Hand-in of term paper on Fronter 11 April 2017 Notification of grade (pass/fail) 21 April 2017 Hand-in of improved term paper for those who failed 4 May 2017 Everyone is informed about final grade for term paper