Lecture 5. In the last lecture, we covered. This lecture introduces you to

Transcription

1 Lecture 5 In the last lecture, we covered. homework 2. The linear regression model (4.) 3. Estimating the coefficients (4.2) This lecture introduces you to. Measures of Fit (4.3) 2. The Least Square Assumptions (4.4) 3. Sampling Distribution of the OLS estimators (4.5)

2 Recap: terminology Y i = β 0 + β X i + u i, i =,, n Dependent variable Regressand Left-hand variable independent variable Regressor Right-hand variable error term 2

3 The OLS estimator solves: 0 n min [ Y ( b + b X )] b, b i 0 i i= 2 The result is called the OLS estimators of β 0 and β. use hat when we refer to estimator Estimated slope = ˆ β Estimated intercept = ˆ β 0 3

4 The OLS Linear Regression Model general notation Actual value residual Y i = Y ˆi + u ˆi, i =,, n where Predicted value ˆ ˆ ˆ Y β + β X i 0 i estimators = i =,, n 4

5 Application to the California Test Score Class Size data Estimated slope = ˆ β = 2.28 Estimated intercept = ˆ β 0 = Estimated regression line: TestScore = STR 5

6 Interpretation of the estimated slope and intercept TestScore = STR Test score = 2.28 STR what if zero students? 6

7 Predicted values & residuals: exercise Antelope, CA, has STR = 9.33 and Test Score = What is their predicted value and residual? 7

8 Exercises Suppose that a researcher, using data on class size (CS) and average test scores from 00 third-grade classes, estimates the OLS regression TestScore _ hat = * CS a) A classroom has 22 students. What is the regression s prediction for that classroom s average test score? TestScore=.. =. 8

9 Exercises Suppose that a researcher, using data on class size (CS) and average test scores from 00 third-grade classes, estimates the OLS regression TestScore _ hat = * CS b) Last year a classroom had 9 students, and this year it has 23 students. What is the regression s prediction for the change in the classroom average test score? TestScore _ hat = 5.82* CS =

10 Exercises Suppose that a researcher, using data on class size (CS) and average test scores from 00 third-grade classes, estimates the OLS regression TestScore _ hat = * CS c) The sample average class size across the 00 classroooms is 2.4. What is the sample average of the test scores across the 00 classrooms? TestScore = ˆ β + ˆ β CS= =

11 Measures of Fit (Section 4.3) how well does the regression line fit the data? The regression R 2 The standard error of the regression (SER)

12 The regression R 2 The regression R 2 is the fraction of the sample variance of Y i explained by the regression. Explained sum of squared Definition of R 2 : R 2 = ESS TSS = n i= n i= ( Yˆ Yˆ ) i ( Y Y ) i 2 2 Total sum of squared 2

13 This terminology in a picture dependent variable ESS R 2 = = TSS n i= n i= ( Yˆ Yˆ ) i ( Y Y ) i 2 2 Y Consider this point independent variable 3

14 The regression R 2 Definition of R 2 : R 2 = ESS TSS = n i= n i= ( Yˆ Yˆ ) i ( Y Y ) i 2 2 R 2 = 0 means ESS = 0 R 2 = means ESS = TSS 0 R 2 4

15 The Standard Error of the Regression (SER) The SER measures the spread of the distribution of u. SER = n ( uˆ ˆ i u) n 2 i= 2 = n uˆ n 2 i= 2 i Two, because we estimate two parameters in this regression 5

16 The Standard Error of the Regression (SER) SER = The SER: n uˆ n 2 i= has the units of u, which are the units of Y measures the average size of the OLS residual The root mean squared error (RMSE) is closely related to the SER: RMSE = n uˆ n = i 2 i 2 i 6

17 Example of the R 2 and the SER TestScore = STR, R 2 =.05, SER = 8.6 what do you conclude? 7

18 OLS regression: STATA output regress testscr str, robust Regression with robust standard errors Number of obs = 420 F(, 48) = 9.26 Prob > F = R-squared = Root MSE = Robust testscr Coef. Std. Err. t P> t [95% Conf. Interval] str _cons TestScore = STR (we ll discuss the rest of this output later) 8

19 The Least Squares Assumptions (SW Section 4.4) Two important questions to ask Is OLS estimator unbiased? Does it have a small variance? There are three assumptions that assures these points 9

20 Assumptions # Y i = β 0 + β X i + u i, i =,, n. The conditional distribution of u given X has mean zero, that is, conditional on Expected value of... E(u i X i = x) = 0, i =,, n Error term Independent variable X equal to particular value x 20

21 In a picture For any given value of X, the mean of u is zero: 2

22 Implication of assumption # corr (X i, u i ) = 0 in our example, Test Score i = β 0 + β STR i + u i, where u i = other factors other factor can be, for example, English learners English learners surely affects test performance if English learners is somehow associated with STR, the above relation does not hold anymore 22

23 Assumptions #2 Y i = β 0 + β X i + u i, i =,, n 2. (X i,y i ), i =,,n, are i.i.d. that is, independently and identically distributed i.i.d example: household survey, national poll not i.i.d : experiment on using PPT in the classroom, inflation 23

24 Assumption #3 Y i = β 0 + β X i + u i, i =,, n 3. Large outliers in X and/or Y are rare. Technically, X and Y have finite fourth moments Outliers can result in meaningless values of ˆ β 24

25 OLS can be sensitive to an outlier: 25