ECO220Y Simple Regression: Testing the Slope

Transcription

1 ECO220Y Simple Regression: Testing the Slope Readings: Chapter 18 (Sections ) Winter 2012 Lecture 19 (Winter 2012) Simple Regression Lecture 19 1 / 32

2 Simple Regression Model y i = β 0 + β 1 x i + ε i Model OLS produces b 0 and b 1 Estimated regression line: ŷ = b 0 + b 1 x Are b 0 and b 1 point or interval estimators of the population parameters? What do we need to obtain interval estimators? ε i N(0, σ 2 ) (Winter 2012) Simple Regression Lecture 19 2 / 32

3 Standard Error of the Slope Estimate Standard error of the slope estimate also tells us how precisely we are able to estimate the slope. (Why?) b 1 = s [ ] xy sxy sx 2 V (b 1 ) = V sx 2 σ 2 σb 2 1 = (n 1)sx 2 σ b1 = b 1 N(β, σ 2 b 1 ) σ 2 (n 1)s 2 x But: We do not know σ! (Winter 2012) Simple Regression Lecture 19 3 / 32

4 Variance of ε i ε i N(0, σ 2 ) Do we observe ε i? We cannot compute σ 2 N N (ε i ε) 2 (ε i 0) 2 σ 2 = i=1 = N i=1 What implicit assumption do we make when setting ε = 0? N (Winter 2012) Simple Regression Lecture 19 4 / 32

5 Estimate of ε i Residual e i is an estimate of the error term ε i Residual e i is the difference between the estimated model and y i Error ε i is the difference between the true model and y i We will use variability of residuals to estimate the variance of the errors. (Winter 2012) Simple Regression Lecture 19 5 / 32

6 Variance of e i σ 2 = N (ε i 0) 2 i=1 N s 2 e = n (e i 0) 2 i=1 n 2 s 2 e is an unbiased estimate of σ 2 n 2 in the denominator because we used up two degrees of freedom to compute estimates of β 0 and β 1 to find e i : e i = y i ŷ i = y i b 0 b 1 x i (Winter 2012) Simple Regression Lecture 19 6 / 32

7 Standard Error of Estimate: s e Standard error of estimate, s e, is an estimate of σ, where σ comes from ε i N(0, σ 2 ) s 2 e = n (e i 0) 2 i=1 n 2 = n i=1 e 2 i n 2 = n (ŷ i y i ) 2 i=1 n 2 = SSE n 2 s e = SSE n 2 (Winter 2012) Simple Regression Lecture 19 7 / 32

8 Variance of the Slope Estimate b 1 = s xy s 2 x σ 2 σb 2 1 = (n 1)sx 2 V (b 1 ) = V σ b1 = b 1 N(β, σ 2 b 1 ) [ sxy s 2 x ] σ 2 (n 1)s 2 x We do not know σ, but we can estimate s e! (Winter 2012) Simple Regression Lecture 19 8 / 32

9 Solution: Replace σ with s Standard Error of the slope estimate (b 1 ): s s b1 = e (n 1)sx 2 Consequences of uncertainty: For hypothesis testing use Student t (not z) Example of standard notation: ŷ = x (5.8) (0.05) s b0 s b1 (Winter 2012) Simple Regression Lecture 19 9 / 32

10 What affects precision of the slope estimate? s b1 = s e (n 1)s 2 x Sample size Variance of independent variable Standard Error of estimate, s e (Winter 2012) Simple Regression Lecture / 32

11 Difference Between Two Graphs? For which b 1 will be a more precise estimate of β 1? (Winter 2012) Simple Regression Lecture / 32

12 Testing the Slope y i = β 0 + β 1 x i + ε i Model Set of Statistical Hypotheses: Two-tailed One-tailed One-tailed H 0 : β 1 = β1 0 H 0 : β 1 = β1 0 H 0 : β 1 = β1 0 H 1 : β 1 β1 0 H 1 : β 1 > β1 0 H 1 : β 1 < β1 0 where β 0 1 is any number, does not have to be 0. test-statistics: t = b 1 β 0 1 s b1 t Student t, with n 2 degrees of freedom (Winter 2012) Simple Regression Lecture / 32

13 Statistical Significance Test of statistical significance for the slope coefficient: H 0 : β 1 = 0 H 1 : β 1 0 t-statistic: t (ν=n 2) = b 1 s b1 Can use rejection region approach or p-value approach (Winter 2012) Simple Regression Lecture / 32

14 Rejection Region Approach (Winter 2012) Simple Regression Lecture / 32

15 Example: Movie Budget Standard error of the slope estimate is Sample size, n = 120 Can we infer that the movie budget and the running time are linearly related? (Winter 2012) Simple Regression Lecture / 32

16 P-value P-value is probability of obtaining a slope estimate as extreme as the one estimated in the direction of H 1 if H 0 is true. In general, for two tailed test, p-value is equal to P[t < b 1 β1 0, t > b 1 β1 0 H 0 is true] s b1 s b1 For one-tailed left-sided test p-value is equal to P[t < b 1 β1 0 H 0 is true] s b1 For one-tailed right-sided test p-value is equal to P[t > b 1 β1 0 H 0 is true] s b1 (Winter 2012) Simple Regression Lecture / 32

17 P-Value Approach (Winter 2012) Simple Regression Lecture / 32

18 Movie Example Set up hypotheses: Compute test-statistic: Find the value of t-stats in t-table Do not forget about the degrees of freedom! Conclusion? (Winter 2012) Simple Regression Lecture / 32

19 Statistical vs Economic Significance Statistically Significant: The degree of correlation between explanatory and dependent variables that is not likely observed due to mere chance. The statistical significance of a variable is entirely determined by the size of t b1. Statistical significance improves as sample size increases. Why? Statistical significance does not automatically imply that you have found something important. Economically Significant: An effect large enough in magnitude that decision makers would consider it important. The economical importance is related to the magnitude and sign of b 1 and indicates whether the explanatory variable has a meaningful and plausible influence on dependent variable. (Winter 2012) Simple Regression Lecture / 32

20 Statistical vs Economic Significance Example Consider hypothetical regression of test scores on class size and assume we have found: Testscore hat = ClassSize (15.45) (1.12) Is b 1 statistically significant? Can you give an answer just by eyeballing? Interpretation: Consider average test score of 650. Reduction in class size by 1 student is associated with improvement in average test score by 5 points, or only 0.8%! Is this result economically significant? (Winter 2012) Simple Regression Lecture / 32

21 β 0 1 does not have to be 0! Can test for other values of β 1, not only 0! For instance, H 0 : β 1 = 3 vs. H A : β 1 > 3 The same procedure as for the standard t-test for statistical significance 1 Set the hypotheses 2 Compute t-statistic: t n 2 = b1 β0 1 se b1 3 Use p-value or rejection region approach Example: Cost of living index (Winter 2012) Simple Regression Lecture / 32

22 Example: Cost of Living Index Sample size: 15 Standard error of the slope estimate: Do we have enough evidence to infer that β 1 1? (Winter 2012) Simple Regression Lecture / 32

23 Confidence Interval Estimator of β 1 Confidence interval (CI) estimator for β 1 : b 1 ± t (α/2,n 2) se b1 Confidence level: 1 α Degrees of freedom: ν = n 2 (Winter 2012) Simple Regression Lecture / 32

24 Stata Output - Read It! regress income education Source SS df MS Number of obs = F( 1, 148) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = income Coef. Std. Err. t P> t [95% Conf. nterval] education _cons Red box coefficients estimates Blue box standard errors of coefficients Purple box Red box / Blue box = t-statistic Green box p-value Orange box - 95% confidence interval of the slope estimate (Winter 2012) Simple Regression Lecture / 32

25 regress income education Source SS df MS Number of obs = F( 1, 148) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = income Coef. Std. Err. t P> t [95% Conf. nterval] education _cons SST = SSR + SSE MSE = SSE / df, df=n-2 Root MSE = Square root of MSE =s.e. of estimate Stata output provides estimates of the coefficients, results of the tests for significance, confidence intervals, and reports the measure of fit - R-squared. (Winter 2012) Simple Regression Lecture / 32

26 Interpretation of the Results Slope coefficient significance - t-stat or p-value is slope different from zero? is slope different form hypothesized value? interpretation - observational or experimental data observational data - slope has descriptive interpretation experimental data - slope has casual interpretation [not always] R-squared How much variation in dependent variable is explained by independent variable? How large is the standard error of estimate? (Winter 2012) Simple Regression Lecture / 32

27 Drawing Valid Conclusion Let s take a look at income-education regression again: regress income education Source SS df MS Number of obs = F( 1, 148) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = income Coef. Std. Err. t P> t [95% Conf. interval] education _cons Is slope statistically significant? How much variation in income is due to education? Can we conclude that we are 95\% confident that one more year of education produces increase in income from 3.4 thousands to 5 thousands? (Winter 2012) Simple Regression Lecture / 32

28 Biased Estimate Definition: Bias E[b 1 ] β 1. In a simple linear regression model, when we have only one explanatory/independent variable x, we can sign the direction of the bias ( + or ). Sign of the bias depends on the covariance between independent variable (x) and omitted variable (error term ε). As a result, if COV (x i, ε i ) > 0, then E[b 1 ] > β 1. If COV (x i, ε i ) < 0, then E[b 1 ] < β 1. But: we also need to think about the relationship between dependent variable and omitted variable.[not in this course] (Winter 2012) Simple Regression Lecture / 32

29 y OK β β + ε i = 0 + 1x i Assumption # (?) is violated i Ability Education Wage Which arrow will be missing with experimental data? (Winter 2012) Simple Regression Lecture / 32

30 Example Consider again a simple linear regression model of income and education: Income i = β 0 + β 1 Education i + ε i Is b 1 biased? What is the direction of bias in this case? Assume that we have only one omitted variable - Ability. COV(Education, Ability)> 0 E[b 1 ] > β 1 Intuition behind this result? (Winter 2012) Simple Regression Lecture / 32

31 More Examples Try to determine the direction/sign of bias in the following cases: Training employment? Lecture attendance test scores? Number of children hours of work? (Winter 2012) Simple Regression Lecture / 32

32 Direction of Bias - Big Picture [Aside] Cov(x i, ε i ) > 0 Cov(x i, ε i ) < 0 Cov(y i, ε i ) > 0 + bias, or OLS estimate bias, or OLS esti- b 1 of β 1 will be on mate of β 1 will be on average larger than the true value of population parameter average smaller than the true value of population parameter Cov(y i, ε i ) < 0 bias + bias Here, you have to think of Cov(y i, ε i ) as direction of the relationship between dependent variable y and omitted variable which is a part of the error term ε. The correct mathematical expression for the sign of the bias is: Sign [Bias(b 1 )] = Sign [β z ρ(x 1, z)] where z is an omitted variable and β z is the OLS coefficient from regression of y on z: y i = α + β z z i + υ i (Winter 2012) Simple Regression Lecture / 32