Multiple Regression: Inference

Transcription

1 Multiple Regression: Inference

2 The t-test: is ˆ j big and precise enough? We test the null hypothesis: H 0 : β j =0; i.e. test that x j has no effect on y once the other explanatory variables are controlled for. ˆ j will never be 0, but how far is it from 0? Need to weigh the size of the estimate against its sampling error. j We define the t-statistic as: t. ˆ se( ˆ ) Reject H 0 : β j =0 if t is sufficiently large: threshold depends on chosen significance level. Note: we test β j =0, and never. ˆ j j ˆ 0 j

3 Normality provides a benchmark for t-test Assumption 6: Normality The population error u is independent of the explanatory variables x1, x2,, xk and is normally distributed with zero mean and variance σ 2 : u ~ Normal(0, σ 2 ). 4 assumptions =>OLS gives us an unbiased estimate of the coefficient. 4+1 assumptions =>OLS gives us an unbiased estimate of the variance of the coefficient estimate and OLS is efficient (BLUE). 4+2 assumptions (=Classical Linear Model assumptions) =>coefficients have a normal distribution. OLS estimators are the best estimators: smallest variance among ALL estimators, not only the linear ones.

4 0 Density What if normality assumption fails? Example: Crime data, variable narr86 use hist narr86,discrete Non-normality of the errors will not be a problem if: -Large sample size -Log transformation of the dep var. -drop outliers narr86

5 Distribution of OLS estimators 5 Gauss-Markov Assumptions + Normality assumption OLS estimators are normally distributed: Or So under the CLM assumptions,. Careful: this is different from the previous theorem, which involved the constant σ in sd( ˆ j), while in the t-test it is the random variable ˆ. Note: normality of the OLS estimators is still approximately true in large samples even without normality of the errors.

6 Testing against two-sided alternatives: null hypothesis H 0 : β j =0 against H 1 : β j 0. Need to decide on a significance level, or the proba of rejecting H 0 when it is true. Common choice for significance level: 5%. When the alternative is two-sided, we are interested in the absolute value of the t-statistic => rejection rule is: t ˆ >c, where critical value c depends on the significance j level and the degrees of freedom (df = n-k-1) : when df<120: see table G2; when df>120: standard normal critical value.

7

8 Example: Correlates of education Source: WAGE2.dta, Wooldridge Population model to be estimated: educ 0 1sibs 2 feduc 3meduc 4brthord. reg educ sibs meduc feduc brthord u Source SS df MS Number of obs = 663 F( 4, 658) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = educ Coef. Std. Err. t P> t [95% Conf. Interval] sibs meduc feduc brthord _cons

9 Test significancy of one variable: H 0 : β 1 =0 Is the relationship between education and the number of siblings significant? Is significantly different from 0? 1 We can reject the null-hypothesis ( = 0) at the 5% significance level. We see this because the absolute value of the t-stat is: ˆ is the critical value for a two-tailed test at 5% when we have more than 120 degrees of freedom (here n-k-1=658). 1 1 t ˆ1 se( ˆ ) criticalvalue

10 What about the other variables? The coefficient of mother s and father s education is statistically significant at the 1% level and below. (t-stat is larger than 2.576). However, we fail to reject the null-hypothesis that birthorder has no effect on education at the 1, 5, and 10% significance level. Note that this last finding suggests that brthord is an irrelevant variable in the regression. Taking it out of the regression does not have a strong effect on any of the coefficient estimates (see regression below)

11 What happens if you drop one irrelevant variable?. reg educ sibs meduc feduc if brthord!=. Source SS df MS Number of obs = 663 F( 3, 659) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = educ Coef. Std. Err. t P> t [95% Conf. Interval] sibs meduc feduc _cons Note: we see that once the irrelevant variable brthord is not included, the standard error of sibs decreases, and the absolute value of the t-stat increases: sibs is now statistically significant at the 1% level.

12 Practical guidance One can NEVER accept the null hypothesis: When low t-stat (or high p-value), we fail to reject the null hypothesis. Statistical significance (economic) importance t significance ~ t-statistic importance ~ magnitude of coefficient ˆ j se( ˆ ) If coefficient is insignificant (low t value), then no meaningful interpretation of sign and magnitude of coefficient=>just ignore it. Practical advice: with bigger samples, std errors decrease, which results in more statistical significance=>decrease significance level to be sure. ˆ j j

13 Testing against one-sided alternatives: H 0 : β j =0 against H 1 : β j <0. Here we only care about the alternative H 1 : β j <0. Why? Introspection, econ theory We are looking for a sufficiently large negative value of t ˆ in order to reject in favor of H 1 j rejection rule: H 0 is rejected in favor of H 1 if t<-c (or t >c). Remember: to reject H 0 against the negative alternative, we must get a negative t statistic.

14 A few things about the critical value Critical value c smaller than for 2-sided test (see table G2). As significance level falls, the critical value increases. So if H0 is rejected at the 1% level, then it is also rejected at the 5 and 10% levels. Testing H 0 against alternative hypothesis H 1 : β j >0 leads to rejection rule: H 0 is rejected in favor of H 1 if t>c (or t >c).

15 Testing other hypotheses about β j : H 0 : β j =a H 0 : β j =a against H 1 : β j a a = the hypothesized ceteris paribus effect of x on y ˆ j a t-stat can be written as: t ˆ j se( ˆ ) We reject H 0 if t c ˆ. Alternatively: reject H0 if a is not j in the 95% confidence interval: ˆ is statistically different j from a at the 5% significance level. t If H 1 : β j >a, reject if. ˆ j Note: depending on whether one sided or sided alternative, c will not be the same, see G2. c j

16 Confidence intervals for β j Using the fact that, a 95% confidence interval for the population parameter β j is given by: ˆ ˆ ˆ ˆ j c. se( j ), j c. se( j ), where the constant c is the 97.5 th percentile in a t n-k-1 distribution (as before). Ex. (see appendix G2): for df=n-k-1=25, a 95% CI is ˆ ( ˆ ) j se j When df>50, we can consider c 2. Application: H 0 : β j =a is rejected if a not in the 95% CI (the same if a is 0).

17 Example: Rationality of house assessments Source: HPRICE1.dta, Wooldridge. regress lprice lassess Source SS df MS Number of obs = 88 F( 1, 86) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = lprice Coef. Std. Err. t P> t [95% Conf. Interval] lassess _cons Test elasticity of actual price w.r.t. assessed price is zero, i.e. the assessment has no impact on actual price. H 0 : β 1 =0 against H 1 : β 1 0.

18 H 0 : β 1 =0 against H 1 : β 1 0. Compute t-stat to test H 0 : β 1 =0: t ˆ 1 ˆ ˆ1 se( 1) =>greater than critical values at any significance level=>we reject the null hypothesis, we conclude that β 1 0, i.e. the value of the assessed price does impact the value of the actual price of the house. Alternatively, we can see that 0 is not included in the confidence interval at 95% given by stata for the value of ˆ j, so we can reject H 0 at the 5% level (at least).

19 H 0 : β 1 =1 against H 1 : β 1 1. Compute t-stat to test H 0 : β 1 =1: ˆ t 0.22 ˆ1 se( ˆ ) =>smaller than critical values at 1, 5, or even10% levels (c is appr. 1.7 for two-tailed test at 10%)=>we cannot reject the null hypothesis. Alternatively, we could have looked at the 95% confidence interval in the stata output. Given that 1 is in between the interval, we cannot reject H 0 at the 5% significance level.

20 Is the price assessment rational? What if we include other characteristics? We estimate the model: lprice 0 1lassess 2llotsize 3lsqrft 4bdrms u We think that once assessed price is controlled for, the other characteristics should not impact the actual price. =>test 3 null hypotheses: H 0 : β 2 =0; H 0 : β 3 =0; H 0 : β 4 =0. Stata commands: eststo clear eststo: reg lprice lassess eststo: reg lprice lassess llotsize lsqrft bdrms eststo: reg lprice llotsize lsqrft bdrms esttab,r2

21

22 Interpretation of the results Is the house assessment rational? i.e. do other house characteristics impact actual sales price of the house when the assessed price is controlled for? The results in column 2 do not allow to reject the 3 null-hypotheses, and hence provide support for the rational assessment interpretation. Not surprisingly, we see that R-squared does not increase much. House characteristics do not explain much more of the variation in sales prices, once assessed prices are controlled for. Moreover, looking at column three, we see that, as one would expect, we do find significant effects of some the house characteristics on the sales price, if the assessed price of the house is not controlled for. Note also that the coefficient of log(sqrft) is negative in column 2 but positive in column 3. We don t have to be worried about this because we only want to think about the interpretation of the sign if the coefficient is significant. Given that it s insignificant in column 2, the counterintuitive sign in column 2 doesn t matter.

23 P-values for t-tests Given an observed t statistic, what is the smallest significance level at which H 0 would be rejected? = P( T > t ): «p-value for testing H 0 : β j =0 against twosided alternative» with T being a random variable with n-k-1 df t the numerical value of the test statistic = probability of observing a t statistic as large as we did if the null hypothesis is true=>think of it as the proba of rejecting H 0 while H 0 is true. =lowest significance level at which you can reject H 0. Note: to obtain the one-sided p-value: just divide the twosided p-value by 2.

24 Testing multiple/joint linear restrictions: the F-test Testing exclusion restrictions: y 0 1x1 2x2 3x3 4x4 5x5 u H 0 : β 3 =0 ; β 4 =0 ; β 5 =0. H 1 : H 0 does not hold, i.e. x 3, x 4 and x 5 combined have an effect on y. Need to test the restrictions jointly. F-test: estimates the model with (=unrestricted) and without (=restricted) x 3, x 4 and x 5, compare the Sum of Squared residuals: how does SSR increase when we drop these variables? If this increase is big enough, we will reject the joint null hypothesis.

25 The F-test F-statistic: F, where q= nb of restrictions. Under H 0, and assuming CLM assumptions hold, F~F q,n-k-1. =>Reject H 0 if SSR r is relatively large compared to SSR ur, more specifically if F>c, where critical value c depends on the chosen significance level, the number of restrictions q, and the degrees of freedom (n-k-1). (see table G3) SSR SSR r ur SSR n k ur q 1 Terminology: if H 0 is rejected, x 3, x 4 and x 5 are jointly statistically significant.

26 Notes on F-test The F-stat is always 0. Even if the t-tests on each coeff conclude x 3, x 4 and x 5 are individually statistically insignificant, it may be that x 3, x 4 and x 5 are jointly statistically significant (e.g. due to multicollinearity). Be careful when comparing two models: the same observations should be used=>careful to missing values!

27 Ex: Effect of personal characteristics and marriage characteristics on the number of extramarital affairs Source: affairs.dta (Wooldridge), data originally used in R.C. Fair (1978), "A Theory of Extramarital Affairs," Journal of Political Economy 86, 45-61, desc naffairs age educ occup yrsmarr ratemarr storage display value variable name type format label variable label naffairs byte %9.0g number of affairs within last year age float %9.0g in years educ byte %9.0g years schooling occup byte %9.0g occupation, reverse Hollingshead scale yrsmarr float %9.0g years married ratemarr byte %9.0g 5 = vry hap marr, 4 = hap than avg, 3 = avg, 2 = smewht unhap, 1 = vry unhap. sum naffairs age educ occup yrsmarr ratemarr Variable Obs Mean Std. Dev. Min Max naffairs age educ occup yrsmarr ratemarr

28 . tab naffairs number of affairs within last year Freq. Percent Cum.. tab ratemarr Total = vry hap marr, 4 = hap than avg, 3 = avg, 2 = smewht unhap, 1 = vry unhap Freq. Percent Cum Total

29 . regress naffairs age educ occup yrsmarr ratemarr Source SS df MS Number of obs = 601 F( 5, 595) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = naffairs Coef. Std. Err. t P> t [95% Conf. Interval] age educ occup yrsmarr ratemarr _cons regress naffairs yrsmarr ratemarr Source SS df MS Number of obs = 601 F( 2, 598) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = naffairs Coef. Std. Err. t P> t [95% Conf. Interval] yrsmarr ratemarr _cons

30 . esttab,r2 scalars(rss df_r) age * (-2.46) educ (-0.01) occup (1.50) (1) (2) naffairs naffairs yrsmarr 0.144*** ** (3.87) (3.15) ratemarr *** *** (-6.26) (-6.20) _cons 4.529*** 3.769*** (4.25) (6.65) N R-sq rss df_r t statistics in parentheses * p<0.05, ** p<0.01, *** p<0.001

31 Interpretation: These two regression estimates allow you to test whether individual characteristics jointly have an effect on the number of affairs a person has in a year. In other words, are affairs mainly explained by characteristics of the marriage itself, or do individual characteristics play a role? H 0 : β 1 =0, β 2 =0, β 3 =0. In order to figure this out, we estimate a regression including both individual and marriage characteristics (the unrestricted model), and one with only marriage characteristics (the restricted model). We use an F-test to test the exclusion restrictions. We obtain SSR ur = 5846, SSR r = 5921, q =3 (number of restrictions), n-k-1 = 595 (degrees of freedom) F = [(SSR r - SSR ur )/q]/[ssr ur /n-k-1] =2.55. The critical value for q=3 and n-k-1=595, at the 5% level, is Hence we cannot reject the null-hypothesis at the 5% significance level. We can however reject the null-hypothesis at the 10% level (critical value is 2.08).

32 Notes on F-test R-squared form of F-stat: because SSR=SST(1-R 2 ), F 2 ( R (1 R P-values for F-test: probability of observing a value of F as large as we did, given H 0 is true=>to reject H 0, p-value has to be low. F-stat for overall significance of a regression: H 0 : β 1 = β 2 = = β k =0 ur 2 ur 2 R r ) q ) n k 1 Non-zero hypotheses can be incorporated in a F-test. F-test for 1 restriction (β j =0) two-sided t-test.

33 Ex. 2: Effect of mother s and father s education on wage. eststo clear. eststo:regress wage educ IQ meduc feduc Source SS df MS Number of obs = 722 F( 4, 717) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = wage Coef. Std. Err. t P> t [95% Conf. Interval] educ IQ meduc feduc _cons test (meduc=0) (feduc=0) ( 1) meduc = 0 ( 2) feduc = 0 F( 2, 717) = 4.26 Prob > F =

34 Interpretation: We want to test whether mother and father s education have a jointly significant effect on wage of the child. H 0 : β meduc =0, β feduc =0. we estimate the unrestricted model and ask stata to do the F-test. Stata indicates there are 2 restrictions, 717 degrees of freedom, and the calculated F-value is It also indicates the P-value of the F-test: Hence we can reject the null-hypothesis at the 5% level but not at the 1%. To be more precise, the probability of observing an F-value of 4.26 when the null-hypothesis holds is 1.45%. Note that the significance of this test is much higher than for the individual t-tests for these parameters. This can be explained by multicollinearity. We could have obtained the same result by estimating the restricted model, and using the R-squared form of the F-statistic. Note that we need to be careful to estimate the restricted model on the same observations (i.e. excluding those for which meduc or feduc have missing values, see below)

35 . eststo:regress wage educ IQ if e(sample) Source SS df MS Number of obs = 722 F( 2, 719) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = esttab,r2 scalars(rss df_r) educ 32.32*** 39.66*** (4.09) (5.27) IQ 4.717*** 5.338*** (4.08) (4.68) meduc (1.25) feduc (1.71) (1) (2) wage wage _cons (-1.17) (-1.03) N R-sq rss df_r t statistics in parentheses * p<0.05, ** p<0.01, *** p<0.001 F ( R (1 R 2 ur 2 ur R 2 r )/ q )/( n k 1) ( )/2 ( )/(717) 4.25 which is larger than 3.00, the critical value at 5%. The regression output shows that the F- value for the test of overall significance of the regression is very high, Indeed the P-value shows significance at very low levels.

36 Test H 0 : linear combination of the parameters=0 H 0 : β 1 =β 2 against H 1 :β 1 β 2. Method 1: test H 0 : β 1 -β 2 =0 Need t-stat: 1 2. =>Need to compute the denominator: with s 12 the estimate of the covariance. Note: ˆ ˆ t se ( ˆ ˆ ) se( ˆ ) se( ˆ ) 1/ 2 se( ˆ ˆ s 1 2) se( ˆ ˆ ) ( ˆ ) ( ˆ 1 2 se 1 se 2)

37 Ex2b: Effect of mother s and father s education on wage. regres wage educ IQ meduc feduc Source SS df MS Number of obs = 722 F( 4, 717) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = wage Coef. Std. Err. t P> t [95% Conf. Interval] educ IQ meduc feduc _cons test meduc = feduc ( 1) meduc - feduc = 0 F( 1, 717) = 0.02 Prob > F =

38 Ex 3: Effect of years of tenure in company and years of experience on wage wage 0 1exper 2tenure 3educ 4IQ u. regress wage exper tenure educ IQ Source SS df MS Number of obs = 935 F( 4, 930) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = wage Coef. Std. Err. t P> t [95% Conf. Interval] exper tenure educ IQ _cons test exper==tenure ( 1) exper - tenure = 0 F( 1, 930) = 2.84 Prob > F =

39 Test H 0 : linear combination of parameters=0 (2) H 0 : β 1 =β 2 against H 1 :β 1 β 2. 1 Method 2: define 1 2 Test H 0 : 0 versus H 1 : =>use standard t-test Need to redefine variables given that: =>Estimate y 0 ( 1 2) x1... k xk u y x ( x x ) k xk u Estimating this model allows us to test H 0 : 0 1 using a simple t-test on one variable (here x 1 ).

40 Ex 3b: Effect of years of tenure in company and years of experience on wage The new model estimated, using method 2, is: y 0 1 exp er 2(exp er tenure ) 3educ 4IQ u. gen sum = exper+tenure. regress wage exper sum educ IQ Source SS df MS Number of obs = 935 F( 4, 930) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = wage Coef. Std. Err. t P> t [95% Conf. Interval] exper sum educ IQ _cons

41 Interpretation: we have reformulated the problem in such a way that we can look at the t-test for the first variable (exper), and use it to test the null-hypothesis. The t-test shows we can reject the null hypothesis at the 10% level but not at the 5%. Note that value of t-test is square root of value of F-test shown earlier: equals 2.84 (approximately). Of course, we should be careful when interpreting the coefficient estimates. To know the total effect of experience, we add up the coefficients of exper and sum. This shows the same effect than previously.

42 Can we use regression outputs to test joint hypotheses?

43 Can we test whether log(lotsize), log(sqrft) and bdrms jointly have a significant effect, once the assessed price is controlled for? Yes, given that we have the R-squared of the restricted and the unrestricted model, and the number of observations, we can calculate the R-squared form of the F-test. Can we test whether price assessments are rational, when we define rationality as? The rationality hypothesis can be restated in these terms: a 1% change in assess would be associated with a 1% change in price; that is β 1 =1. In addition, lotsize, sqrft, and bdrms should not help to explain log(price), once the assessed value has been controlled for. Answer: No, we would need to have access to the data, because one of the coefficients is hypothesized not to equal zero.

44 How can we test for the latter joint hypothesis? There are four restrictions to be tested, three are exclusion restrictions, but β 1 =1 is not. How can we test this hypothesis using F-stat? =>estimate unrestricted and restricted models. Unrestricted model: versus restricted model: The F-stat is simply: y 0 1x1 3x3 4x4 y x u ( SSR 1 0 r SSR /( n 5) ) / 4 The 5% critical value in a F distribution with (4, 83) df is about 2.50, so we fail to reject H0. There is no evidence that the assessed values are not rational. ur SSR ur u ( ) / / 83