Statistical Inference. Part IV. Statistical Inference

Transcription

1 Part IV Statistical Inference As of Oct 5, 2017

2 Sampling Distributions of the OLS Estimator 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

3 Sampling Distributions of the OLS Estimator Regression model y i = β 0 + β 1 x i1 + + β k x ik + u i. (1) To the assumptions 1 5 we add Assumption 6: The error component u is independent of x 1,... x k and u N(0, σ 2 u). (2)

4 Sampling Distributions of the OLS Estimator Remark 4.1: Assumption 6 implies E[u x 1,..., x k ] = E[u] = 0 and var[u x 1,..., x k ] = var[u] = σ 2 u, i.e Assumptions 1 and 2. Remark 4.2: Assumption 3, i.e., cov[u i, u j ] = 0 together with Assumption 6 implies that u 1,..., u n are independent. Remark 4.3: Under assumptions 1 6 the OLS estimators ˆβ 1,..., ˆβ k are Minimum Variance Unbiased Estimators (MVUE). That is they are best among all unbiased estimators (not only linear). Remark 4.4: y x N(β 0 + β 1 x β k x k, σ 2 u), (3) where x = (x 1,..., x k ) and y x means conditional on x.

5 Sampling Distributions of the OLS Estimator Theorem 4.1: Under the assumptions 1 6, conditional on the sample values of the explanatory variables ˆβ j N(β j, σ 2ˆβ j ) (4) and therefore ˆβ j β j N(0, 1), (5) σ ˆβ j [ ] [ ] where σ 2ˆβ = var ˆβj and σ j ˆβj = var ˆβj.

6 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

7 Theorem 4.2: Under the assumptions 1 6 ˆβ j β j s ˆβ j t n k 1, (6) (the t-distribution with n k 1 degrees of freedom) where s ˆβ j = se( ˆβ j ) and k + 1 is the number of estimated regression coefficients. Remark 4.5: The only difference between (5) and (6) is that in the latter the standard deviation parameter σ ˆβ j is replaced by its estimator s ˆβ j.

8 In most applications the interest lies in testing the null hypothesis: H 0 : β j = 0. (7) The t-test statistic is t ˆβ j = ˆβ j s ˆβj, (8) which is t-distributed with n k 1 degrees of freedom if the null hypothesis is true. These t-ratios are printed in standard computer output in regression applications.

9 Example 4.1: Wage example computer output. Dependent Variable: LOG(WAGE) Method: Least Squares Sample: Included observations: 526 Variable Coefficient Std. Error t-statistic Prob. C EDUC R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid Schwarz criterion Log likelihood F-statistic Durbin-Watson stat Prob(F-statistic)

10 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

11 One-sided alternatives or H 1 : β j > 0 (9) H 1 : β j < 0. (10) In the former the rejection rule is to reject the null hypothesis at the chosen significance level, α t ˆβ j > c α, (11) where c α is the 1 α fractile (or percentile) from the t-distribution with n k 1 degrees of freedom, such that P(t ˆβ j > c α H 0 is true) = α. α is called the significance level of the test. Typically α is 0.05 or 0.01, i.e., 5% or 1%. In the case of (10) the H 0 is rejected if t ˆβj < c α. (12)

12 Exploring Further 4.2 (W 5ed, p. 118) (adapted) Community loan approval rates, apprate, are modeled as apprate = β 0 + β 1 percmin + β 3 avginc + β 4 avgwlth + β 5 avgdebt + u, where percmin is the percentage minority in the community, avginc is average income, avgwlth is average wealth, and avgdept is average dept obligations. Suppose we are interested to investigate whether there is difference in loan approval rates across neighborhoods due to racial and ethnic composition, when average income, average wealth, and average dept have been controlled for. We are in particular interested whether there is discrimination against minorities in loan approval rates. How would you state the null hypothesis in terms of the above model? How do you state the alternative hypothesis?

13 Example 4.2: In the wage example, test H 0 : β exper = 0 against H 1 : β exper > 0. =============================== log(wage) Coeff s.e Intercept educ exper tenure =============================== N = 526 Thus, ˆβ exper = , s ˆβ exper = and t ˆβ exper = ˆβ exper s ˆβexper = , and because 2.39 > 2.33 = c.01 (from t-distribution with 522 df), we reject the null hypothesis and infer that experience affects positively on wages.

14 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

15 If the null hypothesis is H 0 : β j = 0, the two-sided alternative is H 1 : β j 0. (13) The null hypothesis is rejected at the significance level α if t ˆβ j > c α/2. (14)

16 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

17 Generally the null hypothesis can be also H 0 : β j = βj, (15) where βj is some given value (for example βj = 1, so H 0 : β j = 1). The test statistic is again a t-statistic t = ˆβ j β j s ˆβj. (16) Under the null hypothesis (15) the test statistic (16) is again t-distributed with n k 1 degrees of freedom. Thus t measures again how many (estimated) standard errors the estimated value, ˆβ j, is away from the hypothesized value, β j, of β j.

18 The general t statistic is usefully written as t = (estimate hypothesized value). (17) standard error Remark 4.7: The computer print outs always give the t-ratios, i.e., test against zero. Consequently, they cannot be used to test the more general hypothesis (15). You can, however, use the standard errors and compute the test statistics of the form (16).

19 Example 4.3: Housing prices and air pollution. A sample of 506 communities in Boston area. Variables: price (y) = median housing price nox (x 1 ) = nitrogen oxide, parts per 100 mill. dist (x 2 ) = weighted dist. to 5 employ centers rooms (x 3 ) = avg number of rooms per house stratio (x 4 ) = average student-teacher ratio of schools in community Specified model log(y) = β 0 + β 1 log(x 1 ) + β 2 log(x 2 ) + β 3 x 3 + β 4 x 4 + u (18) β 1 is the price elasticity of nox. We wish to test H 0 : β 1 = 1 against H 1 : β 1 1.

20 Estimation results: Dependent Variable: LOG(PRICE) Method: Least Squares Sample: Included observations: 506 Variable Coefficient Std. Error t-statistic Prob. C LOG(NOX) LOG(DIST) ROOMS STRATIO R-squared Mean dependent var Adjusted R-squared S.D. dependent var S.E. of regression Akaike info criterion Sum squared resid Schwarz criterion Log likelihood F-statistic Durbin-Watson stat Prob(F-statistic) t = ( 1) = t 501 (0.025) z(0.025) = 1.96, which is far higher than the test statistic. Thus we do not reject the null hypothesis and conclude that there is not empirical evidence that the elasticity would differ from -1.

21 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

22 The p-value is defined as the smallest significance level at which the null-hypothesis could be rejected. Thus we can base our inference on the p-value instead of finding from the tables the critical values. The decision rule simply is that if the p-value is smaller than the selected significance level α we reject the null hypothesis.

23 Technically the p-value is calculated as the probability P(T > t obs H 0), if the alternative hypothesis is H 1 : β > β p = P(T < t obs H 0), if the alternative hypothesis is H 1 : β < β P( T > t obs H 0), if the alternative hypothesis is H 1 : β β (19) where T is a t-distributed random variable and t obs is the value of t-statistic calculated form the sample (observed t-statistic). Remark 4.8: The computer output contains p-values for the null hypothesis that the coefficient is zero and the alternative hypothesis is that it differs form zero (two-sided). Question: If the p-value for hypothesis H 0 : β = 0 versus H 1 : β 0 is p = 0.07, what is the p-value if the alternative hypothesis is H 1 : β > 0?

24 Example 4.4: In the previous example the p-values indicate that all the coefficient estimates differ (highly) statistically significantly from zero. For the null hypothesis H 0 : β 1 = 1 with the alternative hypothesis H 1 : β 1 1 p-value is obtained by using the standardized normal distribution as 2(1 Φ(0.398)) 0.691, where Φ(z) is the cumulative distribution function of the standardized normal distribution.

25 Confidence Intervals for the Coefficients 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

26 Confidence Intervals for the Coefficients From the fact that ˆβ j β j s ˆβj t n k 1 (20) we get for example a 95% confidence interval for the unknown parameter β j as ˆβ j ± c 1 2 αs ˆβ j, (21) where c α/2 is again the 1 α/2 fractile of the appropriate t-distribution. Interpretation!

27 Confidence Intervals for the Coefficients Example 4.5: The 95% confidence interval for β nox = β 1 is or ˆβ nox ± c.025 s ˆβnox = ± (22) = ± ( 1.182, 0.725). (23) We observe that 1 ( 1.182, 0.725).

28 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

29 Hypotheses H 0 : β j = 0 test whether a single coefficient is zero, i.e. whether variable x j has marginal impact on y. Hypothesis H 0 : β 1 = β 2 = = β k = 0 (24) tests whether none of the x-variables affect y. I.e., whether the model is y = β 0 + u instead of The alternative hypothesis is y = β 0 + β 1 x β k x k + u. H 1 : at least one β j 0. (25)

30 Null hypothesis (24) is tested by the F -statistic, called the F -statistic for overall significance of a regression F = SSE/k SSR/(n k 1), (26) which under the null hypothesis is F -distributed with k and n k 1 degrees of freedom. This is again printed in the standard computer output of regression analysis. Example 4.6 In the house price example F = with p-value , which is highly significant as would be expected.

31 The principle of the F -test can be used to test more general (linear) hypotheses. For example to test whether the last q variables contribute y, the null hypothesis is H 0 : β k q+1 = β k q+2 = = β k = 0. (27) The restricted model satisfying the null hypothesis is y = β 0 + β 1 x β k q x k q + u (28) with k q explanatory variables, and the unrestricted model is y = β 0 + β 1 x β k x k + u (29) with k explanatory variables. Thus the restricted model is a special case of the unrestricted one.

32 The F -statistic is F = (SSR r SSR ur )/q SSR ur /(n k 1), (30) where SSR r is the residual sum of squares from the restricted model (28) and SSR u is the residual sum of squares for the unrestricted model (29). Under the null hypothesis the test statistic (30) is again F -distributed with q = df r df ur and n k 1 degrees of freedom, where df r is the degrees of freedom of SSR r and df ur is the degrees of freedom of SSR ur.

33 Remark 4.9: Testing for single is a special case of (27), and it can be shown that in such a case the F -statistic from (30) equals t 2ˆβ j. Remark 4.10: It can be easily shown that F = (R 2 ur R 2 r )/q (1 R 2 ur )/(n k 1), (31) where Rur 2 and Rr 2 are the R-squares of the unrestricted and restricted models, respectively.

34 Exploring Further 4.4 (W 5ed, p. 138) Consider relating individual performance on a standardized test, score, to a variety of other variables. School factor include average class size, per-student expenditures, average teacher compensation, and total school enrollment. Other variables specific to the student are family income, mother s education, father s education, and number of siblings. The model is score = β 0 + β 1 classsize + β 3 tchcomp + β 4 endoll + β 5 faminc + β 6 motheduc + β 7 fatheduc + β 8 siblings + u State the null hypothesis that student-specific variables have no effect on standardized test performance once school-specific factors have been controlled for. What are k and q for this example? Write down the restricted version of the model.

35 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

36 The principle used in constructing the F -test in (30) can be extended for testing general linear restrictions between the parameters. As an example, consider the regression model If the hypothesis is y = β 0 + β 1 x 1 + β 2 x 2 + β 3 x 3 + u. (32) we can set, for example β 3 = 1 β 1 β 2. H 0 : β 1 + β 2 + β 3 = 1, (33)

37 Then we can estimate β 1 and β 2 from y x 3 = β 0 + β 1 (x 1 x 3 ) + β 2 (x 2 x 3 ) + u (34) and estimates β 1 and β 1 compute the RSS r = n (y i ỹ i ) 2, (35) i=1 where ỹ i = β 0 + β 1 x i1 + β 2 x i2 + (1 β 1 β 2 )x i3. (36) In the restricted model one parameter less is estimated than in the unrestricted case. Thus the degrees of freedom in the F -statistic are 1 and n k 1

38 A more straightforward approach to test hypothesis (33) is to estimate regression y = β 0 + β 1 (x 1 x 3 ) + β 2 (x 2 x 3 ) + θx 3 + u (37) which is statistically equivalent to regression (32). The null hypothesis in (33) becomes then H 0 : θ = 1. (38)

39 Example 4.7 Consider the wage example log(wage) = β 0 + β educ educ + β exper exper + β tenure tenure + u (39) We want to test whether experience and tenure affect equally on wages. To answer the question we test the hypothesis H 0 : β exper = β tenure. (40) We use the wage1 data set to estimate the relevant models to test the hypothesis (will be demonstrated in the class room).

40 On Reporting the Regression Results 1 Statistical Inference Sampling Distributions of the OLS Estimator Testing Against One-Sided Alternatives Two-Sided Alternatives Other Hypotheses About β j p-values Confidence Intervals for the Coefficients Testing General Linear Restrictions On Reporting the Regression Results

41 On Reporting the Regression Results (1) Estimated coefficients and interpret them (2) Standard errors (or t-ratios or p-values) (3) R-squared and number of observations (4) Optionally, standard error of regression