Lecture 6 Multiple Linear Regression, cont.

Transcription

1 Lecture 6 Multiple Linear Regression, cont. BIOST 515 January 22, 2004 BIOST 515, Lecture 6

2 Testing general linear hypotheses Suppose we are interested in testing linear combinations of the regression coefficients. For example, we might be interested in testing whether two regression coefficients are equal H 0 : β i = β j. Equivalently, H 0 : β i β j = 0. Such hypotheses can be expressed as H 0 : T β = 0, where T is an m p matrix of constants, such that only r of the m equations in T β = 0 are independent. BIOST 515, Lecture 6 1

3 For example, consider the model y i = β 0 + x i1 β 1 + x i2 β 2 + x i3 β 3 + ɛ i and testing the hypothesis H 0 : β 1 β 2 = 0. This hypothesis is equivalent to H 0 : ( )β = 0. We may also consider the hypothesis H 0 : β 1 β 2 = 0, β 3 = 0 BIOST 515, Lecture 6 2

4 which is equivalent to H 0 : T β = 0 where T = ( ). BIOST 515, Lecture 6 3

5 We can use sums of squares to test general linear hypotheses. The full model is y = Xβ + ɛ with residual sum of squares SSE(F M) = y y ˆβ X y (n p degrees of freedom). Obtain the reduced model by solving T β = 0 for r of the regression coefficients in the full model in terms of the remaining p + 1 r regression coefficients. Substitutin these values into the full model will yield the reduced model, y = Zγ + ɛ, where Z is an n (p + 1 r) matrix and γ is a (p + 1 r) 1 vector of unknown regression coefficients. The residual sum of BIOST 515, Lecture 6 4

6 squares for the reduced model is SSE(RM) = y y ˆγZ y (n p + r degrees of freedom) SSE(RM) SSE(F M) is called the sum of squares due to the hypothesis T β = 0. We can test this hypthesis using F 0 = (SSE(RM) SSE(F M))/r MSE F r,n p 1. BIOST 515, Lecture 6 5

7 CHS smoking example Recall the example where smoking status was recoded to smoke 1i = { 1, never smoked 0, otherwise and smoke 2i = and we fit the model { 1, former smoker 0, otherwise, BP i = β 0 + β 1 smoke 1i + β 2 smoke 2i + ɛ i. Estimate Std. Error t value Pr(> t ) (Intercept) smoke smoke BIOST 515, Lecture 6 6

8 We may be interested in testing H 0 : β 1 = β 2 which is equivalent to testing H 0 : ( )β The full model is BP i = β 0 + β 1 smoke 1i + β 2 smoke 2i + ɛ i, andthe reduced model is BP i = β 0 + β 1 smoke 1i + β 1 smoke 2i + ɛ i = β 0 + β 1 (smoke 1i + smoke 2i ) + ɛ i = γ 0 + γ 1 z i + ɛ i The reduced model is equivalent to the model we fit with current smokers vs. former and never smokers. BIOST 515, Lecture 6 7

9 Full model: Df Sum Sq Mean Sq F value Pr(>F) smoke smoke Residuals Reduced model: Df Sum Sq Mean Sq F value Pr(>F) smoker Residuals ( )/1 F 0 = = 0.11 < Therefore we fail to reject the null hypothesis. BIOST 515, Lecture 6 8

10 We could also test this hypothesis using the t statistic ˆσ2 (C 11 + C 22 2C 12) t 0 = ˆβ 1 ˆβ 2 ŝe( ˆβ 1 ˆβ 2 ) = ˆβ1 ˆβ 2 where C = Therefore t 0 = ( ) ( ) =.335 < t n p 1,.975 BIOST 515, Lecture 6 9

11 Consider the model BP i = β 0 + β 1 smoke 1i + β 2 smoke 2i + β 3 age i + ɛ i. Df Sum Sq Mean Sq F value Pr(>F) smoke smoke AGE Residuals Suppose we want to test H 0 : β 1 = β 2, β 3 = 0 BIOST 515, Lecture 6 10

12 which is equivalent to H 0 : The reduced model is ( ) β = 0. BP i = β 0 + β 1 (smoke 1i + smoke 2i ) + ɛ i = γ 0 + γ 1 z i + ɛ i ( )/2 F 0 = We reject the null hypothesis. = > F 2,494,.95 = BIOST 515, Lecture 6 11

13 Confidence intervals in multiple linear regression Confidence interval for a single coefficient Confidence interval for a fitted value Simultaneous confidence intervals on multiple coefficients BIOST 515, Lecture 6 12

14 Confidence interval for a single coefficient We can construct a confidence interval for β j Given that ˆβ j β j ŝe( ˆβ j ) = ˆβ j β j t n p 1, ˆσ2 C jj as follows. we can define a 100(1 α) confidence interval for β j as ˆβ j ± t n p 1,α/2 ˆσ 2 C jj. BIOST 515, Lecture 6 13

15 Confidence interval for a fitted value We can construct a confidence interval for the fitted response for a set of predictor values, x 01, x 02,..., x 0p. Define the vector x 0 as x 0 = The fitted value at this point is 1 x 01 x 02. x 0p ŷ 0 = x 0 ˆβ.. BIOST 515, Lecture 6 14

16 ŷ 0 is an unbiased estimator of E(y x 0 ), and the variance of ŷ 0 is var(ŷ 0 ) = σ 2 x 0(X X) 1 x 0. Therefore, the 100(1 α)% confidence interval for the fitted response at x 01, x 02,..., x 0p is ŷ 0 ± t n p 1,α/2 ˆσ 2 x 0 (X X) 1 x 0. BIOST 515, Lecture 6 15

17 Example from CHS In the last lecture, we fit the model BP i = β 0 + weight i β 1 + height i β 2 + age i β 3 + gender i β 4 + ɛ. Let s calculate the confidence interval for the fitted value for the 100th subject who has the covariate vector ( ). The fitted value for BP is and x 0(X X) 1 x 0 = and the 95% confidence interval is ± = (71.72, 75.62). BIOST 515, Lecture 6 16

18 Age Height Weight Confidence interval for BP BIOST 515, Lecture 6 17

19 Simultaneous confidence intervals Sometimes we may be interested in specifying a (1 α)100% confidence interval (or region) for the entire set or a subset of the coefficients. ( ˆβ β) X X( ˆβ β) F p+1,n p 1 (p + 1)MSE Therefore, we can define a (1 α)100% joint confidence region for all the parameters in β as ( ˆβ β) X X( ˆβ β) (p + 1)MSE F p+1,n p 1 BIOST 515, Lecture 6 18

20 Bonferroni intervals Another general pproach for obtaining simultaneous confidence intervals is ˆβ j ± ŝe( ˆβ j ), j = 0, 1,..., p. (1) Using the Bonferroni method, we set = t n p 1,α/(2(p+1)) leading to a Bonferroni confidence interval of ˆβ j ± t n p 1,α/(2(p+1)) ŝe( ˆβ j ). BIOST 515, Lecture 6 19

21 Bonferroni intervals CHS example BP i = β 0 + weight i β 1 + height i β 2 + age i β 3 + gender i β 4 + ɛ. The Bonferroni intervals are ˆβ j ± t 493,.005 ŝe( ˆβ j ) Lower Upper (Intercept) WEIGHT HEIGHT AGE GENDER BIOST 515, Lecture 6 20

22 Hidden extrapolation in multiple regression Joint region of original data x 2 x 0 2 x 0 1 x 1 BIOST 515, Lecture 6 21

23 R 2 and adjusted R 2 As in simple linear regression R 2 = 1 SSE SST O. In general, R 2 increases whenever new terms are added to the model. Therefore, for model comparison, we may prefer to use an R 2 that is adjusted for the number of predictors in the model. This is the adjusted R 2 and is equivalent to R 2 adj = 1 MSE SST O/(n 1) BIOST 515, Lecture 6 22

24 Predictors R 2 Radj 2 weight, height, age, gender smoke1, smoke weight, height smoke1, smoke2, age BIOST 515, Lecture 6 23