Module 19: Simple Linear Regression

Transcription

1 Module 19: Simple Linear Regression This module focuses on simple linear regression and thus begins the process of exploring one of the more used and powerful statistical tools. Reviewed 11 May 05 /MODULE

2 Goldman-Tono-Pen Example An ophthalmologist who is assessing intraocular pressures as a part of a community program for the prevention of glaucoma is interested in using a portable device (Tono- Pen) for making these measurements. An important question is how well the measurements made with this device compare to those made with a more standard device (Goldman) used in clinical settings. To address this question, the ophthalmologist compared the two devices by using each on n = 40 eyes. For this comparison, each eye was measured once with each device. 19-2

3 Goldman-Tono-Pen Example Data ID Goldman T-Pen ID Goldman T-Pen

4 Comparing the two Devices One approach to comparing the two devices would be to do a paired t-test, which would be appropriate since the measurements made by the two devices on the same eyes could not be considered independent and since the differences between the two measurements are of interest. 19-4

5 Goldman-Tono-Pen Worksheet Goldman Tono-Pen Goldman Tono-Pen ID x = G y = T d d 2 ID x = G y = T d d Sum Mean

6 Goldman Tono-Pen ID X=G Y=T d=g-t N Sum Mean SD Sum 2 /n 15, , Sum(x 2 ) 16,497 15, SS s SE t = mean(d)/se(d) 1.58 df = n-1 39 t (39) 2.02 d

7 1. Hypothesis: H 0 : Δ = μ G - μ T = 0 vs. H 1 : Δ 0, 2. Assumptions: Differences are a random sample with normal distribution, 3. The α level: α = 0.05, 4. Test statistic: 5. The Rejection Region: Reject if t is not between ±t (39)= The Result: d t = = s n d / d 7. The conclusion: Accept H 0 : Δ = μ G - μ T = 0, since t is between ± d s n= 40, d = 0.8, s d = t = =

8 Hence, from this standpoint, we do not have compelling evidence that the two devices are measuring intra-ocular pressures differently. Is this a sufficient assessment of the situation, or should we look further? 19-8

9 Looking Further One way to look further at this situation is to think about the relationship between the measurements made by the two machines in terms of simple linear regression. In this context, we would wonder if higher values on one machine more directly imply higher values on the other. Simple linear regression focuses on a possible straight line relationship between the measurements made by the two machines. 19-9

10 Simple Linear Regression Concepts In general, simple linear regression finds the best straight line for describing the relationship between two variables. In its simplest form, which is what we consider here, it does not do a very good job of assessing how well the line describes the data, but nevertheless provides useful information

11 y-axis Dependent variable y = a + bx b units of y a 1 unit of x 0 x-axis Independent Variable a = Intercept, that is, the point where the line crosses the y-axis, which is the value of y at x = 0. b = Slope of the regression line, that is, the number of units of increase (positive slope) or decrease (negative slope) in y for each unit increase in x

12 The Regression Line l 3 l 5 Y dependent variable l 1 l 2 l X independent variable 19-12

13 14 12 d 3 Y dependent variable d 1 d2 d 4 d X independent variable 19-13

14 14 12 l 3 d 3 l 5 Y dependent variable l 1 d1 l 2 d 2 l 4 d 4 d X independent variable 19-14

15 The context for simple linear regression is that we have a random sample of persons from a set of well-defined populations, each defined by a specific value for x- variable. We have measurements of another variable, the y-variable so that we have two variables for each person. For simple linear regression, we focus on a straight line that depicts the relationship between these two variables. The best straight line is the one for which the sum of the squared vertical distances of each point from the line is the least. This "least squares" line has slope and intercept xy x y / n b = = 2 2 x ( x) / n a = y bx. SS ( xy) SS ( x), 19-15

16 For this situation, the sample line y = a + b x is an estimate of the population line Y = α + β x, and a and b are estimates of α and β respectively. For a specific value of x, such as x = 10, the value for y calculated from the regression equation is yˆ = a+ b( x= 10), which is called the regression estimate of Y at the value x =

17 Simple Regression Example The following data are diastolic blood pressure (DBP) measurements taken at different times after an intervention for n = 5 persons. For each person, the data available include the time of the measurement and the DBP level. Of interest is the relationship between these two variables

18 n Mean 3,310 22, Sum 1,320 4, , , , , xy y 2 y x 2 x Patient DPB Time

19 For the blood pressure data, x = y = the slope is 50 / 5 = 10, 338 / 5 = 67.6, xy x y / n b = = 2 2 x ( x) / n SS ( xy) SS ( x), and the intercept is b 3,310 (50)(338) / 5 = = (50) / 5 a = y bx, a = 67.6 ( 0.28)10 = 70.4 The best line is y = a + bx = x 19-19

20 Time DBP Patient x y Diastolic Blood Pressure y y = x Minutes x 19-20

21 Example: AJPH, Dec. 2003; 93:

22 19-22

23 Never Smoking Regression Worksheet Year (x) Female (y 1 ) Male (y 2 ) x 2 xy 1 xy 2 y y 2 2 Total Mean Sum Num b Denum b b a

24 For the never smoking data x = / 9 = y = / 9 = , female y = / 9 = male The slopes are b xy x y/ n SS( xy) = =, ( ) / ( ) 2 2 x x n SS x (( )(603.92)/9) b female = = (( ) / 9) (( )(580.51)/9) b male = = (( ) / 9)

25 The intercepts are a= y bx, a = (0.285* ) = female a = (0.871* ) = male The best lines are: y = a + b x= x female female female y = a + b x= x male male male 19-25

26 75 70 y female = x Percentage Never Smokers Female (Y1) Male (Y2) Female (Line) Male (Line) y male = x Year 19-26

27 Regression ANOVA If the regression line is flat in the sense that the regression estimate of Y, being ŷ, is the same for all values of x, then there is no gain from considering the x variable as it is having no impact on ŷ. This situation occurs when the estimated slope b = 0. An important question is whether or not the population parameter β = 0, that is, whether the truth is that there is no linear relationship between y and x. To test this situation, we can proceed with a formal test

28 1. The Hypothesis: H 0 : β = 0 vs H 1 : β 0 2. The α level: α = The assumptions: Random normal samples for y- variable from populations defined by x-variable 4. The test statistic: ANOVA Source df SS MS F Regression 1 SS(Reg ) SS(Reg )/1 MS(Reg )/MS(Res) Residual n-2 SS(Res ) SS(Res )/(n-2) Total n-1 SS(y) 5. The rejection region : Reject H 0 : β = 0 if the value calculated for F is greater than F 0.95 (1, n-2) 19-28

29 2 R = SS ( Reg) / SS ( Total) R 2 is the total amount of variation in the dependent variable y explained by its regression relationship with x

30 Blood Pressure Example SS( Total) = SS( y) = ( y y) (338 ) = 22,892 5 SS( Regression) = bss( xy) 2 2 = 43.2 = b{ xy x y/ n} = 0.28{3310 (50)(338) / 5} = 19.6 SS( Residual) = SS( Total) SS( Regression) = =

31 ANOVA Source df SS MS F Regression Residual Total H 0 : β = 0 vs H 1 : β 0 For α = 0.05 F 0.95(1,3) = 10.1, Hence accept H 0 : β = 0 2 SS ( Regression ) 19.6 R = = = SS ( Total ) 43.2 or 45.37% Note: The above hypothesis test does not asses how well the straight line fits the data

32 Goldman-Tono-Pen Example We can apply these tools to the Goldman-Tono-Pen example. Note that while we test the null hypothesis H 0 : β = 0, it is of little interest as it is not a very meaningful hypothesis

33 Goldman Tono-Pen Example Goldman T-Pen ID x = G y = T d d 2 G 2 T 2 GxT Sum ,497 15,352 15,

34 y ˆ = a + bx yˆ = x Create a new table 19-34

35 Goldman-Tono-Pen Example Tono-Pen y = x Goldman 19-35

36 Regression ANOVA Goldman Tono-Pen Example 1. The Hypothesis: H 0 : β = 0 vs H 1 : β 0 2. The Assumptions: Random samples, x measured without error, y normal distributed for each level of x 3. The α-level: α = The test statistic: ANOVA 5. The rejection region: Reject H 0 : β = 0, if F = MS(Re gression) F0.95(1,38) MS(Re sidual) >

37 6. The result: n = 40, SS(Regression) = SS(Residual) = SS(Total) = F 0.95(1,38) 4.08 ANOVA Source DF SS MS F Regression Residual Total The conclusion: Reject H 0 : β = 0 since >

38 Example: AJPH, Aug. 1999; 89:

39 19-39

40 State Y X Y 2 X 2 XY 1 AL AR AZ CA CO CT FL GA IA IN KS KY LA MA MD MI MN MO MS NC ND NH NJ NY OH OK OR PA RI SC TN TX UT , VA WA WI WV WY Total Mean SD r 0.7 slope 0.24 intercept 3.92 Value at

41 Percentage Peporting Fair or Poor Health Percentage Responding 'Most People Can't Be Trusted At x = 45, y = r = 0.70 y = x 19-41

42 Regression ANOVA Social Capital and Self-Rated Health Example 1. The Hypothesis: H 0 : β = 0 vs H 1 : β 0 2. The Assumptions: Random samples, x measured without error, y normal distributed for each level of x 3. The α-level: α = The test statistic: ANOVA 5. The rejection region: Reject H 0 : β = 0, if F MS( Regression) = > F0.95(1,36) MS( Residual)

43 6. The result: n = 38, SS(Regression) = SS(Residual) = SS(Total) = F (1,36) ANOVA Source DF SS MS F Regression Residual Total The conclusion: Reject H 0 : β = 0 since >

44 Example: AJPH, July 1999; 89:

45 19-45

46 Men Lifetime SES score Women Lifetime SES score Percentage with Poor Health Percentage with Poor Health

47 Socioeconomic Environment and Adult Health Example Men Women X X 2 Y Y 2 XY X X 2 Y Y 2 XY n X SD X: Lifetime socioeconomic status (SES) score Y : Percentage with Poor Health 19-47

48 Socioeconomic Environment and Adult Health Example Men SS(x) = SS(y) = SS(xy) = b = 1.38 a = r = SS(Reg) = SS(Res) = SS(Total) = Women SS(x) = SS(y) = SS(xy) = b = 1.56 a = r = SS(Reg) = SS(Res) = SS(Total) = yˆ = x yˆ = x M W 19-48

49 Socioeconomic Environment and Adult Health Example Men Women 1. The hypothesis: H 0 : β = 0 vs H 1 : β 0 H 0 : β = 0 vs H 1 : β 0 2. The assumptions: Random samples The same as that of men x measured without error y normal distributed for each level of x 3. The α-level : α = 0.05 α = The test statistic: ANOVA ANOVA 5. The rejection region: Reject H 0 : β = 0, if The same as that of men F MS ( Regression ) MS ( Residual ) = > F0.95(1, n 2)

50 Regression ANOVA Socioeconomic Environment and Adult Health Example 6. The result: ANOVA Men Source Regression Residual Total df SS MS F Create a new table Women df SS MS F The conclusion: Reject H 0 : β = 0 since F > F 0.95(1,11) =