Garvan Ins)tute Biosta)s)cal Workshop 16/6/2015. Tuan V. Nguyen. Garvan Ins)tute of Medical Research Sydney, Australia

Transcription

1 Garvan Ins)tute Biosta)s)cal Workshop 16/6/2015 Tuan V. Nguyen Tuan V. Nguyen Garvan Ins)tute of Medical Research Sydney, Australia

2 Introduction to linear regression analysis Purposes Ideas of regression Es)ma)on of parameters R codes

3 Purposes of regression analysis

4 Three types of analysis Analysis of difference Associa)on analysis Correla)on analysis and predic)on

5 Analysis of differences t- test, ANOVA z- test, Chi- square

6 Analysis of association Odds ra)o Risk ra)o Prevalence ra)o etc

7 Analysis of correlation Correla)on analysis Linear regression analysis Logis)c regression Cox's regression etc

8 A note of history Developed by Sir Francis Galton ( ) in his ar)cle Regression towards mediocrity in hereditary structure

9 Linear regression analysis Assess / quan)fy a LINEAR rela)onship between variables Es)mate the magnitude of effect of risk factors on an outcome variable Build model of predic)on

10 Purposes of linear regression analysis Find an equa)on to describe the rela)onship betwee X and Y X is a predictor, risk factor, independent variable Y is the outcome variable, dependent variable Adjustment for confounding factors Predic)on Linear regression model

11 Ideas of linear regression

12 Given two points on a plane (x 1, y 1 ) and (x 2, y 2 ) y-axis (x 2, y 2 ) 0 (x 1, y 1 ) x-axis How to find an equa3on to link the 2 poits?

13 y-axis (x 2, y 2 ) 0 (x 1, y 1 ) Find a slope slope Δy y y = = Δx x x x-axis Find the intercept (value of y when x=0)

14 But we have MANY points... y-axis 0 x-axis

15 and many many points... y x

16 The linear regression model Simple linear regression model Y - response variable, dependent variable Y is a con)nuous variable X - predictor variable, independent variable X can be a con)nuous variable or a categorical variable

17 The linear regression model The statement: α : intercept β : slope / gradient Y = α + βx + ε ε : random error the varia)o in Y for each X value

18 Assumptions The rela)onship between X and Y is linear X does not have random error Values of Y are independent (eg, Y 1 is not related to Y 2 ) ; Random error ε: Normal distribu)on with mean 0, constant variance, ε ~ N(0, σ 2 )

19 Estimation of parameters

20 Aim of estimation The model (popula)on): Y = α + βx + ε We don't know α and β But we can use observed data to to es)mate the two parameters Es)mates of α and β are a and b

21 Can be es)mated by eyeballing But it can be biased and inconsistent We want a method to give unbiased and consistent es)mates y x

22 Criteria for estimation y ˆ = a + d i i bx i = y i yˆ i Y y i X Find a formula (es)mator) to es)mate a and b so that sum of d 2 is minimum à Least square method

23 Carl Friedrich Gauss ( ) Born Brunswick, Germany Prodigy, "the greatest mathema)cian since an)quity" Inventor of the least square method

24 Estimates of parameters For a series of pairs of X i, Y i b = s XY s X 2 = n (X i X) (Y i Y ) i=1 n (X i X) (X i X) i=1 a = Y bx Then, the equa)on is: ˆ Y i = a + bx i

25 Using R The linear regression model: Y = α + β*x + ε R codes (using func)on lm): lm(y ~ x)

26 Francis Galton's data Galton F (1869). Hereditary Genius: An Inquiry into its Laws and Consequences. London: Macmillan Data from 928 adult children born to 205 fathers and mothers Data mid- parent's height child's height

27 Galton's data galton = read.csv("~/google Drive/Garvan Lectures 2014/Datasets and Teaching Materials/Galton data.csv", header=t) attach(galton) head(galton) id parent child

28 child parent

29 Linear regression analysis Research statement: Child's height is related to parent's height Sta)s)cal statement: R codes: Child = α + β.parent + ε m = lm(child ~ parent, data=galton)

30 > m = lm(child ~ parent) > summary(m) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** parent <2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 926 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 926 DF, p-value: < 2.2e-16

31 Interpretation of results Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** parent <2e-16 *** Remember that our model is: Our equa)on is now: Child = a + b*parent Child = *Parent Interpreta)on: Each in increase in parental height is associated with 0.64 in increase in child's height.

32 child Child = *Parent parent

33 Meaning of the regression line child Expected value Child = *Parent When parental height = 64 in Child = *64 = parent When parental height = 70 Child = *70 = 69.2

34 Analysis of variance

35 Questions concerning linear regression Is the model good enough? What criteria to judge the "goodness of fit"? Good = difference between observed and predicted values

36 Residual (e) Residual = the part not explained by the model Predicted child's height = *Parent e = Observed height Predicted height calculated for each child

37 Residual and predicted values using R m = lm(child ~ parent, data=galton) res = resid(m) pred = predict(m) > cbind(parent, child, pred, res) parent child pred res

38 Analysis of variance Child = a + b*parent + e Observed varia)on = model + random Varia)on = sum of squares SS total = total sum of squares SS reg = sum of squares due to the regression model SS error = sum of squares due to random component

39 Geometrical representation SSE Child SSR SST mean Parent SS total = SS reg + SS error

40 Sources of variation > m = lm(child ~ parent, data=galton) > anova(m) Analysis of Variance Table Response: child Df Sum Sq Mean Sq F value Pr(>F) parent < 2.2e-16 *** Residuals Signif. codes: 0 *** ** 0.01 * Total SS = = 5877 Due to "parent": 1236 Residuals (unexplained part): 4640

41 The coefficient of determination (R 2 ) > m = lm(child ~ parent, data=galton) > anova(m) Analysis of Variance Table Response: child Df Sum Sq Mean Sq F value Pr(>F) parent < 2.2e-16 *** Residuals Signif. codes: 0 *** ** 0.01 * Total SS = = 5877 R 2 = 1237 / 5877 = 0.21

42 Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** parent <2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 926 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 926 DF, p-value: < 2.2e-16

43 Meaning of R 2 Residual standard error: on 926 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 926 DF, p-value: < 2.2e-16 Coefficient of determina)on R 2 = 0.21 Interpreta)on: Approximately 21% of childen's height variance could be accounted for by parental height

44 Adjusted R 2 Defini)on: R 2 adj = 1 - (MS error / MS total ) MS error : mean square due to error MS total : mean square (total)

45 Adjusted R 2 > m = lm(child ~ parent, data=galton) > anova(m) Analysis of Variance Table Response: child Df Sum Sq Mean Sq F value Pr(>F) parent < 2.2e-16 *** Residuals Signif. codes: 0 *** ** 0.01 * MS total = ( ) / 927 = 6.34 MS error = 5 R 2 adj = 1 (5 / 6.34) = 0.21

46 Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** parent <2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 926 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 926 DF, p-value: < 2.2e-16

47 Summary A simple linear regression model is used to describe a linear rela)onship between two quan)ta)ve variables The model allows es)ma)on of effect size R func)on for linear regression: lm(y ~ x)