Regression Modelling. Dr. Michael Schulzer. Centre for Clinical Epidemiology and Evaluation

Transcription

1 Regression Modelling Dr. Michael Schulzer Centre for Clinical Epidemiology and Evaluation

2 REGRESSION Origins: a historical note. Sir Francis Galton: Natural inheritance (1889) Macmillan, London.

3 Regression to the mean: Car accidents in Scotland

4 Regression to the mean: Car accidents in Scotland Transit time: Experimental design

5 Regression to the mean: Car accidents in Scotland Transit time: Experimental design Meta-analysis (L Abbé plot)

6 Regression to the mean: Car accidents in Scotland Transit time: Experimental design Meta-analysis (L Abbé plot)

7 Regression to the mean: Car accidents in Scotland Transit time: Experimental design Meta-analysis (L Abbé plot) Education

8 Example: simple linear regression Malignant melanomas in white males vs latitude: scatterplot and regression

9 First step in regression analysis: A scatterplot.

10 Regression by Least squares:

11 Regression by Least squares: Some mathematics: Line equation: Y = α + βx + e Data: (Xi,Yi), I = 1, 2,, n.

12 Regression by Least squares: Some mathematics: Line equation: Y = α + βx + e Data: (Xi,Yi), i = 1, 2,, n. Find estimates a and b of α and β from the data, so that the sum of squares, Σ (Yi [α + βxi])2 is as small as possible.

13 Solution: Y = a + bx where b = ΣXY [(ΣX)(ΣY)]/n = slope estimate a = (ΣY)/n b[(σx)/n] = intercept estimate Estimated regression line passes through the center of the data, [(ΣX)/n, (ΣY)/n].

14 At each observation (Xi,Yi), we compare the observed Yi to the predicted Yi-value on the regression line for the same Xi and calculate the difference between them. This difference is called the residual. Recall that the sum of squares of these residuals has been minimized by the choice of a and b. This residual sum of squares, divided by [n-2] degrees of freedom, provides an estimate of the variance of the data points about the regression line. This estimate is called the mean square error (MSE). The square root of the MSE is known as the standard error of the regression (the smaller the better).

15

16 To test the significance (p-value) of the slope and intercept of the regression line, and to construct correct confidence intervals for them, we make 3 important statistical assumptions about the data: 1. The observations Yi are independent of each other, 2. They follow a normal distribution about the regression line, 3. They have a constant variance about the line. Under these assumptions, the MSE provides estimates of the variance and standard error of a and b. We can then test the significance of the slope (and intercept), and estimate corresponding confidence intervals. We can also estimate prediction intervals of future Y-values at given X-values.

17 But extrapolation is risky! Example.

18 Examples

19 Examples

20 The anova table: a summary of the results. The F-test and t-test.

21

22

23 Checking the assumptions: Residual plots.

24 Checking the assumptions: Residual plots.

25 Checking the assumptions: Residual plots.

26 Checking the assumptions: Residual plots.

27 Checking the assumptions: Residual plots. Outliers, influential points.

28 Checking the assumptions: Residual plots. Outliers, influential points. Testing lack of fit..

29 Checking the assumptions: Residual plots. Outliers, influential points. Testing lack of fit..

30 Checking the assumptions: Residual plots. Outliers, influential points. Testing lack of fit..

31 Checking the assumptions: Residual plots. Outliers, influential points. Testing lack of fit. Wrong observational unit: the ecological fallacy.

32

33 Checking the assumptions: Residual plots. Outliers, influential points. Testing lack of fit. Wrong observational unit: the ecological fallacy.

34 Transformations To stabilize the variance of Y

35 Transformations To stabilize the variance of Y To linearize the relationship of Y with X

36 Transformations To stabilize the variance of Y To linearize the relationship of Y with X To attain a normal distribution for Y.

37 Transformations To stabilize the variance of Y To linearize the relationship of Y with X To attain a normal distribution for Y. The log transformation The square root transformation The reciprocal transformation.

38 Transformations To stabilize the variance of Y To linearize the relationship of Y with X To attain a normal distribution for Y. The log transformation The square root transformation The reciprocal transformation. Examples.

39

40

41

42 Correlation Closely related to regression. Correlation coefficient described by Karl Pearson and RA Fisher. The correlation coefficient r is related to the slope b by: r = b[sd(x)/sd(y)] So r has same sign as b, and the same significance level as b. r is always between 1 and +1. r measures the linear alignment of the data. Examples.

43

44 r2 measures the proportion of variation in Y that has been successfully explained through the linear regression on X. It is known as the coefficient of determination. Example: Melanoma vs latitude: b = sd (X) = 4.61 sd (Y) = r = [(4.61)/(33.43)] = r2 = 0.67 = 67%

45 Multiple regression Example: Mortality from malignant melanomas in white males (Y) vs latitude (X1) and contiguity to ocean (X2) {X2 = 1: contiguous to ocean; X2 = 0: does not border ocean} Y = X X2 Interpretation of the equation. Ancova. (Recall: simple regression of Y on X1 alone: Y = X1 ) An anova table is calculated for each model. Models with and without X2 can be compared by comparing the residual sums of squares. If the decrease in the RSS due to adding contiguity to ocean is significant by an F-test, contiguity should be included in the model. If the coefficient of latitude is greatly changed by the addition of contiguity, then contiguity may be labeled a confounding variable.

46 Multiple regression as a series of simple regressions.

47 Multiple regression as a series of simple regressions. Multiple regression: overall, stepwise forward, stepwise backward, all subsets. Multicollinearity