Important note: Transcripts are not substitutes for textbook assignments. 1

Transcription

1 In this lesson we will cover correlation and regression, two really common statistical analyses for quantitative (or continuous) data. Specially we will review how to organize the data, the importance of scatterplots, and introduce correlation and regression methods. Important note: Transcripts are not substitutes for textbook assignments. 1

2 For correlation and simple regression we have a quantitative response variable Y ( dependent variable ) and a quantitative explanatory variable X ( independent variable ). Here s an example using historically important public health data from 1955 on cigarette smoking and lung cancer mortality per 100,000 population by the late epidemiologist Sir Richard Doll. Sir Richard Doll was the first to widely report an association between lung cancer and cigarette smoking. Here n = 11 countries, the explanatory variable = per capita cigarette consumption in 1930 (CIG1930) and the response variable = lung cancer mortality per 100,000 (LUNGCA). 2

3 Here s a scatter plot of the data. The goal of correlation is to determine if there is a significant association between these two variables. You need to inspect the data to determine if the relationship can be described with a straight or some other type of line. You also want to determine the direction. Do points tend to trend upward or downward? You can also estimate the strength of the association. Do the points adhere closely to an imaginary trend line? Finally, you want to determine if there are any outliers. Are there any striking deviations from the overall pattern? Outliers are really important in correlation, especially with small sample size, because a single outlier can effect the strength of the association. As always it s important to determine if the outlier is a valid data point or a data error. 3

4 Correlational strength refers to the degree to which points adhere to a trend line. The eye is not a good judge of strength. The top plot appears to show a weaker correlation than the bottom plot. However, these are plots of the same data sets. The perception of a difference is an artifact of axes scaling. 4

5 Correlation coefficient r quantifies linear relationship with a number between 1 and 1. When all points fall on a line with an upward slope, r = 1. When all data points fall on a line with a downward slope, r = 1 When data points trend upward, r is positive; when data points trend downward, r is negative. The closer r is to 1 or 1, the stronger the correlation. 5

6 Here is the formula for calculating the Pearson s correlation coefficient. Correlation coefficient tracks the degree to which X and Y go together. Recall that z scores quantify the amount a value lies above or below its mean in standard deviations units. When z scores for X and Y track in the same direction, their products are positive and r is positive (and vice versa). 6

7 Here is the calculation of the correlation coefficient for our cigarette smoking and lung cancer mortality example. The correlation coefficient is which is a strong correlation. 7

8 This is a typical statistical output for a correlation coefficient. The Pearson Correlation for Cigarette consumption vs. lung cancer mortality per 100,000 in 1950 is.737. Note that this is a positive correlation and that it is statistically significant at p=0.01. That means as cigarette consumption increases so does lung cancer mortality. The sign of r indicates the direction of the association: positive (r > 0), negative (r < 0), or no association (r 0). The closer r is to 1 or 1, the stronger the association. The square of the correlation coefficient (r 2 ) is called the coefficient of determination. This statistic quantifies the proportion of the variance in Y [mathematically] explained by X. For the illustrative data, r = and r 2 = Therefore, 54% of the variance in Y is explained by X. 8

9 With correlation, it does not matter whether variable X or Y is specified as the explanatory variable; calculations come out the same either way. This will not be true for regression. As I mentioned earlier, outliers can have a profound effect on r. This figure has an r of 0.82 that is fully accounted for by the single outlier. 9

10 Another thing to watch out for in correlation is that it applies only to linear relationships. This figure shows a strong non- linear relationship, yet r =

11 Another essential point about correlation is that it does not necessarily mean causation. Beware of lurking variables also known as confounding. Here s a great example, A near perfect negative correlation (r =.987) was seen between cholera mortality and elevation above sea level during a 19th century epidemic. We now know that cholera is transmitted by water. The observed relationship between cholera and elevation was confounded by the lurking variable of proximity to polluted water. 11

12 Random selection from a random scatter can result in an apparent correlation so we conduct the hypothesis test to guard against identifying too many random correlations. 12

13 We conduct hypothesis testing using a t statistic to determine the probability of observing the correlation coefficient by random chance. The null hypothesis is that correlation coefficient is 0. The t statistic is a the correlation coefficient divided by the standard error of the correlation coefficient. The Degrees of Freedom is n

14 Here is the hypothesis test for the cigarette and lung cancer mortality example. The calculated t stat is 3.27 with 9 df. The associated p- value is The evidence against H 0 is highly significant. 14

15 We can also calculate confidence intervals for the correlation coefficient using the formulas shown here. The Greek symbol here is omega. 15

16 Here is an example of calculating the 95% CI for the correlation coefficient for our example. The 95% CI for r=.737 is to The 95% CI are wide because of the extremely small sample size (n=11). 16

17 Like any test there are important assumptions or conditions for inference. The first is independent observations which means there can t be a bias or error in selecting the x variable or the y variable. Another assumption for correlation is Bivariate Normality. This means both variables should be normally distributed as shown in this figure. 17

18 Regression describes the relationship in the data with a line that predicts the average change in Y per unit X. The best fitting line is found by minimizing the sum of squared residuals, as shown in this figure. The residuals are distance from the data point to the line. This method of fitting the regression line is often called the least squares method. 18

19 The regression line equation is ŷ = a + bx where ŷ predicted value of Y, a is the intercept of the line, and b is the slope of the line Equations to calculate the slope and intercept are shown here. Make sure you learn the basic regression line equation. 19

20 Slope b is the key statistic produced by the regression. It s defined as the change in y per unit x. The y intercept is where the regression line crosses the Y axis (a) 20

21 This is the output from runing a regression of cigarette consumption on lung cancer mortality. The slope of the regression line is and the y intercept is In these statistical output models, the standardized coefficient is when the y intercept is set to zero. The slope is the most important aspect of these models and we want to determine the direction of the slope (positive or negative) and the statisticial signficance of the slope. 21

22 We are interested in testing the null hypothesis that the slope is 0. That is, does x explain any variation in y. If the slope of x is 0 it does not explain any variation in y. Our model from our sample data lets us estimate the population. Let α represent the population intercept, β represent population slope, and ε i represent the residual error for point i. The residual error for point i is how far the point is from the regression line. We can estimate the standard error of the regression and calculate the confidence intervals for the population slope. 22

23 Statistical software helps us calculate the 95% CI for the slope. The interpretation here is that we are 95% certain that the true population slope is between to Since the 95% CI do not include 0 the slope is statistically significant. 23

24 We can determine the p- value for the slope using a t stat and this equation. Once again the details of the equation is not import but you should know how to interpret the results of this test. 24

25 Here is the t stat test for the slope. The slope is and the p- value is So, even though the slope is small it is still statistically significant. That is, some of the variation in lung cancer mortality is explained by cigarette consumption. 25

26 Inference about the regression line can be remembered by the word LINE. The following conditions are required: Linearity; Independent observations; Normality at each level of X; and Equal variance at each level of X. We can assess these conditions by examining the plots of the residuals from the regression equation. Remember the residuals, or residual error, is the amount of distance a data point is away from the regression line. 26

27 Here s a figure showing normality and equal variance at each level of x. 27

28 The scatterplot should be visually inspected for linearity, Normality, and equal variance. Plotting the residuals from the model can be helpful in this regard. Here are the residuals from our example. 28

29 Departures from normality and equal variance can be assessed using residual plots. Here the data are sparse (that is, small sample size) but the plot shows that there may be greater variability at higher x values. 29

30 With a little experience, you can get good at reading residual plots. Here s an example of linearity with equal variance. 30

31 Here s an example of linearity with unequal variance 31

32 Finally, here s an example of non- linearity with equal variance. These data can be fitted with a model that is slightly more complicated that a simple linear regression model. This concludes lesson

33 33