STATISTICS Relationships between variables: Correlation

Size: px

Start display at page:

Download "STATISTICS Relationships between variables: Correlation"

Hester Malone
5 years ago
Views:

1 STATISTICS 16 Relationships between variables: Correlation The gentleman pictured above is Sir Francis Galton. Galton invented the statistical concept of correlation and the use of the regression line. Blame him, not me.

Correlation 2 1. Quantitative variables can be associated A famous study found an association between the amount of soluble fiber in one s diet and the risk of cardiovascular disease.

2 Correlation 2 1. Quantitative variables can be associated A famous study found an association between the amount of soluble fiber in one s diet and the risk of cardiovascular disease. Specifically, higher dietary soluble fiber is associated with reduced disease risk (and slower disease progression in high-risk individuals). The results of such studies can have an impact on society; e.g., the above results led the American Heart Association to recommend a diet high in soluble fiber. This example illustrates the outcome of a study of the relationship between two variables (e.g., soluble fiber and risk of heart disease). In such studies both variables are measured on the same individual. If the value of one variable tends to be related to the value of the second variable when we look at a large sample of individuals we say that the two variables are associated. Country Wine consumption 1 (x) Heart disease 2 (y) Australia Austria Belgium Canada Denmark Finland France Iceland Ireland.7 Italy Netherlands New Zeland Norway Spain Sweden Switzerland U. K U. S One of the most common reasons to collect data in the first place is to look for associations. Below are some examples of questions of association that have medical implications: W. Germany Wine consumption: liters of alcohol via wine per person per year; 2 Heart disease: Deaths per 1, per year Does alcohol consumption increase, or decrease one s risk of death due to heart disease? (See above table for pairs of variables relevant to this question.) Has the incidence of breast cancer been increasing over the last years? Does a high fiber diet result in a reduced risk of heart disease? Does the concentration of a new drug affect the severity of its side-effects? A common theme in the above examples is that one variable might be used to explain or predict another variable. For example does our data allow us to predict a countries death rate due to heart disease if we have information about wine consumption? In such cases we distinguish between the RESPONSE VARIABLE (The outcome of a study, or an event you wish to predict or explain.) and the EXPLANATORY VARIABLE (the variable you hypothesize to cause a change in the response variable). Practice problem 1 (available on-line) illustrates how to think about the x and y variables.

3 Correlation 3 Unfortunately the terminology in this setting can be quite variable, so a table is presented below to help clarify the situation. Variable measuring the outcome Response variable Response variable Dependent variable y-variable (because it is plotted on the y-axis) Variable explaining the change in outcome Explanatory variable Predictor variable Independent variable x-variable (because it is plotted on the x-axis) 2. SCATTERPLOTS allow visual interpretation of association The first step in the process of studying an association between two variables is to make a scatterplot. A SCATTERPLOT is a graphical display of the relationship between two quantitative variables measured on the same individual, where each individual is represented by a single point in the plot. On the right is an example scatterplot of the relationship between wine consumption and risk of heart disease. Since we are interested in predicting one s risk of heart disease by the amount of wine consumed, we plot the risk of heart disease (the response variable) on the y-axis. Now what? Well, it s quite simple; just look at the overall pattern in the plot. This is best done by looking at the following different aspects of the plot Figure 1: Scatter plot of association between risk of heart disease (in deaths per 1, per year) and wine consumption (in liters of alcohol per year) for 19 countries. Trend: Is there a positive or negative association? Outliers: Are there any striking deviations from the overall pattern? Form: Is the trend linear, curved, clustered, or something else? Strength: Is the trend strong or weak? Variance: Does variance in y change with x?

4 Correlation 4 Southern European countries Lastly, it is sometimes the case that the individual samples from a population can be further classified into CATEGORICAL VARIABLES. In such cases this information also can be visualized in a scatter plot by using a different color or symbol to indicate each category (e.g., Figure 2 illustrates that southern European countries [France, Italy & Spain] have the lowest deaths due to heart disease and the highest rates of wine consumption). 3. When the association is linear, think in terms of CORRELATION + Canada Figure 2: The same dataset as in figure 1, but with the category Southern European country indicated by an open box, and Canada indicated by as plus (+). If two variables are (1) quantitative and (2) have a linear association, then the strength and direction of the association can be measured by using the CORRELATION COEFFICIENT (abbreviated as r ). uantifying The Strength of Linear Association: he Correlation Coefficient, r The DIRECTION of the association is indicated by the sign of the correlation coefficient (r), and the magnitude of r measures the STRENGTH (illustrated below in Quantifying The Strength of Linear Association: Figure 3). The Correlation Coefficient, r Figure 3: Illustration of how the correlation coefficient (r) provides a measure of the strength and direction of an association. The value of r is always between +1 and -1. r=.9 r=.9 r=.7 r=.7 r= r= r=.7 r=.7 r=.9 r=.9 r = -.9 r = -.7 r = r =.7 r =.9 Value of r Relationship among variables r = 1 The two variables have perfect correlation with no scatter Measures strength strength of linear of linear association between between x and x and y y Measuresr > strength (positive) of linear The two variables association tend to increase between or decrease together x and y Variables Variables x and x and y are y are quantitative r = The two variables do not vary together in any way Variables x and y are quantitative Always Always r < (negative) lies lies between between The -1 two and variables -1 and 1are 1inversely 1 related 1 r r 1 1 Always lies between -1 and 1 1 r 1 Unchanged if we if we replace replace x by x by ax+b ax+be.g., e.g., in to in cm, to cm, F to F to C C Unchanged Unchanged if we replace by by switching switching x by x ax+b and x and y e.g., y incorr(x,y)=corr(y,x) to cm, F to C Unchanged Influenced by switching by by outliers outliers x and y corr(x,y)=corr(y,x)

5 Correlation 5 4. The correlation coefficient (r): how it works In this section we take a mathematical look at how the correlation coefficient works. Don t worry, it s not too complicated. The essential task is the standardization of both the x and y values. " $ # x i! x s x % ' and & " $ # y i! y s y % ' &!! and!! Now each point is expressed in how many SDs above or below the mean that it lies; i.e., the familiar Z-SCORE from earlier in this course! There is nothing new here; recall that you learned about standardizing values when you learned about the normal distribution. Thus you know that a positive z- score indicates the original value lies above the mean and a negative z-score that the value lies below the mean. Standardization also means that the units have been removed from the correlation coefficient. The next task is to take the product for all pairs of!! and!!. The product will be either positive or negative. The product is then summed over all pairs of samples. This is the step that summarizes the strength and direction of the association for all data (see Box 1 for additional details).! = 1! 1!!!!!!!!!!! =!!!!! 1 Other formulas exist that are more convenient for doing the calculation by hand, but everyone uses a computer these days, so we will just stop here.

6 Correlation 6 BOX 1: MATHEMATICAL BASIS OF A POSITIVE CORRELATION COEFFICIENT (r)!!!!!!!!!!!!!!!!!!!!!!!!! 4 quadrants defined by:!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Z y = (+) Z y = Z y = (-) Z x = (-) Z x = Z x = (+) Taking the product of standardized scores (Z y and Z x ) defines 4 quadrants within the scatter plot. The dashed lines show the locations of the standardized mean of y and x (Z y = and Z x = ). If a pair of values for y and x agree (i.e., they are both above the mean, or they are both below the mean), then their product is positive (blue). If this is not the case, then their product is negative (red). In the above example the value of r is positive because there are more blue data points than red data points. Caution: You can compute the correlation coefficient for any set of data; however it will not be possible to interpret correctly if the following ASSUMPTIONS are not met. Individuals must be selected at random from the population. For example, you can t choose individuals based on the value of one or both of the variables! The data must be comprised of independent observations. Examples of non-independent samples include siblings, or cases where sampling one member of the population somehow influences the chance of sampling another member of the population. Measurement of the x- and y-values must be independent of each other. For example, scores for mid-term exams and the final grades are not measured independently of each other. The x- and y-values must be sampled from populations that follow (approximately) a normal distribution. Outliers can strongly influence the estimate of r. All co-variation between variables must be linear. Correlation coefficients can be misleading when the data relationship is not linear! The explanatory variable, x, was not experimentally controlled. Experimenters often systematically control the explanatory variable (e.g., dose, time, flow, etc.); in such cases the setting is LINEAR REGRESSION. The confidence intervals on the r will not be correct.

7 Correlation 7 Caution: O UTLIERS can impact all statistical calculations, but correlation is especially sensitive. The following figure illustrates how the presence of a single point can have a large impact on the correlation coefficient (r). A 4 r = outlier B r = wine consumption: lieters of alcohol/person/year Figure 4: The impact of an outlier on the correlation coefficient (r). Panel A shows the plot of the data in a case where a data entry error has led to an outlier (upper right corner). Panel B shows the change in the results that is obtained when the outlier is corrected. In practice, these issues require that you always check a scatter plot before you consider summarizing your data with a correlation coefficient. Furthermore, it is always good to look at the scatter plot (if possible) when interpreting someone else s correlation coefficient! 5. The value of r requires careful interpretation. Let s return to the relationship between wine consumption and death rate due to heart disease. First let s switch the x- and y-variables; the result is that the correlation coefficient is unaffected (Figure 5 below). r = r = Figure 5: The x- and y-variables have been switched in the above two panels. It is clear that the association is exactly the same, and only the orientation of the data has changed. Note that the value of the correlation coefficient is unaffected by switching the x- and y- variables.

8 Correlation 8 Clearly the designation of the EXPLANATORY and RESPONSE VARIABLES impacts the way we think about the association, but it does not impact the correlation. So, how should we think about the data in the example; the correlation coefficient is -.843, indicating a negative relationship between the two variables. There are four possible explanations: 1. Alcohol consumption decreases the risk of death due to heart disease. 2. A death rate due to heart disease directly affects average alcohol consumption. 3. Both the level of alcohol consumption and the death rate due to heart disease are under the control of some other variable. 4. The two variables are unrelated, and the observation of a correlation was due to random chance. A: causation B: common response C: confounding! "! "! " # # Figure 6: The solid arrows show the true cause-and-effect relationship. The dashed arrows show the observed associations. (A) Simple causal relationship between the x and y variables. (B) A common response of the x and y variables to a lurking variable, called w, results in an observed association. (C) The true cause-and-effect relationships can result in confounding associations. (Adapted from Moore and McCabe.) You cannot decide between the first three possibilities above without having more information (perhaps obtained by further data collection or experimentation). The last possibility might be rejected by using a statistical framework that you will learn later in the course. The main point is that correlation does not prove causality. An excellent example of this is the well-known positive correlation between the rate of drowning and the rate of ice cream consumption. Why do you think such a correlation might exist? Practice problems 2 and 3 will be worked in class to illustrate the proper use of scatter plots.

Chapter 6 Scatterplots, Association and Correlation

Chapter 6 Scatterplots, Association and Correlation Looking for Correlation Example Does the number of hours you watch TV per week impact your average grade in a class? Hours 12 10 5 3 15 16 8 Grade 70