Correlation and Regression

Transcription

1 A. The Basics of Correlation Analysis 1. SCATTER DIAGRAM A key tool in correlation analysis is the scatter diagram, which is a tool for analyzing potential relationships between two variables. One variable is plotted on the horizontal axis and the other is plotted on the vertical axis. The pattern of their intersecting points can graphically show relationship patterns. The tighter the points are to a line drawn through the center of the data (regardless of the slope), the more pronounced the correlation between the two variables. Most often a scatter diagram is used to illustrate potential cause-and-effect relationships. While the diagram may show a relationship, it does not by itself prove that one variable causes the other. Examples Weak Correlation Strong Correlation No Correlation 2. LINEAR CORRELATION COEFFICIENT A correlation exists between two variables when one of them is related to the other in some way. The linear correlation coefficient (r) is the calculated strength of this relationship, and its rather imposing formula follows: *r is a statistic, so it is calculated from a sample that yields n pairs of x and y. The corresponding population parameter is ρ (rho) 1

2 3. PROPERTIES OF THE LINEAR COEFFICIENT a) Correlation coefficient r can range from to The value of represents a perfect negative correlation while a value of represents a perfect positive correlation. A value of 0.00 represents a lack of correlation. b) The value of r is independent of the scale used. That is, an r based on inches will be the same if the inches are converted into meters. c) The value of r does not depend on the choice of x or y. It is a measure of the linear relationship between the two, and will be the same regardless of how y and x are assigned. Interchange all x and y values and r remains the same! d) r measures the strength of a linear relationship; it will not work if the relationship is non-linear. 4. WHEN THERE IS A SIGNIFICANT LINEAR CORRELATION BETWEEN TWO VARIABLES One of five situations can be true: a) There is a direct cause and effect relationship b) There is a reverse cause and effect relationship c) The relationship may be caused by a third variable d) The relationship may be caused by complex interactions of several variables e) The relationship may be coincidental 5. COMMON ERRORS There are some common errors that are made when looking at correlation. a) Avoid concluding causation. Just because there is a linear relationship doesn't mean that one thing caused the other. It could be any of the five situations above. b) Avoid data based on rates or averages. Variation is suppressed when using a rate or an average. c) Watch out for linearity. All that we're testing here is the strength of a linear relationship. There are other kinds of relationships, including quadratic, cubic, quartic, exponential, and logarithmic. A simple scatter plot is a good way to look for patterns. 2

3 B. Hypothesis Testing Revisited : Evaluating Linear Correlation 1. METHOD 1 d) Establish the Null and the Alternative H 0 =0 (No significant correlation) H 1 0 (Significant correlation) e) Select a significance level α f) Calculate r using the following: Use this table as an aid to using the above formula: INPUT x y x*y x^2 y^2 Totals: 0 0 x y xy x2 y2 g) Determine the critical value t c This value is obtained from the t distribution table using n-2 degrees of freedom and two tails. h) Calculate the test statistic t i) If the absolute value of t exceeds the critical values, reject the null hypothesis that there is no significant correlation. Otherwise, fail to reject the null hypothesis. 3

4 2. METHOD 2 a) Establish the Null and the Alternative H 0 =0 (No significant correlation) H 1 0 (Significant correlation) b) Select a significance level α c) Calculate r using the following: d) Use r from the previous step as the test statistic. e) Determine the critical value r Obtain from the table Critical Values of the Pearson Correlation Coefficient r f) If the absolute value of the calculated r exceeds the critical value extracted from the table in the previous step, reject the null hypothesis that there is no significant correlation. Otherwise, fail to reject the null hypothesis. Example 1 The accompanying table list weights (in pounds) of plastic discarded by a sample of households, along with the sizes of the household. Is there significant linear correlation? Use α=0.05 Method 1 a) Establish the Null Hypothesis H 0 =0 (No significant correlation) b) Establish the Alternative Hypothesis H 1 0 (Significant correlation) c) Select a significance level α Significance level α=

5 d) Calculate r using the following table as an aid: Weight Household (Pounds) Size x y x*y x 2 y Totals: x xy x 2 y 2 r e) Determine the critical value t c This value is obtained from the t distribution table using n-2 degrees of freedom and two tails. ± f) Calculate the test statistic t =3.823 g) Since the absolute value of t exceeds the critical values, reject the null hypothesis and conclude that there is significant correlation. 5

6 STA Method 2 Follow steps a) through c) in Method 1 to arrive at r d) Use r from the previous step as the test statistic. r e) Determine the critical value r Obtain from the table Critical Values of the Pearson Correlation Coefficient r r critical = f) Since the absolute value of the calculated r (0.842) exceeds the critical value (0.707) extracted from the table in the previous step, reject the null hypothesis and conclude that there is significant correlation. C. Regression Once it has been established that there is significant correlation between two variables using one of the two previously explained methods, the next step is to develop a linear equation that mathematically describes the relationship. This equation can then be used for predictions of the value of the dependent variable y for various values of the independent variable x. A regression line is the line described by the equation and the regression equation is the formula for the line. The regression equation is given by: y = ax + b where x is the independent variable, y is the dependent variable, a is the slope of the line and b is the intercept. The regression line is also called the line of best fit. 1. FINDING THE EQUATION OF THE REGRESSION LINE USING THE FORMULAS 6

7 STA Example 2 The following is a table for the calories and sodium content for 10 beef hot dogs: Calories x Sodium y xy X Find the equation of the regression line using the form y=ax+b 80.8 Therefore, the equation of the regression line is: The purpose of the above equation is to be able to make predictions of y based on various values of x; in other words, to make predictions of sodium content depending how many calories are present in a hot dog. To that end, predict the sodium level y in the following three hot dogs with calories x: a) x=170 calories b) x=100 calories =2.47* mg of sodium =2.47* mg of sodium c) x=210 calories This prediction is not meaningful since 210 calories is outside of the domain of the original data (90 180) that was used to construct the regression equation. Be aware of the domain of x values that were used to construct the regression equation; these values represent the interval over which the regression equation is valid. 7

8 Example 3 Re-do Example 2 using the TI-83/84 a) Enter the given values of x and y into Lists in the calculator (L1 and L2 were used, but it could be any lists as long as the proper lists are chosen for the calculations). See list entry below: b) Press STAT, CALC, and select LinReg (ax+b). c) Enter the lists that will be used for the regression calculation 2 ND 1,2 ND 2 d) Select ENTER As with the manual method, the equation of the regression line is: Additional information from the last TI-83/84 screen shot, above right: r is the correlation coefficient, and was covered extensively in the beginning of this handout r 2 is known as the Coefficient of Determination, and is the ratio of explained variation to the total variation. In this case, it means that 82% of the variation of y can be explained by the relationship between x and y. The remaining 18% of the variation is unexplained and is due to other factors or to sampling error. 8