Chapter 7. Scatterplots, Association, and Correlation

Transcription

1 Chapter 7 Scatterplots, Association, and Correlation Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

2 Objective In this chapter, we study relationships! Instead, we investigate the association and correlation between two (quantitative) variables. Examples: Do you think more time spent on STAT 141 will get you a better grade? At least, I think so. Do you support the claim that people with advanced education get paid at higher salary levels? Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

3 Definitions Explanatory Variable Explanatory variable, also called independent variable, usually denoted by x, explains or causes changes in the response variable. Response Variable Response variable, also called dependent variable, normally symbolled by y, measures the outcome of a study. Example 7.1 Identity the x and y variables. 1 What is the effect of rainfall on crops yield? 2 How does the midterm score influence the final grade? Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

4 Scatterplots A scatterplot plots pairs of observations of two variables (x and y). Each dot contains a pair of observations, the x-coordinate is the value of the x variable and the y-coordinate is the value of y variable. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

5 Bin Zou STAT 141 University of Alberta Winter / 29

6 Direction We can tell the direction of the association from the scatterplots. A pattern running from the upper left to the lower right (going downwards) is called negative. Another definition: If y decreases when x increases, then two variables are negatively associated. A pattern running the other way (going upwards) is called positive. Another definition: If y increases when x increases, then two variables are positively associated. The patter in the previous slide shows positive association. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

7 Form and Strength If all the points in a scatterplot roughly follow a straight line, then we call the form of the scatterplot linear. Other forms: curved, clusters, no pattern, etc. The strength of the relationship is determined by how close the points in the scatterplot lie to a simple form, such as a straight line. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

8 Correlation The correlation measures the directions and strength of the linear relationship between two quantitative variables. A commonly used measure of correlation is called correlation coefficient, defined as r := 1 n ( )( ) xi x yi ȳ n 1. i=1 The correlation coefficient, r, is the sum of the products of the standardized values for each paired observations all divided by n 1. s x s y Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

9 Facts about correlation coefficient r Two variables must be quantitative. Does NOT work for non-linear relationships. The correlation coefficient of x and y is the same of y and x. r has no unit. r takes values only between 1 and 1. The absolute value of r measures the strength of the linear relationship. The closer to 1, the stronger the linear relationship. When r gets close to 0, that means almost no linear relationship. The sign of r determines the direction of the linear relationship. If r is positive, the relationship is positive. And vice versa. r is strongly affected by outliers. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

10 Bin Zou STAT 141 University of Alberta Winter / 29

11 Correlation Causation Correlation: the change of one variable is associated with the change of the other. Causation: the change of one variable causes the change of the other. On average, we should see more time spent on STAT 141 is associated with higher grade. Hence, time and grade are (positively) correlated. But, increasing the time spent on STAT 141 does not guarantee better result. So, there is NOT causation relationship between time and grade. The final grade is also affected by study method, previous knowledge on statistics, etc. Bin Zou STAT 141 University of Alberta Winter / 29

12 Chapter 8 Linear Regression Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

13 Objective If a scatterplot indicates a linear relationship between two variables, how do we find the line that best describes this relationship? Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

14 Linear Model A liner model (a straight line) is represented by y = b 0 + b 1 x. b 0, y-intercept, is the y value when x=0. b 1, called the slope, is the amount that y changes when x increases by 1. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

15 Residuals Since the linear relationship is not perfect most time, there are gaps between the observed value (black dots) and the predicted value, or fitted value (red dots). The difference between the observed value and the predicted value is called the residual, y ŷ. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

16 Criterion for Best Fit The criterion to select the best fit line is least squares of residuals. Two variables x and y, each with n observations: (x 1,y 1 ),(x 2,y 2 ),,(x n,y n ). Assume the best fit line is: ŷ = b 0 + b 1 x. For each x i, we can calculate the predicted ŷ i by ŷ i = b 0 + b 1 x i, and then residual y i ŷ i. Note: we use ŷ to distinguish from y. The best fit line should produce the least sum of squares of all n residuals: (y i ŷ i ) 2. i=1 The process of finding the best fit line is linear regression. Estimate two coefficients: b 0 and b 1. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

17 Regression Line Given two quantitative variables, x and y: The mean and the standard deviation of x are x and s x. The mean and the standard deviation of y are ȳ and s y. The correlation coefficient between x and y is r. The the least linear regression line is given by ŷ = b 0 + b 1 x, where b 1 = r s y s x, b 0 = ȳ b 1 x. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

18 Facts about the Regression Line The distinction between explanatory and response variables is essential in regression. The regression of y on x is NOT the same as the the regression of x on y. Both coefficients b 0 and b 1 have units. b 0 has the same unit as y, and b 1 has the unit of y unit per x unit. Example: x time in hour, y distance in km. The the unit of b 1 is km per hr. The least-squares regression line goes through the point ( x, ȳ). Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

19 Examples Here are the summary statistics for the Burger King data: x variable Protein : x = 17.2g, s x = 14.0g, y variable Fat : ȳ = 23.5g, s y = 16.4g, r = Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

20 R 2 The coefficient of determination, R 2, measures the proportion of variation in y which is explained by the linear regression on x. Important! R 2 = r 2. r is the correlation coefficient. 1 r 1, thus 0 R 2 1. The greater the R 2, the more accurate the regression model. In the previous example, r = 0.83, then R 2 = = 0.69 = 69%. Hence, 69% of the variation in total fat (y variable) is explained by the linear regression on protein (x variable). Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

21 Chapter 9 Regression Wisdom Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

22 Residual Plot A residual plot is a scatterplot of the residuals against the explanatory variable (x variable). Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

23 Outliers We have seen outliers before in histograms and boxplots. In a histogram, outliers are observations that are far away from the main body of the distribution. In a boxplot, outliers are observations that do not fall within the fences. In a scatterplot, outliers are points that stand away from the majority. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

24 Outliers can significantly affect the regression. If there are outlier(s) in the scatterplot, think ahead before we run the regression. Without the outlier (red dot), the regression line is drawn in blue. With the outlier included, the regression line is the red line. Bin Zou STAT 141 University of Alberta Winter / 29

25 Leverage A point whose x value is far from the mean of x variable is said to have high leverage. The x value of the red cross point is a lot greater than the x value of all blue points (the mean is somehow around 7.5). Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

26 Influential Points A point is called influential if omitting it from the analysis gives a very different model. As discussed in the section of Outliers, removing the red point from the data changes the regression line to the blue line. Hence, the red point is an influential point. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

27 Comparison The red point has high leverage, because its x value is large. But this point is NOT an influential point, since it is approximately on the extension line of the majority points. Bin Zou (bzou@ualberta.ca) STAT 141 University of Alberta Winter / 29

28 Chapter 10 Re-Expression Data Get It Straight! Reading Material! Bin Zou STAT 141 University of Alberta Winter / 29

29 Logarithmic Transform Assumption All data observations must be strictly positive! Objective of ln Transform To obtain stronger linear relationship. Original data: (x, y) Transformed data: (ln(x), y), or (x, ln(y)), or (ln(x), ln(y)). Bin Zou STAT 141 University of Alberta Winter / 29