Announcements. Lecture 18: Simple Linear Regression. Poverty vs. HS graduate rate

Transcription

1 Announcements Announcements Lecture : Simple Linear Regression Statistics 1 Mine Çetinkaya-Rundel March 29, 2 Midterm 2 - same regrade request policy: On a separate sheet write up your request, describing what specifically you think you should have earned more points on. Do NOT write on your exam if you want it to be considered for a regrade. When you submit a regrade request your entire exam will be regraded, not just the question(s) mentioned in your request. Due at the beginning of class on Tuesday, April 3. Over the weekend: 3 page paper on working in teams (posted on the course website) and online quiz. Project 2 proposal: Due by midnight on April (Sunday) in a Google Doc. All relevant instructions posted on the course website. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 1 / 43 Recap Poverty vs. HS graduate rate Review question The scatterplot below shows the relationship between HS graduate rate in the 1 states in the US (including DC) and the % of residents who live below the poverty line (income below $22,3 for a family of 4). True or False: If the p-value is sufficiently large you can reject H A. (a) True (b) False 9 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 2 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 3 / 43

2 Response vs. explanatory Eyeballing the line Eyeballing the line Response variable is on the y-axis, and explanatory variable is on the x-axis. 9 Which of the following appears to be the line that best fits the linear relationship between % in poverty and % HS grad? Choose one. 9 (a) (b) (c) (d) Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 4 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 / 43 Residuals Residuals Residuals Residuals (cont.) Residuals are the leftovers from the model fit: Data = Fit + Residual 9 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 / 43 Residual Residual is the difference between the observed and predicted y. y^ y RI 4. y.44 e i = y i ŷ i 9 y^ DC % living in poverty in DC is.44% more than predicted. % living in poverty in RI is 4.% less than predicted. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 7 / 43

3 Describing the relationship Quantifying the relationship 9 The relationship between % in poverty and is linear negative somewhat strong - not a huge amount of scatter around the line describes the strength of the linear relationship between two variables. It takes values between -1 (perfect negative relationship) and +1 (perfect positive relationship). A value of indicates no relationship. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 9 / 43 Guessing the correlation Calculating the correlation Which of the following is the best guess for the correlation between % in poverty and? (a). (b) -.7 (c) -.1 (d).2 (e) Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 / 43 Using computation: cor(poverty$poverty, poverty$graduates) Using a formula: R = 1 n 1 n i=1 x i x s x y i ȳ s y Note: You won t be asked you to calculate the correlation coefficient by hand, because nobody does it by hand. But you might be given a scatterplot and asked to guess the correlation. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 11 / 43

4 Guessing the correlation Assessing the correlation Which of the following is the best guess for the correlation between % in poverty and? Which of the following is has the strongest correlation, i.e. correlation coefficient closest to +1 or -1? (a).1 (b) -. (c) -.4 (d).9 (e). % female householder, no husband present (a) (c) (b) (d) Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 13 / 43 Best line Play the game! A measure for the best line istics.net/ stat/ correlations Group name: sta1 We want a line that has small residuals One option: Minimize the sum of magnitudes (absolute values) of residuals e 1 + e e n Another option: Minimize the sum of squared residuals e e e2 n The line that minimizes the sum of squared residuals is the least squares line Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 1 / 43

5 Best line Why minimize squares? The least squares line 1 Most commonly used 2 Easier to compute by hand and using software 3 In many applications, a residual twice as large as another is more than twice as bad Notation: Intercept: ŷ = β + β 1 x predicted y slope explanatory variable intercept Parameter: β Point estimate: b Slope: Parameter: β 1 Point estimate: b 1 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 17 / 43 Given... Slope Slope (x) (y) mean x =.1 ȳ = 11.3 sd s x = 3.73 s y = 3.1 correlation R =.7 The slope of the regression can be calculated as In context... b 1 = s y s x R b 1 = = Interpretation For each % point increase in HS graduate rate, we would expect the % living in poverty to decrease on average by.2% points. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 19 / 43

6 Intercept Clicker Intercept The intercept is where the regression line intersects the y-axis. The calculation of the intercept uses the fact the a regression line always passes through ( x, ȳ). ȳ = b + b 1 x intercept 2 4 ȳ = b + b 1 x b = 11.3 (.2).1 = 4. Which of the following is the correct interpretation of the intercept? (a) For each % point increase in HS graduate rate, % living in poverty is expected to increase on average by 4.%. (b) For each % point decrease in HS graduate rate, % living in poverty is expected to increase on average by 4.%. (c) Having no HS graduates leads to 4.% of residents living below the poverty line. (d) States with no HS graduates are expected on average to have 4.% of residents living below the poverty line. (e) In states with no HS graduates % living in poverty is expected to increase on average by 4.%. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 2 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 21 / 43 Regression line Interpretation of slope and intercept Interpreting regression line parameter estimates = Intercept: When x =, y is expected to equal the intercept. Slope: For each unit increase in x, y is expected to increase/decrease on average by the slope. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 22 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 23 / 43

7 Extrapolation Extrapolation Extrapolation Examples of extrapolation Applying a model estimate to values outside of the realm of the original data is called extrapolation. Sometimes the intercept might be an extrapolation intercept 2 4 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 24 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 2 / 43 Extrapolation Extrapolation Examples of extrapolation Examples of extrapolation Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 2 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 27 / 43

8 Conditions: (1) Linearity 1 Linearity 2 Nearly normal residuals 3 Constant variability The relationship between the explanatory and the response variable should be linear. Methods for fitting a model to non-linear relationships exist, but are beyond the scope of this class. Check using a scatterplot of the data, or a residuals plot. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 2 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 29 / 43 Anatomy of a residuals plot Conditions: (2) Nearly normal residuals 1 9 RI: = 1 =.3 % in poverty = =.4 e = % in poverty DC: =.3.4 = 4. = =. % in poverty = 4..2 = 11.3 e = % in poverty = =.44 frequency 2 4 The residuals should be nearly normal. This condition may not be satisfied when there are unusual observations that don t follow the trend of the rest of the data. Check using a histogram or normal probability plot of residuals residuals Sample Quantiles Normal Q Q Plot Theoretical Quantiles Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 3 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 31 / 43

9 Conditions: (3) Constant variability Checking conditions The variability of points around the least squares line should be roughly constant. This implies that the variability of residuals around the line should be roughly constant as well. Also called homoscedasticity. Check using a histogram or normal probability plot of residuals. What condition is this linear model obviously violating? (a) Constant variability (b) Linear relationship (c) Non-normal residuals (d) No extreme outliers Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 32 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 33 / 43 R 2 Checking conditions R 2 y g$residuals What condition is this linear model obviously violating? (a) Constant variability (b) Linear relationship (c) Non-normal residuals (d) No extreme outliers x y g$residuals x The strength of the fit of a linear model is most commonly evaluated using R 2. R 2 is calculated as the square of the correlation coefficient. It tells us what percent of variability in the response variable is explained by the model. The remainder of the variability is explained by variables not included in the model. For the model we ve been working with, R 2 =.2 2 =.3. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 34 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 3 / 43

10 Interpretation of R 2 R 2 Types of outliers Which of the below is the correct interpretation of R =.2, R 2 =.3? (a) 3% of the variability in the % of HG graduates among the 1 states is explained by the model. (b) 3% of the variability in the % of residents living in poverty among the 1 states is explained by the model. (c) 3% of the time uates predict % living in poverty correctly. (d) 2% of the variability in the % of residents living in poverty among the 1 states is explained by the model. How do(es) the outlier(s) influence the least squares line? To answer this question think of where the regression line would be with and without the outlier(s). 2 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 3 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 37 / 43 Types of outliers Some terminology Outliers are points that fall away from the cloud of points. How do(es) the outlier(s) influence the least squares line? Outliers that fall horizontally away from the center of the cloud are called leverage points. High leverage points that actually influence the slope of the regression line are called influential points Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 3 / 43 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 39 / 43

11 Influential points Data are available on the log of the surface temperature and the log of the light intensity of 47 stars in the star cluster CYG OB log(temp) log(light intensity) w/ outliers w/o outliers Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 4 / 43 Types of outliers Which of the below best describes the outlier? (a) influential (b) low leverage (c) high leverage (d) none of the above Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 41 / 43 Types of outliers Does this outlier influence the slope of the regression line? 1 Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 42 / 43 Recap Which of following is true? (a) Influential points always change the intercept of the regression line. (b) High leverage points always reduce R 2. (c) All outliers are influential points. (d) When the data set includes an influential point, the relationship between the explanatory variable and the response variable is always nonlinear. (e) None of the above. Statistics 1 (Mine Çetinkaya-Rundel) L: Simple Linear Regression March 29, 2 43 / 43