Chi-square tests. Unit 6: Simple Linear Regression Lecture 1: Introduction to SLR. Statistics 101. Poverty vs. HS graduate rate

Transcription

1 Review and Comments Chi-square tests Unit : Simple Linear Regression Lecture 1: Introduction to SLR Statistics 1 Monika Jingchen Hu June, 20 Chi-square test of GOF k χ 2 (O E) 2 = E i=1 where k = total number of cells/categories, df = k-1 Chi-square test of independence k χ 2 (O E) 2 df = E i=1 where k = total number of cells = R C, df = (R - 1) (C - 1) Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 2 / 30 Modeling numerical variables Modeling numerical variables So far we have worked with single numerical and categorical variables, and explored relationships between numerical and categorical, and two categorical variables. In this unit we will learn to quantify the relationship between two numerical variables, as well as modeling numerical response variables using a numerical or categorical explanatory variable. In the next unit we ll learn to model numerical variables using many explanatory variables at once. Modeling numerical variables Poverty vs. HS graduate rate The scatterplot below shows the relationship between HS graduate rate in all 50 US states and DC and the % of residents who live below the poverty line (income below $23,050 for a family of 4 in 20). Response? Explanatory? Relationship? Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 3 / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 4 / 30

2 Correlation Quantifying the relationship Guessing the correlation Correlation 1. Which of the following is the best guess for the correlation between and? Correlation describes the strength of the linear association between two variables. It takes values between -1 (perfect negative) and +1 (perfect positive). A value of 0 indicates no linear association. (a) 0. (b) (c) -0.1 (d) 0.02 (e) Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 5 / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 / 30 Guessing the correlation Correlation 2. Which of the following is the best guess for the correlation between and % HS female householder? Assessing the correlation Correlation 3. Which of the following is has the strongest correlation, i.e. correlation coefficient closest to +1 or -1? (a) 0.1 (b) -0. (c) -0.4 (d) 0.9 (e) 0.5 % female householder, no husband present (a) (c) (b) (d) Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 7 / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 / 30

3 Residuals Residuals Residuals Residuals are the leftovers from the model fit: Data = Fit + Residual Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 9 / 30 Residuals (cont.) Residual Residual is the difference between the observed and predicted y. y^ y RI 4. y 5.44 e i = y i ŷ i y^ DC % living in poverty in DC is 5.44% more than predicted. % living in poverty in RI is 4.% less than predicted. Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 / 30 Best line Best line A measure for the best line The least squares line We want a line that has small residuals: 1 Option 1: Minimize the sum of magnitudes (absolute values) of residuals e 1 + e e n 2 Option 2: Minimize the sum of squared residuals least squares Why least squares? e e e2 n 1 Most commonly used 2 Easier to compute by hand and using software 3 In many applications, a residual twice as large as another is more than twice as bad Notation: Intercept: ŷ = β 0 + β 1 x predicted y slope explanatory variable intercept Parameter: β 0 Point estimate: b 0 Slope: Parameter: β 1 Point estimate: b 1 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 / 30

4 The least squares line The least squares line Given... Slope (x) (y) mean x =.01 ȳ = sd s x = 3.73 s y = 3.1 correlation R = 0.75 The slope of the regression can be calculated as b 1 = s y s x R. Intercept The intercept is where the regression line intersects the y-axis. The calculation of the intercept uses the fact the a regression line always passes through ( x, ȳ). b 0 = ȳ b 1 x Application exercise: Estimating regression coefficients Calculate the slope and the intercept of the regression line for predicting from. Write out an interpretation of the slope coefficient, and then determine which of the following is the best interpretation of the intercept? (a) For each % point increase in HS graduate rate, % living in poverty is expected to increase on average by 4.%. (b) For each % point decrease in HS graduate rate, % living in poverty is expected to increase on average by 4.%. (c) Having no HS graduates leads to 4.% of residents living below the poverty line. (d) States with no HS graduates are expected on average to have 4.% of residents living below the poverty line. (e) In states with no HS graduates % living in poverty is expected to increase on average by 4.%. Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 / 30 The least squares line Recap: Interpreting the slope and the intercept Interpretation of slope and intercept Intercept: When x = 0, y is expected to equal the intercept. Slope: For each unit increase in x, y is expected to increase/decrease on average by the slope. Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 / 30

5 Prediction Using the linear model to predict the value of the response variable for a given value of the explanatory variable is called prediction, simply by plugging in the value of x in the linear model equation. There will be some uncertainty associated with the predicted value - we ll talk about this next time Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Extrapolation Applying a model estimate to values outside of the realm of the original data is called extrapolation. Sometimes the intercept might be an extrapolation. intercept Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 / 30 Examples of extrapolation Examples of extrapolation Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30

6 Examples of extrapolation 1 Linearity 2 Nearly normal residuals 3 Constant variability Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Conditions: (1) Linearity Anatomy of a residuals plot The relationship between the explanatory and the response variable should be linear. Methods for fitting a model to non-linear relationships exist, but are beyond the scope of this class. Check using a scatterplot of the data, or a residuals plot. 15 RI: = 1 =.3 % in poverty = =.4 e = % in poverty =.3.4 = 4. 5 DC: = =. % in poverty = = 11.3 e = % in poverty = = 5.44 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30

7 Conditions: (2) Nearly normal residuals Conditions: (3) Constant variability frequency The residuals should be nearly normal. This condition may not be satisfied when there are unusual observations that don t follow the trend of the rest of the data. Check using a histogram or normal probability plot of residuals. Sample Quantiles Normal Q Q Plot The variability of points around the least squares line should be roughly constant. This implies that the variability of residuals around the 0 line should be roughly constant as well. Also called homoscedasticity. Check using a residuals plot residuals Theoretical Quantiles 0 90 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 2 / 30 Checking conditions Checking conditions 4. What condition is this linear model obviously violating? y 5. What condition is this linear model obviously violating? y (a) Constant variability (b) Linear relationship (c) Non-normal residuals (d) No extreme outliers g$residuals (a) Constant variability (b) Linear relationship (c) Non-normal residuals (d) No extreme outliers x g$residuals x Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30 Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, 20 2 / 30

8 R 2 R 2 R 2 Interpretation of R 2 The strength of the fit of a linear model is most commonly evaluated using R 2. R 2 is calculated as the square of the correlation coefficient. It tells us what percent of variability in the response variable is explained by the model. The remainder of the variability is explained by variables not included in the model. For the model we ve been working with, R 2 = = 0.3. Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30. Which of the below is the correct interpretation of R = 0.2, R 2 = 0.3 for this model? (a) 3% of the variability in the % of HS graduates among the 51 states is explained by the model. (b) 3% of the variability in the % of residents living in poverty among the 51 states is explained by the model. (c) 3% of the time uates predict % living in poverty correctly. (d) 2% of the variability in the % of residents living in poverty among the 51 states is explained by the model. Statistics 1 (Monika Jingchen Hu) U - L1: Introduction to SLR June, / 30