Intro to Linear Regression

Transcription

1 Intro to Linear Regression

2 Introduction to Regression Regression is a statistical procedure for modeling the relationship among variables to predict the value of a dependent variable from one or more predictor variables. Imagine that I ask you to guess the weight of a college-aged male who is hidden from view What would your best guess be?

3 Introduction to Regression weight weight

4 Introduction to Regression Regression is a statistical procedure for modeling the relationship among variables to predict the value of a dependent variable from one or more predictor variables. Imagine that I ask you to guess the weight of a college-aged male who is hidden from view What would your best guess be? What if I also gave you his height? Intuitively, it should be clear that you can do better

5 Introduction to Regression

6 Introduction to Regression The Pearson correlation, which we covered in the last lecture measures the degree to which a set of data points form a linear (straight line) relationship. Simple regression describes the linear relationship between a dependent variable () and one predictor variable (X) The resulting line is called the regression line.

7 Regression and Linear Equations ou should remember the following from your high school algebra course: Any straight line can be represented by an equation of the form = bx + a, where b and a are constants. The value of b is called the slope and determines the direction and degree to which the line is tilted. The value of a is called the -intercept and determines the point where the line crosses the -axis. In the context of linear regression, a and b are called regression coefficients

8 Regression and Linear Equations b 0.5 a 1.0 ˆ bx a 0.5X 1

9 Clicker Question In the equation ˆ 12.6X 5 a) a difference of one unit in X will lead to a 5 point difference in the prediction. b) will decrease as X increases. c) a difference of one unit in X will lead to a 17.6 point difference in the prediction. d) a difference of one unit in X will lead to a 12.6 point difference in the prediction.

10 Residuals: Errors of Prediction How well a regression line fits a set of data points can be measured by calculating the distance between the data points and the line. Using the formula Ŷ = bx + a, it is possible to find the predicted value of Ŷ for any X. The residual, or error of prediction, between the predicted value and the actual value can be found by computing the difference -Ŷ The regression line is selected to be the best fit in the leastsquares sense. This means that we want to compute the line that minimizes the sum of squared residuals: SS residual ˆ 2

11 Residuals: Errors of Prediction Ŷ bx a X, ˆ

12 The Standard Error of Estimate The measure of unpredicted variability or error for the regression line is called the standard error of estimate (s e or s -Ŷ ) ou can think of it as analogous to the standard deviation if we were to use the mean M as our estimate of the variable s SS df M 2 n1 s ˆ SS df residual residual ˆ 2 n 2

13 Computing Regression Coefficients b change in change i n X SP SS X X or rs s X a M bm X

14 Example X M M s s X X cov X

15 Example: Computing Regression Coefficients M M s s X X cov X Compute r : cov X 36.8 r s s X Compute b : rs b 5.44 s 2.6 X Compute a : a M bm X So, ˆ bx a 5.44X

16 Example: Predicting from X ou are told that a college-aged male is 74 inches tall. Given the computed regression coefficients, what is your best estimate of his weight? X : height : weight ˆ bx a 5.44X ˆ

17 Example: Computing Accuracy of Prediction Regression M M s s X X cov X 36.8 r Two measures for accuracy of prediction: standard error of estimate (s e or s -Ŷ ) interpreted as standard deviation about the regression line r 2 interpreted as % of variance accounted for by regression model r 2 cov s s 2 X 2 2 X variation explained by X total variation r 2 r ˆ 2 n 1 s ˆ s 1 r n2 n

18 Example: Computing Accuracy of Prediction Regression Just as σ or s can be used to compute confidence intervals for population means, s -Ŷ can be used to compute confidence intervals for t crit ˆ df s

19 Standardized Regression The standardized regression coefficient (β) is computed by first standardizing both the predictor and dependent variables (i.e., by converting both the X values and the values to z-scores) and then computing the regression coefficient (b) on the transformed scores For standardized regression, the y-offset is always zero For standardized regression with a single predictor variable, β is always equal to r. Standardized regression coefficients are only really useful in multiple regression, where there are multiple predictor variables). In these cases, standardizing can make it easier to determine the relative contribution of the different predictor variables to the regression model

20 Multiple Regression Often, researchers measure several variables that are hypothesized to predict a particular dependent variable For example, we might be interested in how well both SAT scores and high school GPAs predict college GPA s Multiple regression is an appropriate tool for such situations. Multiple regression describes the linear relationship between multiple predictor variables (X 1,,X n ) and one criterion variable () The resulting surface is called the regression surface.

21 Multiple Regression with Two Predictor Variables In the same way that linear regression produces an equation that uses values of X to predict values of, multiple regression produces an equation that uses two different variables (X 1 and X 2 ) to predict values of. The equation is determined by a least squared error solution that minimizes the squared distances between the actual values and the predicted values. For two predictor variables, the general form of the multiple regression equation is: Ŷ= b 1 X 1 + b 2 X 2 + a The resulting plane is called the regression plane.

22 Multiple Regression

23 Multiple Regression

24 Multiple Regression