Steps to take to do the descriptive part of regression analysis:

Transcription

1 STA 2023 Simple Linear Regression: Least Squares Model Steps to take to do the descriptive part of regression analysis: A. Plot the data on a scatter plot. Describe patterns: 1. Is there a strong, moderate, or weak relationship between the two variables? 2. Is the relationship between the two variables positive or negative? 3. Is the relationship linear or nonlinear? * If nonlinear, don't fit a linear model to this data; learn some nonlinear options. TI 83+: enter X data into L1 and Y data into L2: stat, edit, enter scatter plot: 2nd statplot, enter, enter so that plot one is ON type = 1st plot on first row xlist = L1 and ylist = L2 mark = your choice zoom, #9 which is zoom stat trace for coordinates B. Compute the coefficients for the regression line. Write the equation. Plot the line on the scatter plot. TI 83+: to get coefficients, a and b 1 : stat, calc, #4 LinReg (ax+b), enter to graph regression line on scatter plot: compute the regression as above, but add a destination for the result: stat, calc, #4 LinReg (ax+b) L1, L2, Y1 To get to the variable Y1, press VARS > Y Vars > #1: Function #1: Y1 Then press either Graph or Zoom, #9 C. Interpret the slope of the regression line in terms of the variables. D. (Compute) and interpret I, the correlation coefficient. Compare your answer to your visual interpretations in part A. r measures the strength of the linear relationship between x and y. 1 r 1 The first time you do this, you might have to set the following to get r and R 2 to appear:

2 2 From a clear screen: 2nd 0 (Catalog), scroll down to Diagnostic On, press enter twice. When it says done, the above commands should print out r and R 2. E. (Compute) and interpret R 2, the coefficient of determination. R 2 measures the percent of variation in variable y explained or accounted for by the linear model containing x. 0 R 2 1 TI 83+: see above in part D. This is a fill in the blank page that I ve completed for the following example. My suggestion is for you to do such a page for each data set you analyze on your homework. An example: This information represents ten students from a computer literacy course: X is the number of study hours on the computer at the lab and Y is the final exam score. X Y A. The scatter plot shows a strong, positive, linear relationship between the number of study hours spent in the lab and final exam score. B. The equation of the least squares regression line is y =.83X C. The slope is.83 To help interpret slope, map out the following: rise Slope = run = change in final exam score change in number of study hours =.83 1 So, on average, for each additional study hour, final exam score increases by.83 point. I think there s a better scale for our data. Multiply numerator and denominator by X = 8.3 I ve changed the scale, but not the value (the ratio of rise to run) of the slope. 10 On average, for each additional 10 study hours, final exam score increases by 8.3 points. Now, that s better. D. The correlation coefficient is r. r =.94 This indicates a strong, positive, linear relationship between study hours and final exam score.

3 3 E. The coefficient of determination is R 2. R 2 =.88 This indicates that 88% of the variation in final exam scores is explained by this linear model (see part B) that uses number of study hours as a predictor. This indicates a useful predictor has been identified. Notes on Simple Linear Regression: Regress means to revert to more typical values such as the mean. Until now in our course, to predict an average final exam score for a computer course, we would collect a random sample of prior scores, assume things will work similarly this semester, compute the sample mean, and perhaps put a confidence interval around it so we could report a margin of error for our estimate. Regression allows us to take additional advantage of other, related variables that might help us better predict exam score. In simple linear regression, we include one predictor variable for a linear model (recall y = mx + b). An example follows: Y = final exam score Y = predicted final exam score X = number of study hours in the computer lab Linear Model/Least Squares Regression Model/Simple Linear Regression Model: Y = a X + b Y = 0.83X is the y intercept of the line of best fit through the data points 0.83 is the slope of the line of best fit through the data points "Best fit" is defined by the line that minimizes the sum of the squared errors (residuals) between the actual Y values and the predicted Y values. See end of this handout. Y is the variable being predicted. Y is the dependent variable. X is the predictor variable. X is an independent variable. Y depends on X, at least partially. Linear Regression is a tool to use when: 1. you have two quantitative variables, one of which you think will help predict the other 2. there is a strong, linear correlation between the two variables

4 4 Two ways to assess the strength of the linear correlation between the two variables: 1. scatter plot 2. correlation coefficient, r See Sullivan: pages for pictures and properties of the correlation coefficient. When the proper assumptions are met (see text), a linear regression model can be a useful tool. However, when the model is constructed, our work is not done! We must check the model somehow to see how good it is. That is, how closely will the predicted Y values come to the actual Y values? How much of the variation in the Y values is explained by the model with the selected X variable in it? There are numerical values that assess these aspects of the model. R 2, the coefficient of determination, is one numerical value that helps determine how good the model is. It tells the percentage of variation in the Y values explained by the variation in the X variable's values when using the constructed model. For our example, R 2 =.878. This means that almost 88% of the variation in final exam score was explained by the number of study hours in the computer lab, when using the least squares, linear regression model. I'd say that's a strong predictor variable! One must be careful to only make appropriate predictions. Do not predict a value of Y for any value of X that is outside the range of the X values used to create the model. This is called extrapolation. Also, one must not confuse correlation with causation. Cause and effect must be proven with appropriate experimental design studies. This isn't the whole story, obviously. However, it gives you a broad picture of what's possible with regression. Further topics include the study of the residuals to assess the utility of the model, Adding other predictor variables to the model to increase predictability, Applying nonlinear models to increase predictability, etc.

5 5 Residuals X: #study Y: final Y Y Y (Y Y ) 2 hours score Σ (Y Y ) = 0 Σ is minimized

6 6 Follow steps A E to do the descriptive part of a linear regression analysis for each example. Number 1 X: amount of fertilizer (lbs/100 sq ft) Y: yield of tomatoes (lbs) X Y Number 2 X: hours/week work Y: GPA X Y Number 3: X: maps distributed (1000's) Y: increase in ridership (1000's) X Y

7 7 Number 4 X: age (years) Y: days absent X Y Number 5 X: newspaper advertising ($1000's) Y: sales ($1000's) X Y

8

9 9 1. For the regression line y = x, predict y for an x value of 10. (A) 40 (B) 50 (C) 40 (D) 30 (E) Given the following information, find the correct value of the correlation coefficient. b 0 = 25.2 b 1 =.86 R 2 =.79 (A).62 (B).62 (C).79 (D).89 (E).89 ****************************************************************************** USE THE FOLLOWING INFORMATION FOR QUESTIONS 3 5: Following are the results of a simple linear regression analysis of performance score on number of days in training. Use the information below to answer these questions. (Statisticians often say, Regress y on x. So, x is number of days of training, and y is score. r = 0.98 R 2 = 0.97 b 0 (y int.) = 8.59 b 1 (slope) = Which one of the following is the correct equation for the regression line? (A) y = x (B) y = x (C) y = x (D) y = x (E) y = x 4. Interpret the slope of the least squares line. (A) For each additional day of training, we estimate the performance score will increase points. (B) For each additional day of training, we estimate the performance score will increase 8.59 points. (C) For a person with 0 days of training, we estimate the performance score to be 8.59 points. (D) For each additional performance score point, we estimate the number of days of training will increase by 8.59 days. 5. Interpret R 2, the coefficient of determination. (A) We are 97% confident that number of days in training is linearly related to performance score.

10 10 (B) 97% of the sum of the squared errors of prediction (SSE) can be explained by the linear relationship between number of days in training and performance score. (C) The least squares line makes correct predictions 97% of the time. (D) 97% of the total variation in performance score can be explained by the linear relationship between number of days of training and performance score. (E) 97% of the predicted performance scores will have residuals equal to zero.