Least Squares Regression

Transcription

1 Least Squares Regression Sections 5.3 & 5.4 Cathy Poliak, Ph.D. Office in Fleming 11c Department of Mathematics University of Houston Lecture Cathy Poliak, Ph.D. Office in Fleming 11c Sections (Department 5.3 & 5.4of Mathematics University of Lecture Houston 14 ) / 32

2 Outline 1 Least-Squares Regression 2 Prediction 3 Coefficient of Determination 4 Residuals 5 Residual Plots Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Sections (Department 5.3 & 5.4of Mathematics University of Lecture Houston 14 ) / 32

3 Popper Set Up Fill in all of the proper bubbles. Make sure your ID number is correct. Make sure the filled in circles are very dark. This is popper number 07. Cathy Poliak, Ph.D. Office in Fleming 11c Sections (Department 5.3 & 5.4of Mathematics University of Lecture Houston 14 ) / 32

4 Scatterplots and Correlation We are wanting to know the relationship between two random variables. To quickly look at the relationship we use the scatterplot and correlation to determine: Direction (positive or negative) Strength (weak, moderately weak, strong) Form (linear or non-linear) Cathy Poliak, Ph.D. Office in Fleming 11c Sections (Department 5.3 & 5.4of Mathematics University of Lecture Houston 14 ) / 32

5 Popper 07 Questions We are looking at the relationship between how many items are on a shelf (shelf space) and the number of items sold in a week. The correlation coefficient is r = What is the direction of the relationship between shelf space and weekly sales? a) Positive b) Negative c) No direction 2. What is the strength the relationship between shelf space and weekly sales? a) Strong b) Moderate c) Weak 3. What form appears to be in the relationship between shelf space and weekly sales? a) Linear b) Non-linear Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Sections (Department 5.3 & 5.4of Mathematics University of Lecture Houston 14 ) / 32

6 Popper 07 Questions Use the following scatterplot to answer the questions below. y x 4. What is the correlation coefficient for this set of data? a. 1 b c d If both X and Y are multiplied by 2, what would happen to the correlation coefficient? a. It will stay the same. c. It will be multiplied by 2. b. It will be divided by 2. d. It will be zero. Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Sections (Department 5.3 & 5.4of Mathematics University of Lecture Houston 14 ) / 32

7 Example The marketing manager of a supermarket chain would like to use shelf space to predict the sales of coffee. A random sample of 12 stores is selected, with the following results. Store Shelf Space (ft) Weekly Sales (# sold) Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Sections (Department 5.3 & 5.4of Mathematics University of Lecture Houston 14 ) / 32

8 Scatterplot Number Sold Shelf Space(feet) Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Sections (Department 5.3 & 5.4of Mathematics University of Lecture Houston 14 ) / 32

9 Examining relationships Correlation measures the direction and strength of the straight-line relationship between two quantitative variables. If a scatterpolt shows a linear relationship, we would like to summarize this overall pattern by drawing a line on the scatterplot. A regression line is a straight line that describes how a response variable y changes as an explanatory variable x changes. This equation is used when one of the variables helps explain or predict the other. Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Sections (Department 5.3 & 5.4of Mathematics University of Lecture Houston 14 ) / 32

10 Least-Squares regression The least-squares regression line (LSRL) of Y on X is the line that makes the sum of the squares of the vertical distances of the data points from the line as small as possible / 32

11 Least-Squares Number Sold Shelf Space(feet) / 32

12 Equation of the least-squares regression line Let x be the explanatory variable and y be the response variable for n individuals. From the data calculate the means x and ȳ and the standard deviations s x and s y of the two variables, and their correlation r. The least-squares line is the equation ŷ = a + bx / 32

13 Equation of the least-squares regression line The least-squares line is the equation ŷ = a + bx. In the example of the supermarket sales, let Y = weekly sales and X = shelf space. The least-squares regression equation to predict weekly sales based on shelf space is ŷ = x / 32

14 Calculating the least squares regression equation by hand Let X be the explanatory variable and Y be the response variable for n individuals. 1. From the data calculate the means x and ȳ and the standard deviations s x and s y of the two variables, and their correlation r. 2. Calculate the slope: 3. Calculate the y-intercept: b = r s y s x a = ȳ b x 4. Then the equation is: ŷ = a + bx. Where you input the slope into b and the y-intercept into a and leave y and x alone (do not put any numbers into y and x) / 32

15 Finding Equation for Coffee Sales Given these values determine the least-square regression line (LSRL) equation for predicting number sold based on shelf space. Explanatory variable: Shelf space (X) x = 12.5 feet, s x = feet. Response variable: Sales (Y ) ȳ = units sold, s y = units sold. Correlation: r = / 32

16 Example of LSLR for Cost of Car The following are descriptive statistics for estimating cost of the car Y by the age of the car (X), r = Estimated Cost (Y ) ȳ = s y = Age (X) x = s x = Determine the Least Squares Equation (LSLR) of the data / 32

17 Least-Square Regression Line Using R Use the command: lm(yvariable Xvariable). For example of the coffee sales. > shelf=c(5,5,5,10,10,10,15,15,15,20,20,20) > sales=c(160,220,140,190,240,260,230,270,280,260,290,310) > lm(sales~shelf) Call: lm(formula = sales ~ shelf) Coefficients: (Intercept) shelf Where the Intercept value is a and the other value is b, the slope. Thus the equation in this example is: ŷ = x / 32

18 Least-Squares Regression Line Using TI-83(84) 1. Make sure the diagnostics is turned on by clicking 2ND CATALOG and scroll down to Diagnostics. 2. Choose STAT CALC then 8:LinReg(a+bx). 3. Make sure your Xlist is L1 and Ylist is L2 and select Calculate. 4. a is your y-intercept and b is your slope / 32

19 Prediction The equation of the regression line makes prediction easy. Substitute an x-value into the equation. Predict the weekly sales of coffee with shelf space of 12 feet. ŷ = = Thus the is the predicted weekly number of units of coffee sold that is has 12 feet of shelf space / 32

20 Popper Questions The least-squares regression line equation to predict cost of the car(y ) by the age of the car (X) is: ŷ = x. 6. Predict the cost of an automobile for a 5 year old car. a) $18358 b) $1534 c) $10688 d) $ Predict the cost of an automobile for a 20 year old car. a) $18358 b) -$1534 c) -$12,322 d) $ / 32

21 Facts about least-squares regression Fact 1: A change of one standard deviation in x corresponds to a change of r standard deviations in y. (b 1 slope) Fact 2: The least-squares regression line always passes through the point ( x, ȳ). That is why we can get b 0 the y-intercept. Fact 3: The distinction between explanatory and response variables is essential in regression / 32

22 R 2 The square of the correlation R 2 describes the strength of a straight-line relationship. The formal name of it is called the Coefficient of Determination. This is the percent of variation of Y that is explained by this equation. R 2 is a measure of how successful the regression was in explaining the response / 32

23 R 2 for coffee sales The correlation is r = The coefficient of determination is R 2 = = This means that 68.4% of the variation in coffee sales can be explained by the equation / 32

24 Example The following is an output of a least-squares regression equation in the TI-84. What percent of the variation in the y-variable can be explained by this regression equation? a) 67% b) % c) 77% d) 88% / 32

25 Is good at predicting the response? The coefficient of determination, R 2 is the percent (fraction) of variability in the response variable (Y ) that is explained by the least-squares regression with the explanatory variable. This is a measure of how successful the regression equation was in predicting the response variable. The closer R 2 is to one (100%) the better our equation is at predicting the response variable. From the previous question we can say that 84.33% of the variation in the monthly gas bills can be explained by the least squares regression line equation / 32

26 Is the LSLR good at predicting the response? A residual is the difference between an observed value of the response variable and the value predicted by the regression line. residual = observed y predicted y We can determine residuals for each observation. The closer the residuals are to zero, the better we are at predicting the response variable. We can plot the residuals for each observation, these are called the residual plots / 32

27 Example of residuals The regression equation to determine number of units sold (y) for coffee by shelf space (x) is: ŷ = x. A store that has 10 feet of space sold 260 units of coffee. Determine the residual for this store. 1. Determine the predicted units sold for x = The observed units sold is the given value The residual is the difference between the observed y and the predicted y / 32

28 Residuals of coffee sales The regression equation is Weekly Sales = Shelf Space Shelf Observed Predicted Residual Space Weekly Sales Weekly Sales / 32

29 Residual Plots The plot of the residual values against the x values can tell us about our LSRL model. This is a scatterplot of where the x-values are on the horizontal axis and the residuals are on the vertical axis. In R: plot(x,resid(lm(y x))) If the equation is a good way of predicting the y-variable (our model is a good fit), the residual plot will be a bunch of scattered points with no pattern that is centered about zero on the vertical axis / 32

30 Residual Plot residuals shelf / 32

31 R code > shelf=c(5,5,5,10,10,10,15,15,15,20,20,20) > sales=c(160,220,140,190,240,260,230,270,280,260,290,310) > plot(shelf,resid(lm(sales~shelf)),ylab="residuals") > abline(0,0) / 32

32 Examining a residual plot A curved pattern shows that the relationship is not linear. Increasing spread about the zero line as x increases indicates the prediction of y will be less accurate for larger x. Decreasing spread about the zero line as x increases indicates the prediction of y to be more accurate for larger x. Individual points with larger residuals are considered outliers in the vertical (y) direction. Individual points that are extreme in the x direction are considered outliers for the x-variable / 32