Least-Squares Regression

Transcription

1 MATH 203 Least-Squares Regression Dr. Neal, Spring 2009 As well as finding the correlation of paired data {{ x 1, y 1 }, { x 2, y 2 },..., { x n, y n }}, we also can plot the data with a scatterplot and find the Least Squares Line of Best Fit through the data. This line, having equation y = a x + b, provides an approximate linear functional relationship between the values of x i and y i. Of course, it is only a good fit if the correlation is near ±1, which means that there is a strong linear dependence between the measurements X and Y. The slope a is given by a = x y (x )(y ) x 2 (x ) 2, where x y = 1 n x n i y i and x 2 = 1 n x 2 i=1 n i. i=1 After the slope a is calculated, then the intercept b is given by b = y a x. Then Y a X + b. This least squares line is the line that minimizes the sum of squared errors between the actual y i values and the linear approximations ax i + b. That is, the sum n (ax i + b y i ) 2 is minimized with these choices of a and b. i=1 To compute both the correlation and the least-squares line, enter paired data into lists L1 and L2 (or some other pair of lists), press STAT, scroll to CALC, press 4 for LinReg(ax+b), then enter the command LinReg(ax+b) L1, L2. Some Other Quick Facts 1. The point ( x, y ) is always on the least-squares regression line. 2. The slope of the least-squares regression line also is given by a = r X Y, where r is the sample correlation, X = x 2 ( x ) 2, and Y = y 2 ( y ) The value r 2 is the coefficient of determination. It measures the proportion of the observed values accounted for by the regression fit. Because 0 r 2 1, an r 2 near 1 means that there is a strong fit, and an r 2 near 0 means that there is virtually no fit of the data.

2 Examples. (i) Make a scatterplot; (ii) Compute r and explain what it means; (iii) Find the equation of the least-squares regression line and graph it through the scatterplot. 1. Is there a relationship between the tar and nicotine levels in cigarettes? Brand Tar (mg) Nicotine (mg) Alpine Benson & Hedges Bull Durham Camel Lights Carlton Chesterfield Golden Lights Kent Kool L&M We see the general trend: As the tar level increases, then the nicotine level increases. STAT CALC Enter command r Because r is so close to +1, there is a strong positive linear relationship between tar and nicotine. The least-squares regression line is given by y = a x + b x Due to the close fit, this linear function could be used to predict a nicotine level y for a given tar level x. For instance, if x = 20 mg of tar, then y mg of nicotine. Y= From VARS STATISTICS EQ Press ENTER From CALC (2nd TRACE) value, X = 20 Because r , nicotine level is 97.4% determined by tar level when using y = x as a predictor, and 2.6% determined by other factors.

3 2. If a person has high body density, then they should have less body fat. The following data lists measurements of body densities and percentages of body fat from a random sample of people. Is the relationship observable? Person Body Density Body Fat % % % % % % % % % % We see the strong trend: As body density increases, then body fat decreases. STAT CALC Enter command r Because r is so close to 1, there is a strong negative linear relationship between body density and body fat. The least-squares regression line is given by y = a x + b x , which in this case gives an almost perfect linear fit. Y= From VARS STATISTICS EQ Press ENTER Because r , we can say that body fat is 99.99% determined by one s body density when using y = x as a predictor.

4 3. Is there a relationship between driving speed and MPG for your gas-guzzling SUV? Speed (mph) MPG There clearly seems to be some sort of relationship (perhaps quadratic). The mpg increases as speed increases to a certain point; but then as speed increases further, the mpg drops off. There is no correlation! STAT CALC r = 0. The correlation measures the strength and direction of the linear association between the variables. Just because the correlation equals 0, it does not mean that there is no relationship. In this case, there simply is not a permanent linear relationship between speed and mpg; but there certainly is a relationship. Here, the least-squares regression line is constant and is given by y = In this case, the least-squares regression line is not a good fit of the data. However, it is the best linear approximation of the data, which really does no good here because there is no permanent linear relationship between speed and mpg. Because r 2 = 0, the line does not fit the data at all. Using the line y = 19.6 as a predictor, then the mpg is not at all determined by its speed.

5 4. Is there a relationship between height and GPA? The following data is a collection of measurements from a random sample of WKU students. Student Height (inches) GPA There does not appear to be any relationship between height and grade point average. High and low (and middle) GPAs are attained from students of all heights. The correlation is nearly 0. STAT CALC r When there is no relationship whatsoever between the variables, then we say that they are independent. When variables are independent, then the true correlation will equal 0. So the correlation coefficient from a random sample of measurements should be very close to 0 when the two variables have no association between them. Here, the least-squares regression line is given by y = x Because the correlation is nearly 0, there is no linear relationship between height and GPA. (In fact, there is no relationship at all because GPA is independent of height.) So again, the least-squares regression line is not a good fit of the data. Because r , less than 1% of a person s GPA is determined by height when using the line y = x as a predictor.

6 5. Sixteen binge drinkers at Ohio State University had a beer party. Thirty minutes later, campus police measured their blood alcohol content (BAC). Here are the data: Student # Beers BAC Student # Beers BAC Is there a correlation between BAC and the number of beers drunk by the student? What would you predict the average BAC to be for people having 6 beers? The BAC can vary from person to person. For instance, the BACs of the students having 3 beers were 0.02, 0.04, and But a general trend exists: As the number of beers increases, so does the blood alcohol content. STAT CALC r The high correlation shows that there is a relatively strong positive linear relationship. But the relationship is not precisely linear, perhaps due to the varying effects of alcohol on different people. Here, the least-squares regression line is given by y 0.018x , which can be interpreted as giving the average BAC for the various amounts of beer. Evaluating the line for x = 6 beers, we obtain an average BAC of about 0.095, which is well above the legal limit! So don't drink and drive! From CALC (2nd TRACE) value With r , the blood alcohol content is 80% determined by the number of beers when using y x as a predictor. (The BAC is 20% determined by other factors such as body weight, male or female, etc. )