Least-Squares Regression

Transcription

1 MATH 482 Least-Squares Regressio Dr. Neal, WKU As well as fidig the correlatio of paired sample data {{ x 1, y 1 }, { x 2, y 2 },..., { x, y }}, we also ca plot the data with a scatterplot ad fid the least squares regressio lie through the data. This lie, deoted y = a x + b, provides a approximate liear fuctioal relatioship betwee the values of x i ad y i. Of course, it is oly a good fit if the correlatio is ear ±1, which meas that there is a strog liear depedece betwee the measuremets X ad Y. The slope a is give by a = x y x y x 2 x 2, where x y = 1 x i y i ad x 2 = 1 x 2 i = 1 i. i = 1 After the slope a is calculated, the the itercept b is give by b = y a x. The we have y = a x + b. Theorem. The least squares lie is the lie that miimizes the sum of squared errors betwee the actual y i values ad the liear approximatios a x i + b. That is, the sum (a x i + b y i ) 2 is miimized with these choices of a ad b. Proof. Let f (a,b) = (a x i + b y i ) 2. We take the first partial derivatives with respect to a ad b, equate them to 0, the solve for a ad b. First, 0 = f b = 2(a x i + b y i ) which gives 0 = a x i + b y i = a x + b y. Solvig for b gives b = y a x. i= 1 i=1 Note that the 2d derivative of f with respect to b equals 2 which is always positive. Thus the critical poit of b = y a x does yield a miimum value by the 2d deriv. test. Next, we have 0 = f a = 2(a x 2 i + b y i )x i = 2a x i + 2b x i 2 x i y i = 2a x 2 + 2b x 2x y. i=1 Thus, 0 = a x 2 + b x x y = a x 2 + (y a x ) x x y. Solvig for a gives a = x y x y x 2 x 2. Now the 2d derivative of f with respect to a equals 2 x 2 So the critical poit value for a does yield a miimum value. which is always positive.

2 To compute both the sample correlatio r ad the least-squares lie o a TI, eter paired data ito lists L1 ad L2 (or some other pair of lists), press STAT, scroll to CALC, press 4 for LiReg(ax+b), the eter the commad LiReg(ax+b) L1, L2. Some Other Quick Facts 1. The poit ( x, y ) is always o the least-squares regressio lie. 2. The slope of the least-squares regressio lie also is give by a = r σ Y, where σ X r = x y x y = x y x y x 2 x 2 y 2 y 2 σ, ad σ X σ X = Y x 2 x 2, ad σ Y = y 2 y The value r 2 is the coefficiet of determiatio. It measures the proportio of the observed values accouted for by the regressio fit. Because 0 r 2 1, a r 2 ear 1 meas that there is a strog fit, ad a r 2 ear 0 meas that there is virtually o fit of the data. Examples. (i) Make a scatterplot; (ii) Compute r ad explai what it meas; (iii) Fid the equatio of the least-squares regressio lie ad graph it through the scatterplot. (iv) Explai what r 2 meas i each case. 1. Aalyze the relatioship betwee the tar ad icotie levels i cigarettes. Brad Tar (mg) Nicotie (mg) Alpie Beso & Hedges Bull Durham Camel Lights Carlto Chesterfield Golde Lights Ket Kool L&M STAT EDIT STAT PLOT Adjust Settigs ZOOM 9 We see a geeral tred: As the tar level icreases, the icotie level seems to icrease.

3 STAT CALC Eter commad r Because r is so close to +1, there is a strog positive liear relatioship betwee tar ad icotie. As the tar level icreases, the icotee level teds to icrease liearly. The least-squares regressio lie is give by y = a x + b x Due to the close fit, this liear fuctio could be used to predict a icotie level y for a give tar level x. For istace, if x = 20 mg of tar, the y mg of icotie. Y= From VARS STATISTICS EQ Press ENTER From CALC (2d TRACE) value, X = 20 Here r Usig the lie y = x as a predictor, icotie level is 97.4% determied by tar level ad 2.6% determied by other factors. 2. If a perso has high body desity, the they should have less body fat. The followig data lists measuremets of body desities ad percetages of body fat from a radom sample of people. Is the relatioship observable? Perso Body Desity Body Fat % % % % % % % % % % STAT EDIT STAT PLOT Adjust Settigs ZOOM 9 We see the strog tred: As body desity icreases, the body fat decreases.

4 STAT CALC Eter commad r Because r is so close to 1, there is a strog egative liear relatioship betwee body desity ad body fat. As body desity icreases, the body fat teds to decrease liearly. The least-squares regressio lie is give by y = a x + b x , which i this case gives a almost perfect liear fit. Y= From VARS STATISTICS EQ Press ENTER Because r , we ca say that whe usig body fat is 99.99% determied by oe s body desity. as a predictor, the 3. Is there a relatioship betwee drivig speed ad MPG for your gas-guzzlig SUV? Speed (mph) MPG STAT EDIT STAT PLOT Adjust Settigs ZOOM 9 There clearly seems to be some sort of relatioship (perhaps quadratic). The mpg icreases as speed icreases to a certai poit; but the as speed icreases further, the mpg drops off.

5 There is o correlatio! STAT CALC r = 0. The correlatio measures the stregth ad directio of the liear relatioship betwee the variables. Just because the correlatio equals 0, it does ot mea that there is o relatioship. I this case, there simply is ot a permaet liear relatioship betwee speed ad mpg; but there certaily is a relatioship. Here, the least-squares regressio lie is costat ad is give by y = I this case, the least-squares regressio lie is ot a good fit of the data. However, it is the best liear approximatio of the data, which really does o good here because there is o permaet liear relatioship betwee speed ad mpg. Because r 2 = 0, the lie does ot fit the data at all. Usig the lie y = 19.6 as a predictor, the the mpg is ot at all determied by its speed. 4. Is there a relatioship betwee height ad GPA? The followig data is a collectio of measuremets from a radom sample of WKU studets. Studet Height (iches) GPA STAT EDIT STAT PLOT Adjust Settigs ZOOM 9

6 There does ot appear to be ay relatioship betwee height ad grade poit average. High ad low (ad middle) GPAs are attaied from studets of all heights. STAT CALC r The correlatio is early 0. Whe there is o relatioship whatsoever betwee the variables, the we say that they are idepedet. Whe variables are idepedet, the the true correlatio will equal 0. So the correlatio coefficiet from a radom sample of measuremets should be very close to 0 whe the two variables have o associatio betwee them. Here, the least-squares regressio lie is give by y = x Because the correlatio is early 0, there is o liear relatioship betwee height ad GPA. (I fact, there is o relatioship at all because GPA is probably idepedet of height.) So agai, the least-squares regressio lie is ot a good fit of the data. Because r , whe usig the lie = as a predictor, a perso s GPA is 0.35% determied by height ad 99.65% determied by other factors. 5. Sixtee bige drikers at Ohio State Uiversity had a beer party. Thirty miutes later, campus police measured their blood alcohol cotet (BAC). Here are the data: Studet # Beers BAC Studet # Beers BAC Is there a correlatio betwee BAC ad the umber of beers druk by the studet? What would you predict the average BAC to be for people havig 6 beers?

7 STAT EDIT STAT PLOT Adjust Settigs ZOOM 9 The BAC ca vary from perso to perso. For istace, the BACs of the studets havig 3 beers were 0.02, 0.04, ad But a geeral tred exists: As the umber of beers icreases, the blood alcohol cotet teds to icrease. STAT CALC r The high correlatio shows that there is a relatively strog positive liear relatioship. But the relatioship is ot precisely liear, perhaps due to the varyig effects of alcohol o differet people. Here, the least-squares regressio lie is give by y x , which ca be iterpreted as givig the average BAC for the various amouts of beer. Evaluatig the lie for x = 6 beers, we obtai a average BAC of about 0.095, which is well above the legal limit! So do't drik ad drive! From CALC (2d TRACE) value Here r Usig y x as a predictor, the blood alcohol cotet is 80% determied by the umber of beers, ad 20% determied by other factors such as body weight, male or female, amout of food i stomach, etc.