Regression. Chapter 11 Part 4. More than you ever wanted to know about how to interpret the computer printout

Size: px

Start display at page:

Download "Regression. Chapter 11 Part 4. More than you ever wanted to know about how to interpret the computer printout"

Alberta Hart
5 years ago
Views:

1 Regreo Chapter Part 4 More tha you ever wated to kow about how to terpret the computer prtout February 7, 009

2 Let go back to the etrol/brthweght problem. We are ug the varable bwt00 for brthweght o brthweght meaured gram. The varable etrol meaured mg/4 hour.. de Cota data from W:\WP5\Bometry\AAABotatSprg008\Hadout\Chapter \Data\GreeeTouch.dta ob: 3 Gree-Touchtoe Study var: 5 4 Feb 008 5:05 ze: 65 (99.9% of memory free) torage dplay value varable ame type format label varable label d float %9.0g etrol float %9.0g Etrol mg/4 hr brthweght float %9.0g Brthweght g/00 bwt00 float %9.0g Brthweght gm age byte %8.0g Age of Mom year Sorted by: etrol Brth weght gram Ug value of etrol to predct brth weght Etrol mg/4 hr The varable etrol called the depedet varable, the explaatory varable or the predctor varable. The varable brthweght called the depedet varable or the repoe varable. We have regreed brthweght o etrol. We are tryg to predct brthweght ug etrol. Page --

3 We wll tart wth the table the upper left-had corer of the output. It called a ANOVA (aaly of varace) table.. regre bwt00 etrol Source SS df MS Number of ob = F(, 9) = 7.6 Model Prob > F = Redual R-quared = Adj R-quared = Total Root MSE = bwt00 Coef. Std. Err. t P> t [95% Cof. Iterval] etrol _co Source refer to the ource of varato. The ource are: Model = varato due to regreo (ee ext page for equato) Redual = varato about regreo or the error varato Total = corrected total (.e. have corrected degree of freedom to accout for etmatg the overall mea) SS = um of quare df = degree of freedom MS = mea quare = SS/df You ll ee o the ext page that I eed to be able to dtguh amog the varou um of quare ad degree of freedom, o we ll ue the otato below. SSR = um of quare due to regreo = ( y$ y) SSE = um of quare due to the redual or error = ( y y$ ) SST = total um of quare = ( y y) df = degree of freedom for the regreo (or for the model) R = umber of predctor varable ( th cae we have predctor varable) df T = degree of freedom for the total = - (total ample ze - ) Page --

4 df E = - [ th cae = ( - ) - = - ] df T df R = degree of freedom for the redual (or error) I the table below: MSR = mea quare due to the model or MS due to Regreo MSE = mea quare error or MS due to redual A ANOVA Table for the Smple Lear Regreo Source SS df MS F rato Model ( y$ y) = SSR df R = ( y$ y) df R = MSR MSR MSE Redual ( y y$ ) = SSE df df E T = df = R ( y y$ ) df E = MSE Total ( y y) = SST df T = Page -3-

5 Let u aume that the old le the ordary leat quare (OLS) regreo le for our varable bwt00 ad etrol. We kow that each par of varable aocated wth exactly oe OLS regreo le but to try to make clear the relatohp amog the compoet, I have aumed we a ecod OLS le baed o the ame data. Th mea the degree of freedom for the compoet wth be the ame for both le. Page -4-

6 Degree of Freedom To get the degree of freedom we tart wth the Total. The degree of freedom for Total = ample ze - = - Next go to the degree of freedom for the model (.e. for regreo). Degree of freedom for the model = the umber of predctor or depedet varable, whch th cae. The y- tercept ot a predctor varable. The degree of freedom for the redual = (df for Total) - (df for Model) = ( - ) - = -. Notce that MS = SS/df. MSE ( y y$ ) = MSE Aother otato for the MSE yx. Th emphaze the fact that the MSE the etmate for the commo varace of the curve at the rght. We uually deote th commo varace by σ. μ yxt μ yxp So $ σ = MSE = yx x t x p Th make ee becaue y$ p = $ μ y xp ad the uual varace volve the varablty of value about the mea of thoe value. We dvde by - to get a ubaed etmate for σ the commo varace of the ormal curve. Page -5-

7 . regre bwt00 etrol Source SS df MS Number of ob = F(, 9) = 7.6 Model Prob > F = Redual R-quared = Adj R-quared = Total Root MSE = 38. Root MSE = quare root of MSE = yx (.e. the tadard devato or $ σ ) What are: F(,9) = 7.6 ad Prob > F = :. regre bwt00 etrol Source SS df MS Number of ob = F(, 9) = 7.6 Model = MSR Prob > F = Redual = MSE R-quared = Adj R-quared = Total Root MSE = 38. MSR F = MSE = mea quare for error (redual) ad MSE MSR = mea quare for regreo (model) the degree of freedom for the regreo (SSR) {or for the model} ad 9 the degree of freedom for the redual (SSE). MSR ad MSE are each ch quare o the F rato a rato of ch quare. Prob > F = equvalet to p = or the p-value Th tet a global hypothe whe you have more tha oe depedet varable. If your regreo equato were y = β 0 + β x + β x + β 3 x 3 + β 4 x 4, the you would be tetg H 0 : β = β = β 3 = β 4 = 0 (ote β 0 wa delberately left out ce t ot a coeffcet for a x) H A : At leat oe of β (for =,,3,4) dfferet from zero. Becaue we have oly oe depedet varable (.e. y = β 0 + β x), the F tet tet oly H 0 : β = 0 veru H A : β 0 We wll ee later that the cae where we have oly oe predctor varable, the F tet ad the t-tet, o the le of the regreo output for the lope, gve the ame awer. Below we Page -6-

8 coder oly the oe predctor cae. If we decde that the ull hypothe for the lope true ( β 0 β β 0 y β = 0 regreo le become y = (becaue = 0). H 0 : β = 0 ), the our But the etmate for f (remember lat cla we gave the formula for the etmate of β 0 a y $ β x ). Th mea that we do t ga aythg wth the regreo le. Whch tur mea that the um of quare for regreo, zero. ( y$ y) approache But th mea that F = MSR MSE approache zero. So falg to reject the ull hypothe ad a mall F go together. What f our deco to reject the ull hypothe favor of the alteratve? Suppoe our deco to reject the ull hypothe for the lope favor of the alteratve hypothe ( y gve by. H A : β 0 ). Th mea that the regreo le ot the ame a the le Th tur mea that we thk the um of quare for the regreo (.e. greater tha zero. ( y$ y) ) ( x, y) y y = ( y y$ ) + ($ y y) y y $y y y y y$ For each pot t the cae that. But for each we have that tay fxed. So f gettg bgger (.e. the regreo le better tha the le ), mut be gettg maller (go back to pcture o page 4). So a MSR get bgger, MSE gettg maller. Whch mea that F gettg larger. So rejectg the ull hypothe ad large F go together. Page -7-

9 R-quared: Lat cla we looked at R ad howed ug the formula below that 0 # R #. ( y$ y) () R = ( ) R = ( y y) R called the coeffcet of determato. ( y y$ ) ( y y) Equato above how that R gve the percetage of the total varato explaed by the regreo le. Equato how that f the le ft well (.e. the redual are mall), the R cloe to oe.. regre bwt00 etrol Source SS df MS Number of ob = F(, 9) = 7.6 Model Prob > F = Redual R-quared = Adj R-quared = Total Root MSE = 38. The adjuted R ( R )( ) = So the adjuted R R adjuted for ample ze. Th defto for adjuted R for mple lear regreo. oly good Now go to the bottom half of the regreo table bwt00 Coef. Std. Err. t P> t [95% Cof. Iterval] etrol _co t-tet: bwt00 Coef. Std. Err. t P> t [95% Cof. Iterval] etrol = $β = etmate of the lope Th le the table tet H 0 : β = 0 veru H A : β 0. Page -8-

10 Whe you have oly oe depedet varable, th t-tet ad the F tet are the ame. [Note 4.4^. 7.6 They would be exactly equal f we d carred more decmal place] Later whe we have y = β 0 + β x + β x + β 3 x 3 + β 4 x 4, the t-tet ad F tet wll be dfferet. I th cae of multple depedet varable, we wll alo have a t-tet for each of β, β, β 3 ad β 4 (aga β 0 left out becaue t ot a coeffcet of a x). For the t-tet o the lope (β ): We reject the ull hypothe favor of the alteratve hypothe (.e. we beleve the lope dfferet from zero or we beleve there a lear tred to the data). t-tet: t = 4.4, df = 9, p < come from F(, 9). I how you below where thee umber come from. Where do the t-tet tattc come from: Let u look at the lope frt (.e. β ). Frt remember that the patter for tetg a mea: The ull hypothe looked lke: H 0 : μ = μ 0 Ad the t-tattc wa t = x μ0 / β = β ( ) 0 β β (th a more geeral vero of the H 0 : β = 0 0 ( ) Slope: To tet the hypothe H 0 :, where ome hypothezed value for ), the tet tattc ued t = $ ( 0) β β $ β yx where $ =. β x ad x the tadard devato of the x. Th tet tattc (.e. t) follow the t dtrbuto wth - degree of freedom, whe H 0 true. Where doe the - come from? Page -9-

11 I our earler adveture wth the t-tattc, the df = - becaue the ued the t-tattc ( x x) wa equal to. Earler th hadout we have ee that the degree of freedom for yx = MSE -. We ll ee below that yx freedom we are ug the ame patter a before. ued the t-tet for the lope. So order to get the degree of For the etrol problem we are tetg 0 H 0 : β = 0 o the earler otato β = 0 $ β. = We have that (the etmate of the lope). um(etrol) Varable Ob Mea Std. Dev. M Max etrol So x = ad = 3. From the regreo prtout we have MSE = 38. = MSE = = yx = = yx. So $ = β x Therefore, t = = Now we ll coder the y tercept: Page -0-

12 To tet the hypothe H 0 : β 0 0 = β ( ) 0, we ue the tet tattc t = $ ( 0) β0 β0 $ β 0 where $ = β 0 yx + x ( ) x ad x the varace of the x.. $ = + = β ( ) t = = Oe of the reao thee equato make ee that $β ad β0 $ are lear fucto of the y. A uch they are ormally dtrbuted ce y ormally dtrbuted. The two t-tet the regreo table fd out f the coeffcet of the predctor varable (.e the lope) zero ad f the tercept zero. We ca tet for value other tha zero (ee below). Frt you have to ru the regreo. Sce we already have the regreo table, I requet that t ru quetly. That, do t prt out the table.. quetly regre bwt00 etrol The tet below equvalet to the t-tet for the lope equal zero gve the regreo output.. tet etrol = 0 ( ) etrol = 0.0 F(, 9) = 7.6 Prob > F = Page --

13 . tet etrol = (th tet f the lope = ) ( ) etrol =.0 F(, 9) = 6.60 Prob > F = tet etrol = 70 ( ) etrol = 70 F(, 9) = 0.39 Prob > F = Below jut to how you that the formula gve u the ame awer, I have calculated the t tattc for the tet that the lope 70 (ug the equato o page 9). I the quared the t order to get the F the tet above.. d (( )/4.687)^ All the tet above are for lope. The oe below for the y-tercept.. tet _co = 0 ( ) _co = 0 F(, 9) = Prob > F = Whe we coder model wth more tha oe predctor varable we wll fd that fttg a model ot jut a oe hot deal. It a ere of coderato. The flow dagram below gve oe patter that frequetly ued. Page --

14 Aume traght-le model. Fd bet etmate of aumed model. Ae whether the bet etmate help to decrbe Y. Aume ew model. No I the aumed model approprate? Ye STOP Flow dagram of the forward method. Aumpto uderlyg traght-le model: ) Extece: For ay fxed value of the varable, a radom varable wth a certa probablty dtrbuto havg fte mea ad varace. The (populato) mea of th dtrbuto wll be deoted by μ yx ad the populato varace a σ yx. ) Idepedece: The y -value are tattcally depedet of oe aother. Th aumpto volated uder codto where you have multple meauremet o the ame dvdual. 3) Learty: The mea value of y, μ yx, a traght-le fucto of x. 4) Homocedatcty: The varace of y the ame for ay x (.e. for all of the x the ormal dtrbuto have the ame varace). Th mea that x y Page -3-

15 σ = σ x. yx Ade: Homo mea ame ad cedatc mea cattered. 5) Normal Dtrbuto: For ay fxed value of, ha a ormal dtrbuto. Th aumpto make t poble to evaluate the tattcal gfcace of the relatohp betwee ad a reflected by the ftted le. x y Example (that look more lke a le approprate tha t doe the etrol problem):. de Cota data from W:\WP5\Bometry\AAAABotat76Sprg009\Hadout\Chapter \Data\wtloplotwde.dta ob: 79 var: 7 6 Feb 009 8:06 ze:,054 (99.9% of memory free) torage dplay value varable ame type format label varable label d float %9.0g group byte %8.0g Treatmet Group age byte %8.0g Age year heght float %4.f Heght che heghtm float %9.0g Heght meter bm0 float %6.0f Pre-Treatmet BMI bm6 float %4.f Pot Treatmet BMI Sorted by: Ca we predct the pot-treatmet BMI (bm6) ug the pre-treatmet or baele BMI (bm0). The htogram below for bm6 whch the outcome varable. x y Frequecy Pot Treatmet BMI Page -4-

16 The outcome varable hould be ormally dtrbuted ad bm6 look omewhat kewed. Ade: Remember that t oly the outcome varable that ha to be ormal. However, whe we look at kewe below (0.9 a compared to 0 for a perfectly ymmetrc graph), t doe t look too bad. Notce that kurto look ort of ok. Later I ll gve you other way to tet ormalty.. um(bm6),det Pot Treatmet BMI Percetle Smallet % %.3. 0% 4.. Ob 79 5% Sum of Wgt % 8.9 Mea Larget Std. Dev % % Varace % Skewe % Kurto Whe we plot bm0 (the predctor) ad bm6 (the outcome), we get a catter of pot that look lke a traght-le mght be a reaoable model to try (ee below) Pot Treatmet BMI Pre-Treatmet BMI Page -5-

17 So rug the leat quare regreo aaly for the traght le, we get:. regre bm6 bm0 Source SS df MS Number of ob = F(, 77) = Model Prob > F = Redual R-quared = Adj R-quared = Total Root MSE = bm6 Coef. Std. Err. t P> t [95% Cof. Iterval] bm _co predct yhat (opto xb aumed; ftted value). predct e,red So our le $y = x where $y the etmate of the pot-treatmet BMI ad x the pre-treatmet BMI. Ug the regreo table above we get that the um of quare for regreo or for model (.e. SSR) 79 ( y$ y) = We ca get the ame awer for SSR ug brute force.. um(bm6) Varable Ob Mea Std. Dev. M Max bm ge model = (yhat )^. ge ummodel = um(model) Page -6-

18 . lt ummodel ummod~ Note that MSR (mea quare for regreo) = SSR/df = / = df for MSR the umber of predctor varable. 79 Sum of Square for Redual = SSE = ( y y$ ) = Same awer for SSE obtaed by brute force.. ge equared = e^. ge umeq = um(equared). lt umeq umeq Note that MSE (mea quare for redual or error) = SSE/df = /77 =.660. d / Page -7-

19 Th mea our error pretty mall (we could have gueed th from the graph - ee ext page). df for MSE = df for total - df for model = ( ) - = - Remember that MSE = yx. So the etmate of the commo varace for our ormal curve at each of the value of.660. Root MSE = yx = 660. =.63 x 79 The Total Sum of Square = ( y y) = ( y ) = Same awer for total um of quare by brute force.. ge t = (bm )^. ge umt = um(t). lt umt umt Note that MST (mea quare for total) = SST/df = 4.4/78 = d 4.4/ df for SST - = 78. Page -8-

20 Pot Treatmet BMI Ftted value 50.0 Pot Treatmet BMI Pre-Treatmet BMI Global F tet: F = MSR/MSE = d / Th a oe-taled upper taled tet. So we get the p-value ug Ftal. d Ftal(,77,89.37) 5.69e-43 = 5.69/(.78^43) Th mea the rato eetally zero. Sce the df for MSR = ad the df for MSE = 77, the df for the F tet are ad 77. R R = SSR/SST = /4.4 = d / Page -9-

21 Adjuted R ( R )( ) ( 095. )( 78) 77 Adjuted R = = = d - (( )*78)/ Slope b = ( x x)( y y) ( x x) = 0.97 Slope obtaed by brute force:. um(bm0) Varable Ob Mea Std. Dev. M Max bm The ext two commad gve u the top part of the equato for b.. ge croprod = (bm0-3.07)*(bm ). ge umcro = um(croprod) The ext two commad gve u the bottom part of the equato for b.. ge xmmeaq = (bm0-3.07)^. ge umxmmea = um(xmmeaq) Below I have lted the um for the top (frt colum) ad bottom ( d colum). Page -0-

22 . lt umcro umxmmea umcro umxm~ So b the rato of the two um.. d 69.98/ Itercept a = y bx = a = (0.974*3.07) = d (0.974*3.07) Stadard error for b (or $β ) yx $ = β x = yx = Root MSE =.63 x = ad = 79. d =.63/(5.470*qrt(78)) Page --

23 Stadard error for a (or 0 $β ) $ = β 0 yx + x ( ) x =.069. d.63*qrt((/79) + (3.0^/(78*5.470^))) t-tet for lope = 0 t = $ ( 0) β β $ β = d ( )/ df = for th t-tet (MSE dvded by - ) = 77 ad tet two-taled. d *ttal(77,8.80) 5.603e-43 e =.78 o we have 5.603/.78^43 = o p < % cofdece terval for lope ( $ β, $ ) t, ( α / ) $ β β + t, ( α / ) $ β = (0.905,.039). d vttal(77,0.05) d * d * Page --

REVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION

REVIEW OF SIMPLE LINEAR REGRESSION SIMPLE LINEAR REGRESSION I lear regreo, we coder the frequecy dtrbuto of oe varable (Y) at each of everal level of a ecod varable (X). Y kow a the depedet varable. The