Simple Linear Regression: 1. Finding the equation of the line of best fit

Size: px

Start display at page:

Download "Simple Linear Regression: 1. Finding the equation of the line of best fit"

Cornelia Singleton
6 years ago
Views:

1 Cocerao Wegh kg mle Lear Regresso:. Fdg he equao of he le of es f Ojecves: To fd he equao of he leas squares regresso le of o. Backgroud ad geeral rcle The am of regresso s o fd he lear relaosh ewee wo varales. Ths s ur raslaed o a mahemacal rolem of fdg he equao of he le ha s closes o all os oserved. Cosder he scaer lo o he rgh. Oe ossle le of es f has ee draw o he dagram. ome of he os le aove he le ad some le elow Hegh m The vercal dsace each o s aove or elow he le has ee added o he dagram. These dsaces are called devaos or errors he are smolsed as d... d d. Whe drawg a regresso le he am s o make he le f he os as closel as ossle. We do hs makg he oal of he squares of he devaos as small as ossle.e. we mmse d. If a le of es f s foud usg hs rcle s called he leas-squares regresso le. amle : A ae s gve a dr feed coag a arcular chemcal ad s cocerao hs lood s measured suale us a oe hour ervals. The docors eleve ha a lear relaosh wll es ewee he varales. Tme hours Cocerao We ca lo hese daa o a scaer grah me would e loed o he horzoal as as s he deede varale. Tme s here referred o as a corolled varale sce he eermeer fed he value of hs varale advace measuremes were ake ever hour. Cocerao s he deede varale as he cocerao he lood s lkel o var accordg o me. The docor ma wsh o esmae he cocerao of he chemcal he lood afer 3.5 hours. he could do hs fdg he equao of he le of es f Tme hours There s a formula whch gves he equao of he le of es f.

2 ** The sascal equao of he smle lear regresso le whe ol he resose varale Y s radom s: Y or erms of each o: Y Here s called he erce he regresso sloe s he radom error wh mea s he regressor deede varale ad Y he resose varale deede varale. ** The leas squares regresso le s oaed fdg he values of ad values deoed he soluos as ˆ ˆ & ha wll mmze he sum of he squared vercal dsaces from all os o he le: d ˆ The soluos are foud solvg he equaos: ad ** The equao of he fed leas squares regresso le s Yˆ ˆ ˆ or erms of each o: Yˆ ˆ ˆ For smlc of oaos ma ooks deoe he fed regresso equao as: ˆ * ou ca see ha for some eamles we wll use hs smler oao. Y where ˆ. ˆ ad Noaos: ˆ ; ; ad are he mea values of ad resecvel. Noe : Please oce ha fdg he leas squares regresso le we do o eed o assume a dsruo for he radom errors. However for sascal ferece o he model arameers ad s assumed our class ha he errors have he followg hree roeres: Normall dsrued errors Homoscedasc cosa error varace var for Y a all levels of Ideede errors usuall checked whe daa colleced over me or sace.. d. ***The aove hree roeres ca e summarzed as: ~ N Noe : Please oce ha he leas squares regresso s ol suale whe he radom errors es he deede varale Y ol. If he regresso s also radom s he referred o as he rrors arale I regresso. Oe ca fd a good summar of he I regresso seco. of he ook: ascal Iferece d edo George Casella ad Roger Berger. We ca work ou he equao for our eamle as follows:... 6 so so so 3 7 These could all e foud o a calculaor f ou eer he daa o a calculaor.

3 o ˆ. 843 ad ˆ ˆ o he equao of he regresso le s ŷ = To work ou he cocerao afer 3.5 hours: ŷ = = sf If ou wa o fd how log would e efore he cocerao reaches 8 us we susue ŷ = 8 o he regresso equao: 8 = olvg hs we ge: = 3.6 hours Noe: I would o e sesle o redc he cocerao afer 8 hours from hs equao we do kow wheher he relaosh wll coue o e lear. The rocess of rg o redc a value from ousde he rage of our daa s called eraolao. amle : The heghs ad weghs of a samle of sudes are: Hegh m h Wegh kg w [ h 7.7 h 8.75 w 737 w 557 hw 96. ] a Calculae he regresso le of w o h. Use he regresso le o esmae he wegh of someoe whose hegh s.6m. Noe: Boh hegh ad wegh are referred o as radom varales her values could o have ee redced efore he daa were colleced. If he samlg were reeaed aga dffere values would e oaed for he heghs ad weghs. oluo: a We eg fdg he mea of each varale: h 7.7 h w 737 w 67 Ne we fd he sums of squares: 3

4 hh h h w 737 ww w h w hw hw The equao of he leas squares regresso le s: wˆ ˆ ˆ h where hw 8.86 ˆ 55.5 hh.597 ad ˆ w ˆ h o he equao of he regresso le of w o h s: ŵ = h To fd he wegh for someoe ha s.6m hgh: ŵ = = 66.4 kg mle Lear Regresso:. Measures of arao Ojecves: measures of varao he goodess-of-f measure ad he correlao coeffce ums of quares.4 Toal sum of squares = Regresso sum of squares + rror sum of squares T R Toal varao = laed varao + Uelaed varao Toal sum of squares Toal arao: T Y Regresso sum of squares laed arao he Regresso: rror sum of squares Uelaed arao: Y Y Y ˆ R Yˆ Y 4

5 Coeffces of Deermao ad Correlao Coeffce of Deermao s a measure of he regresso goodess-of-f I also rereses he rooro of varao Y elaed he regresso o R R ; R T Pearso Produc-Mome Correlao Coeffce -- measure of he dreco ad sregh of he lear assocao ewee Y ad The samle correlao s deoed r ad s closel relaed o he coeffce of deermao as follows: r sg ˆ R ; r The samle correlao s deed defed he followg formula: r [ ][ ] Y YY ][ The corresodg oulao correlao ewee Y ad s deoed ρ ad defed : [ ] CO ar Y ary Y Y ar ary Therefore oe ca see ha he oulao correlao defo oh ad Y are assumed o e radom. Whe he jo dsruo of ad Y s varae ormal oe ca erform he followg -es o es wheher he oulao correlao s zero: Hoheses H : ρ = H A : ρ o correlao correlao ess Tes sasc r H ~ r Noe: Oe ca show ha hs -es s deed he same -es esg he regresso sloe = show he followg seco. Noe: The samle correlao s o a uased esmaor of he oulao correlao. You ca sud hs ad oher roeres from he wk se: h://e.wkeda.org/wk/pearso_roduc-mome_correlao_coeffce amle 3: The followg eamle aulaes he relaos ewee ruk dameer ad ree hegh. 5

Tree Hegh Truk Dameer 35 8 8 5 64 49 9 44 4 8 7 7 89 79 49 33 6 98 89 36 6 3 78 36 69 7 47 44 49 45

6 Tree Hegh Truk Dameer =3 =73 =34 =4 =73 caer lo: r [ ][ [ ][84 3 ].886 ] 6

7 r =.886 relavel srog osve lear assocao ewee ad gfcace Tes for Correlao Is here evdece of a lear relaosh ewee ree hegh ad ruk dameer a he.5 level of sgfcace? H : ρ = No correlao H : ρ correlao ess r A he sgfcace level α =.5 we rejec he ull hohess ecause ad coclude ha here s a lear relaosh ewee ree hegh ad ruk dameer. A for Correlao Daa ree; Iu hegh ruk; Daales; ; Ru; Proc Corr daa = ree; ar hegh ruk; Ru; r Noe: ee he followg wese for more eamles ad erreaos of he ouu lus how o draw he scaer lo roc Glo A: h:// adard rror of he smae Resdual adard Devao The mea of he radom error s equal o zero. A esmaor of he sadard devao of he error s gve ˆ s 7

8 mle Lear Regresso: 3. Ifereces Cocerg he loe Ojecves: measures of varao he goodess-of-f measure ad he correlao coeffce -es Tes used o deerme wheher he oulao ased sloe arameer s equal o a re-deermed value ofe u o ecessarl. Tess ca e oe-sded re-deermed dreco or wo-sded eher dreco. -sded -es: H : = o lear relaosh H : lear relaosh does es Tes sasc: s s s s Where s A he sgfcace level α we rejec he ull hohess f / Noe: oe ca also coduc he oe-sded ess f ecessar. F-es ased o k deede varales A es ased drecl o sum of squares ha ess he secfc hoheses of wheher he sloe arameer s -sded. The ook descres he geeral case of k redcor varales for smle lear regresso k =. H : H : A T: F RR F os MR M F : os k k R / k / k Aalss of arace ased o k Predcor arales for smle lear regresso k = ource df um of quares Mea quare F Regresso k R MR=R/k F os =MR/M rror -k- M=/-k- --- Toal - T % Cofdece Ierval for he sloe arameer : /s 8

9 If ere erval s osve coclude > Posve assocao If erval coas coclude do o rejec No assocao If ere erval s egave coclude < Negave assocao amle 4: A real esae age wshes o eame he relaosh ewee he sellg rce of a home ad s sze measured square fee. A radom samle of houses s seleced Deede varale = house rce $s Ideede varale = square fee House Prce $s quare Fee oluo: Regresso aalss ouu: Regresso ascs Mulle R.76 R quare.588 Adjused R quare.584 adard rror Oservaos ANOA df M F Regresso Resdual Toal gfcace F Coeffce s adard rror a Ierce quare Fee P- value Lower 95% Uer 95%

10 smaed house rce square fee measures he esmaed chage he average value of Y as a resul of a oe-u chage Here =.977 ells us ha he average value of a house creases.977$ = $9.77 o average for each addoal oe square foo of sze R R T Ths meas ha 58.8% of he varao house rces s elaed varao square fee. s The sadard error esmaed sadard devao of he radom error s also gve he ouu aove. es for a oulao sloe Is here a lear relaosh ewee ad? Null ad alerave hoheses H : = o lear relaosh H : lear relaosh does es Tes sasc: 3.39 s A he sgfcace level α =.5 we rejec he ull hohess ecause ad coclude ha here s suffce evdece ha square fooage affecs 8. 5 house rce. Cofdece Ierval smae of he loe: /s The 95% cofdece erval for he sloe s ce he us of he house rce varale s $s we are 95% cofde ha he average mac o sales rce s ewee $33.7 ad $85.8 er square foo of house sze Ths 95% cofdece erval does o clude. Cocluso: There s a sgfca relaosh ewee house rce ad square fee a he.5 level of sgfcace Predc he rce for a house wh square fee: house rce sq.f

11 The redced rce for a house wh square fee s 37.85$s = $3785 amle 5 A: Wha s he relaosh ewee Moher s srol level & Brhwegh usg he followg daa? srol Brhwegh mg/4h g/ Daa BW; /*Readg daa A*/ u esrol rhw@@; daales; ; ru; PROC RG daa=bw; /*Fg lear regresso models*/ model rhw=esrol; ru;

12 Face Alcao: Marke Model Oe of he mos mora alcaos of lear regresso s he marke model. I s assumed ha rae of reur o a sock R s learl relaed o he rae of reur o he overall marke. R = + R m + R: Rae of reur o a arcular sock R m : Rae of reur o some major sock de : The ea coeffce measures how sesve he sock s rae of reur s o chages he level of he overall marke. amle: Here we esmae he marke model for Norel a sock raded he Toroo ock chage. Daa cossed of mohl erceage reur for Norel ad mohl erceage reur for all he socks. UMMARY OUTPUT Regresso ascs Mulle R.5679 R quare Adjused R.3855 adard.633 Oservao 6 ANOA df M F gfcace F Regresso Resdual Toal Coeffcesadard rro a P-value Ierce T T esmaed regresso sloe: Ths s a measure of he sock s marke relaed rsk. I hs samle for each % crease he T reur he average crease Norel s reur s.8877%. R quare R Ths s a measure of he oal rsk emedded he Norel sock ha s marke-relaed. ecfcall 3.37% of he varao Norel s reur are elaed he varao he T s reurs.

13 3 Lear Regresso Mar Form Daa:. The mulle lear regresso model scalar form s. ~ N The aove lear regresso ca also e rereseed he vecor/mar form. Le. The =. smao: Leas square mehod: The leas square mehod s o fd he esmae of mmzg he sum of square of resdual sce Y. adg elds Noe: For wo marces A ad B A B AB ad A A mlar o he rocedure fdg he mmum of a fuco calculus he leas square esmae ca e foud solvg he equao ased o he frs dervave of

14 4 The fed regresso equao s ˆ. The fed values vecor: ˆ ˆ ˆ ˆ The resduals vecor: e e e ˆ e Noe: a a a where ad a a a a. A A j j j a where A s a smmerc mar. Noe: ce s a smmerc mar. Also s a smmerc mar. Noe: Y s called he ormal equao. Noe: e.

15 5 Therefore f here s erce he he frs colum of s. The e e e e e e Noe: for he lear regresso model whou he erce e mgh o e equal o. Proeres of he leas square esmae: Two useful resuls: Le e a radom vecor A s a mar ad C s a vecor. Le ad cov cov cov cov cov cov cov cov cov cov cov cov cov cov cov ar ar ar. The a C C A A. C A A A Noe: I

16 6 The roeres of leas square esmae:.. The varace covarace mar of he leas square esmae s cov cov cov cov cov cov ar ar ar [Dervao:] sce. Also I σ σ σ sce I amle 6: Heller Coma maufacures law mowers ad relaed law equme. The maagers eleve he qua of law mowers sold deeds o he rce of he mower ad he rce of a comeor s mower. We have he followg daa: Comeor s Prce Heller s Prce Qua sold The regresso model for he aove daa s. The daa mar form are

17 Y. The leas square esmae s The fed regresso equao s ˆ. The fed equao mles a crease he comeor s rce of u s assocaed wh a crease of.44 u eeced qua sold ad a crease s ow rce of u s assocaed wh a decrease of.69 u eeced qua sold. uose ow we wa o redc he qua sold a c where Heller rces mower a $6 ad he comeor rces s mower a $7. The qua sold redced s amle 7: We show how o use he mar aroach o oa he leas square esmae ad s eeced value ad he varace. Le. The Thus ad = sce

18 8 a c d c ad d c a Therefore Y Y Y Y Y Also ad cov cov ar ar Ackowledgeme: I comlg hs lecure oes we have revsed some maerals from he followg weses: Quz 3. Due Tuesda a he egg of he lecure *Yes I wll defel collec* Please rove ha: Toal sum of squares = Regresso sum of squares + rror sum of squares; ha s R T Please derve he mehod of mome esmaors of he regresso model arameers

Chapter 8. Simple Linear Regression

Chapter 8. Simple Linear Regression Chaper 8. Smple Lear Regresso Regresso aalyss: regresso aalyss s a sascal mehodology o esmae he relaoshp of a respose varable o a se of predcor varable. whe here s jus oe predcor varable, we wll use smple