Francis Galton ( ) The Inventor of Modern Regression Analysis

Size: px

Start display at page:

Download "Francis Galton ( ) The Inventor of Modern Regression Analysis"

Madeleine Sutton
6 years ago
Views:

1 Decrptve Stattc The Cure S Far: Prbablty Thery Prbablty Dtrbut Samplg Dtrbut Smple Radm Samplg Symbl f Caual Aaly: Crcle repreetg theretcal (r latet) varable that may be caue r effect (r bth) ur thery. Square repreetg emprcal meaure f latet varable. Thee are varable actually meaured the data. Sgle-headed arrw repreetg caual effect frm e varable t ather. Stattcal Iferece Duble-Headed Arrw repreetg crrelat r cvarace betwee varable but t cauat betwee them. Etmat Hypthe Tetg Regre & Crrelat Multple Regre Galt' Orgal Regre Father' Heght a a Predctr f S' Heght father. ummarze father td fathertd Ftted value Varable Ob Mea Std. Dev. M Max father td e fathertd e Frac Galt (8-9) The Ivetr f Mder Regre Aaly Galt' Orgal Regre Father' Heght a a Predctr f S' Heght Stadardzed value f (father). regre father, beta Father' Heght Std Value f Ftted value Surce SS df MS Number f b = F(, 76) = 36.3 Mdel Prb > F =. Redual R-quared = Adj R-quared =.56 Ttal Rt MSE =.4366 Cef. Std. Err. t P> t Beta father _c S' Heght Weght MPG graph twway catter weght, ttle("scatter f Car Weght v. MPG") Scatter f Car Weght v. MPG weght

2 3 4 5 Scatter f Car Weght v. MPG weght Aume that we are dealg wth a gle relathp ad that t cta ly tw varable: f MPG f WGT Che the fuctal frm f the relathp betwee ad : MPG WGT Sme ther pblte are: = e whch mple: lge lge whch mple: lg lg lg e e e Thee tw frm, whch are lear ca be trafrmed by takg atural lg f bth de. The, the reultg lgged equat are lear the lg. Here' ather frm that lear ad /: (Lear Equat) Rea fr ertg a tchatc errr term: The errr term cta all the frmat that, f we kew t, wuld allw u t cmpletely expla varat. There are radm errr f bervat r meauremet Over ad abve the ttal effect f all relevat factr, there a bac ad upredctable elemet f radme huma repe whch ca be adequately characterzed ly by the clu f a radm errr term. Aumpt abut the tchatc mdel:,,, E E fr all fr j;, j,,, E j fr j;, j,,, The Cmplete Mdel Wgt () MPG ()

3 ,,, E fr all fr j;, j,,, E j fr j;, j,,, Leat Square Etmatr The Data: Arthmetc Mea Dete the etmated le thrugh the data a: e ad, = etmate f tw ukw parameter etmated value f fr ay e = the dfferece betwee the actual ad etmated value f. r, e Cequetly, ur gal t mmze: e f, Dervat f Leat Square Etmatr fr Mmze: a ad e r, ubttutg, T mmze the um f quared devat, a eceary cdt that the partal dervatve f the um wth repect t â ad huld bth be zer. We thu have: e e Smplfyg thee tw equat gve the tadard frm f the rmal equat fr a traght le: Nte that dvdg th equat by gve whch mea that leat-quare etmate are uch that the etmated le pae thrugh the pt f mea (, ).

4 Nw, we ca ubtract the mea f frm bth de f the rgal equat: Let' let lwer cae letter dete devat frm the mea, that x y y S we ca wrte the leat quare le equat a: ŷ x Ad the redual e may be dcated by: e y y y x Nw, we ca rewrte ur um f quared redual a: e y x Mmzg th expre wth repect t Ad, we ca fd by rememberg that the regre le pae thrugh the pt f mea, amely, gve: x y x VAR x x y x y x x COV, VAR COV, x y,,, E fr all fr j;, j,,, E j fr j;, j,,, x E E e The leat quare etmatr f the lpe ubaed x x E E e x The leat quare etmatr f the tercept ubaed Turch ad that f the aumpt abut the errr term hld, the the leat quare etmatr f the lpe ad tercept term are ubaed etmatr. Trut hm th. Mrever, thee etmatr are bet lear ubaed etmatr. That they are effcet (they have the mallet varace f ay lear etmatr). The Cmplete Mdel Wgt () MPG () The Mdel: x MPG wgt That, mleage determed by the weght f the car ad me radm, but ucrrelated factr whe effect fr each bervat are ctaed the errr term,

5 . regre weght Surce SS df MS Number f b = F(, 5) = 334. Mdel Prb > F =. Redual R-quared = Adj R-quared =.6853 Ttal Rt MSE = Cef. Std. Err. t P> t [95% Cf. Iterval] weght _c MPG WGT MPG WGT d MPG.48 dwgt That, f the weght f a autmble re by e pud, mle per gall wll fall by /. Or, f the weght re by, pud, MPG wll fall by.4. predct hat graph hat weght, cect(.) ymbl() Ftted value weght MPG WGT The regre ceffcet ad are radm varable, each wth ther w amplg dtrbut. Wgt () Here' a amplg dtrbut fr The Cmplete Mdel MPG () The ly urce f radm behavr, apart frm Wgt the errr term. S, t' the dtrbut ad varace f the errr term that determe the dtrbut f ad t tadard errr. where TSSx TSS ad RSS where RSS K ad the ample ze ad K the umber f etmated parameter ( th cae: ) ad. Nw, f the errr term,, rmally, detcally ad depedetly dtrbuted, we ca frm the t-tattc: t ad tet hypthee abut the true lpe ceffcet.

6 t H H a : : TSS Cfdece Iterval: t df K H t : H :. regre weght a Surce SS df MS Number f b = F(, 5) = 334. Mdel Prb > F =. Redual R-quared = Adj R-quared =.6853 Ttal Rt MSE = Cef. Std. Err. t P> t [95% Cf. Iterval] weght _c t df K Cfdece Iterval t regre wgt, level(9) df K The "perfect" regre: The le clude all data pt wth errr. % f the varat 5 f the death rate explaed by detary fat. Death Rate (per,) Detary fat (gram/day) R. ad R. Ttal Sum f Square (TSS) the um f quared devat f the depedet varable arud t mea ad a meaure f the ttal varablty f the varable: TSS Explaed (r Mdel) Sum f Square (ESS) the um f quared devat f predcted value f arud t mea: ESS Redual Sum f Square (RSS) the um f quared devat f the redual arud ther mea value f zer: RSS Remember, t' RSS that leat quare regre eek t mmze.

7 R explaed varace/ttal varace. regre weght ESS TSS Surce SS df MS Number f b = F(, 5) = 334. Mdel Prb > F =. Redual R-quared = Adj R-quared =.6853 Ttal Rt MSE = Cef. Std. Err. t P> t [95% Cf. Iterval] weght _c r Varace frmula Cvarace frmula Crrelat Ceffcet r The crrelat ceffcet, r, a tadardzed meaure f a bvarate lear relathp betwee tw varable,, ad. Negatve: hgh value f ted t ccur wth lw value f, ad lw wth hgh. Ptve: hgh value f ted t ccur wth hgh value f, ad lw wth lw.. crrelate weght (b=54) weght weght r Clam: the bvarate crrelat ceffcet mply the regre lpe ceffcet whe e tadardzed varable regreed ather tadardzed varable. ege t=td() Frt, tadardze & weght: ege tweght=td(weght) Secdly, check ther mea ad t. devat:. ummarze t tweght Varable Ob Mea Std. Dev. M Max t e tweght e Thrd: Regre t tweght: regre t tweght. regre t tweght Surce SS df MS Number f b = F(, 5) = 334. Mdel Prb > F =. Redual R-quared = Adj R-quared =.6853 Ttal Rt MSE =.5697 t Cef. Std. Err. t P> t [95% Cf. Iterval] tweght _c 3.96e

8 Furth: Create a predcted ver f t:. predct that graph hat weght, cect(.) ymbl(o) xlabel( 5 t 45) Ffth: Graph actual ad predcted value f t: graph t that tweght, cect(.) ymbl() Ftted value Stadardzed value f () Ftted value.437 Ftted value Stadardzed value f () Slpe = lpe = Stadardzed value f ( weght) weght weght * * r I bvarate regre ly, the tadardzed regre ceffcet equal the crrelat ceffcet. regre weght,beta. regre weght,beta Surce SS df MS Number f b = F(, 5) = 334. Mdel Prb > F =. Redual R-quared = Adj R-quared =.6853 Ttal Rt MSE = Cef. Std. Err. t P> t Beta weght _c * r R All f thee equalte hld ly fr bvarate regre & crrelat Thg wll be a lttle mre cmplcated multple regre ESS / K F RSS / K / K / K F ESS / K ESS / df RSS / K RSS / df dety Where K = umber f etmated parameter (, lpe ad tercept) = ample ze H H a F(,5) = 334. : F,5 : F,5

9 ESS/df RSS/df F(,5) graph hat weght, cect(.) ymbl(o) xlabel( 5 t 45). regre weght Surce SS df MS Number f b = F(, 5) = 334. Mdel Prb > F =. Redual R-quared = Adj R-quared =.6853 Ttal Rt MSE = Cef. Std. Err. t P> t [95% Cf. Iterval] weght _c Prb > F Ftted value E E E E weght,5 E, 5, , fr t, t 3,,, S The cfdece terval fr the mea etmate f t, t S S,, : wdth f the cfdece terval deped up dtace frm the mea. Dervat f: Var Var E E E E Var Var Cv, Ntg that: Var E S Var E S, Cv E S S ubttutg t the varace frmula gve: Var S S S ad tg that : S ad ubttutg, we get: Var S S S

10 predct hat (cmpute predcted value f ) graph hat hgh lw weght, ymbl() cect(. [-] [-]) xlabel ylabel (Stata 7) predct SEhat, tdp (cmpute ) (cmpute t-value dplay vttal(df,.5/) where df = -K ) dplay vttal(5,.5/)-> >.98 geerate lw= hat -.98* SEhat geerate hgh= hat +.98* SEhat Ftted value hgh lw Cfdece Iterval get wder farther frm ceter f data weght graph twway lftc weght catter weght, t("regre f Mleage Car Weght") Regre f Mleage Car Weght Cfdece tervel get wder farther frm ceter f data Weght (pud) 95% CI Ftted value Mle per Gall E Radm part E Sytematc (-radm) part E E E E the frecat errr, where E E E E whch mple that a ubaed etmatr f

11 , Var Var Var Cv Var Var Var becaue Cv, Var Var Var S Var Var Var S Var Var( ) Var Var S S Var The varace f the frecat errr,, t, t S S t, t S S,, S predct SEhat, tdf Th terval alway wder becaue f thee. geerate lw= hat -.98* SEhat geerate hgh= hat +.98* SEhat graph hat hgh lw hgh lw weght, ymbl() cect(. [-] [-] [.] [.]) xlabel ylabel (Stata 7) graph twway lftc weght, cplt(rle) lftc weght, tdf cplt(rle) catter weght, t("regre f Mleage Car Weght") Ftted value hgh lw Regre f Mleage Car Weght 6 4 Predct Iterval Cfdece Iterval weght Predct Iterval Cfdece Iterval Weght (pud) 95% CI Ftted value Ftted value Mle per Gall

12 graph hat hgh lw hgh lw weght, ymbl() cect(. [-] [-] [.] [.]) xlabel( 5 t 45) ylabel Ftted value 46 exphat Ftted value hgh lw weght MPG WGT (Lear Equat) weght = e whch mple: lg e lge geerate l = l() weght. regre l weght Surce SS df MS Number f b = F(, 5) = Mdel Prb > F =. Redual R-quared = Adj R-quared =.76 Ttal Rt MSE =.469 l Cef. Std. Err. t P> t [95% Cf. Iterval] weght _c whch mple: lg lg lg expdbllg 46.6 e e e weght. regre l lweght Surce SS df MS Number f b = F(, 5) = Mdel Prb > F =. Redual R-quared = Adj R-quared =.7 Ttal Rt MSE =.48 l Cef. Std. Err. t P> t [95% Cf. Iterval] lweght _c predct dbllg (pt xb aumed; ftted value). geerate expdbllg=exp( dbllg). graph expdbllg weght, cect(.) ymbl()

Simple Linear Regression Analysis

Simple Linear Regression Analysis LINEAR REGREION ANALYSIS MODULE II Lecture - 5 Smple Lear Regreo Aaly Dr Shalabh Departmet of Mathematc Stattc Ida Ittute of Techology Kapur Jot cofdece rego for A jot cofdece rego for ca alo be foud Such