The Simple Linear Regression Model: Theory

Transcription

1 Chapter 3 The mple Lear Regress Mdel: Ther 3. The mdel 3.. The data bservats respse varable eplaatr varable : : Plttg the data.. Fgure 3.: Dsplag the cable data csdered b Che at al (993). There are 79 bservats f the umber f hurs eeded t splce pars f wres fr a partcular tpe f telephe cable If the plt s t lear tr a smple trasfrmat t leart..e. lg, square rt, square. R. A. Rgb ad D. M. taspuls eptember 5

2 3.. Assumpts fr the mdel ) The assumpt abut the leart f the mdel Y fr,,..., ) The assumpt abut the errr dstrbut fr a) Full dstrbutal assumpt fr errr term. ~ N(, ) ad ad j fr j are depedet. stmat ths case f the parameters, ad s acheved b Mamum Lkelhd. b) Assumpt abut the frst ad secd mmets f the dstrbut fr. ( ) Var( ) Cv(, j ) stmat ths case ca be acheved b Least quares. ) The assumpt abut the -varable. The -varable s t a radm varable ad t s fed at the bserved values 3. Least squares estmat f parameters Let (, ) ( ) where the are bserved values fr the radm varable Y R. A. Rgb ad D. M. taspuls eptember 5

3 I rder t fd the least square estmatrs fr ad we eed t mmse (, ) (fr fed 's ad s) wth respect t the parameters ad. That s we fd ad ad we set them equal t zer. ( ) ( ) ( ) ( ) wth sluts ( )( ) ( ) The quattes are called the ftted values. The quattes are called the resduals. 3.3 Prpertes f the least square estmatrs Nte that bth ad are lear fucts f the s. Fr eample fr we have ( )( ) ( ) ( ) C R. A. Rgb ad D. M. taspuls eptember 5 3

4 where (Prve the abve statemet fr ). ad C ( ) pected values fr ad ) : s a ubased estmatr f. Prf Y a l l 3 () sce f c z c z () sce a (prve t) (3) sce (prve t) ) : s a ubased estmatr f. Prf: Y R. A. Rgb ad D. M. taspuls eptember 5 4

5 r a 3.3. The Varaces f ad ) Var Prf. Var s Var ( ) vary ) Prf. var var var var cv, But cv(, ) (see ercse 3.), s we have R. A. Rgb ad D. M. taspuls eptember 5 5

6 var( ) hece Var ( ) The Gauss-Markff therem The least-squares estmatrs ad have mmum varaces amg all the lear ubased estmatrs The Nrmalt assumpt f ad Nte that f Y s a lear fuct f rmall dstrbuted varables U.e. Y c U c U Y wll be Nrmall dstrbuted Y ~ N,..e. The L.. estmatrs ad are lear fucts f Y whch s Y ~ N, s ad wll be Nrmall dstrbuted as ~ N, ~ N, R. A. Rgb ad D. M. taspuls eptember 5 6

7 3.4 Hpthess testg 3.4. stmat f where s are the ftted values the resduals ad are the resdual degrees f freedm (df) t-test fr ad Nte that f, z ~ N the t ~ z ~ t ad z ad are depedet we have that ad that ~ N, ~ Als ad ( ) are depedet (t prve). R. A. Rgb ad D. M. taspuls eptember 5 7

8 t se t ~ s where se ( ) s the stadard errr f. We ca test hpthess fr usg the statstcs t. Fr eample t test calculate Nw f H : H : t se t t, a reject the ull hpthess H ad accept the alteratve H, therwse accept H. Nte that a s the sgfcat level f the test ad t the cstat parameter f the lear mdel. T test hpthess abut.e. H : H : R. A. Rgb ad D. M. taspuls eptember 5 8

9 use the test statstc t se C.I. fr ad A a % C. I. fr s gve b t a, se ad fr s gve b t a, se 3.5 Predct ad Cfdece Itervals 3.5. Cfdece Itervals fr a b Nte that the epected value fr the value f the -varable whe the eplaatr varable s at s : The ftted value at the pt s defed as wth epected values as ad R. A. Rgb ad D. M. taspuls eptember 5 9

10 R. A. Rgb ad D. M. taspuls eptember 5 3 the ftted value s ubased fr. The varace fr s, cv var var var s a estmate fr the varace s gve b. vâr s where s /. ce s a lear cmbat f Nrmall dstrbuted varables, t s Nrmall dstrbuted;.e., ~ N, ~ N z r ~ t t s a C.I fr s gve b a se t, where

11 se 3.5 Predct Iterval fr /, a future bservat fr. Let / dete a future bservat f the -varable at the -varable value. The ce / f a future bservat /. s a ubased estmatr fr t ca be used t predct the mea I geeral rder t evaluate hw gd ur predctr s fr predctg a further bservat we have t kw the mea square errr fr predct r P. Deft: P ŷ Therem: Let ŷ be a estmate f ad let * be a ew bservat such that ( * ). The P Var( ) M( ) where M( ) ( ). Prf: P * s depedet f ŷ, as ŷ * s a ew bservat. Hece ( ) ( ) as ( * ) Hece * P( ) = Var M = Var Var bas R. A. Rgb ad D. M. taspuls eptember 5 3

12 I the smple lear regress eample we have Var Var sce ŷ s ubased A a % predct terval fr / s gve b t a, s 3.6 Mamum lkelhd estmat f the parameters, ad the smple lear regress. The lkelhd fuct s the prbablt f bservg the sample seeg as a fuct f the parameter rather tha a fuct f the radm varables. Fr depedet radm varables,... the lkelhd wll be ; f ;... f ; L f I the smple regress mdel we have where Y.e. ~ (, ) d N Y d ~ N b, R. A. Rgb ad D. M. taspuls eptember 5 3

13 ad (,, ). var Y The lkelhd fr e bservat s L b, ; ep b, fr depedet bservats the lkelhd wll be Nte that L, b, /... ep b square estmat apprach. = ep. b s the fuct that we mmsed the least I rder t fd the ML's fr, ad we have t mamse L,, respect t the parameters r equvaletl mamse lgl, b,,, Nw,, lg s we dfferetate wth respect t, ad wth R. A. Rgb ad D. M. taspuls eptember 5 33

14 4 slvg fr,, we have Nte: ) ad are als the least-square estmatrs. Ths s because the mamsat f the lg-lkelhd (fr fed ) s the equvalet f the mmsat f the least-square quatt. ) We geerall prefer t use a ubased estmatr f gve b s D df devace the resdual degrees f freedm ercse 3.: mple lear regress ther a) Csder the smple lear regress mdel f the frm Y a b fr,..., where Y s the respse varable, R. A. Rgb ad D. M. taspuls eptember 5 34

15 s the depedet varable, a ad b are parameters t be estmated., fr,..., are depedet Nrmall dstrbuted varables wth mea ad varace. ) Fd the lkelhd fuct fr a sgle bservat ad hece shw that the lg-lkelhd fuct fr all bservats frm the abve mdel s l( a, b, ) lg( ) ( a b ) ) Gve the Nrmal equats used t fd the Mamum lkelhd estmatrs fr the parameters a, b ad, ad state the resultg mamum lkelhd estmatrs f a, b ad. b) e ) tate the dstrbut f b ad where e a b,.e. the resdual fr the th bservat ad the umeratr the epress s the Resdual um f quares (R). Nte that the varace f b s where ( ). ) We kw that b ad e are depedet. We als kw that f z ~ N (, ) ad w ~, depedetl, z the t ( w / ) ~. Use ths t cstruct a (-)% cfdece terval fr b. R. A. Rgb ad D. M. taspuls eptember 5 35

16 R. A. Rgb ad D. M. taspuls eptember 5 36 ercse 3.: mple lear regress ther Fr a smple lear regress mdel, prve that, cv Nte: a utled methd s as fllws: (Wh?), cv Nw... / / as j (Wh?) Hece deduce, cv.

17 ercse 3.3: mple lear regress ther Fr the regress f b s gve b a, ~ N(, ) b shw that the least squares estmate b. A test f b= ca be based up T hw that, uder H : b, we have (T) =.. Als shw that V T Deduce that ~ N(, ) Ad hece that s ~ t R. A. Rgb ad D. M. taspuls eptember 5 37

18 Practcal 3: mple Lear Regress The data set the fle HARD (K):\CTM\OM\MA\RGRION\FOOT GTATION TIM.AV cmprses measuremets f fetal ft legth mm (Y) ad gestatal age weeks (X) fr 45 fetuses.. Prduce a scatter plt f Y agast X usg the prcedure > Graphs > catter > mple catter > Defe Y As ft X As gest > OK. Cmmet ths plt.. Ft a smple lear regress le usg > Aalse > Regress > Lear Depedet: ft Idepedet(s): gest t declare ur ad varable ad ft the mdel. Yu ca use the PLOT pt t get the resdual plts. > Plts Y: ZRID X: ZPRD Hstgram Nrmal prbablt plt > Ctue > OK ) tate the mdel ftted ad ts parameter estmates. Iterpret these estmates. ) Test whether there s a lear relatshp betwee ft legth ad gestatal age. ) tate the assumpts ecessar fr ur mdel t be vald. v) D the resdual plts shw that a f the assumpts des t hld? R. A. Rgb ad D. M. taspuls eptember 5 38