STA261H1.doc. i 1 X n be a random sample. The sample mean is defined by i= 1 X 1 + ( ) X has a N ( σ ) 1 n. N distribution. Then n. distribution.

Transcription

1 TA6Hdoc tttcl Iferece RADOM AMPLE Defto: Rdom mple clled rdom mple from dtruto wth pdf f ( (or pf ( depedet d hve detcl dtruto wth pdf f (or pf P (depedet-detcll-dtruted P f re It ofte deoted d ote Let f e mple from dtruto wth pdf f ( The ot pdf of ( ( f ( f f f AMPLE MEA Defto: mple Me d mple Vrce Let e rdom mple The mple me defed mple vrce defed ( d the Theorem If re d ech wth ( µ wth me µ ( µ More geerll f dtruto the h orml dtruto µ d vrce ( re depedet d ech orml dtruto me µ µ d vrce h ( µ the h UEFUL DITRIUTIO Theorem uppoe depedet d rdom mple from ( µ ( h χ dtruto dtruto The d ( re Pge of 3

2 TA6Hdoc The t d F Dtruto Two mportt dtruto ueful tttcl ferece re: W If W ~ ( d V ~ χ r re depedet the h t-dtruto V r U r U ~ χ r d V ~ χ r re depedet the h F-dtruto V r THE CETRAL LIMIT THEOREM Let rdom mple wth fte me µ d vrce Let The E( E( µ ( vr( vr( > TATITICAL MODEL Coder rdom mple f θ pdf (or from dtruto wth pdf f θ ( The fml { θ Ω} f (where P θ pf θ uow prmeter Ω prmeter pce tttcl model We ow tht the dtruto uder vetgto the fml ut do t ow whch oe ed o the mple vlue we fd etmte for θ; Oce we fd θ we ow the dtruto θ LIKELIHOOD FUCTIO Defto: The Lelhood Fucto Let e rdom mple from dtruto wth pdf f θ ( (or pf P θ L : Ω R gve L( θ cfθ ( cfθ ( fθ ( ( θ cp ( cp ( P ( defed L θ θ θ The lelhood where c > (or Defto: The Mmum Lelhood Etmte (MLE The fucto ˆ θ : Ω clled the mmum lelhood etmtor ( θˆ clled the mmum lelhood etmte of θ f for ech Ω θ ˆ( ( L( θ L θ The Algorthm Th ugget tht order to ot the MLE of θ we mmum the lelhood fucto ce vero of the lelhood vero wth c gve the me mmum vlue we ue th vero I mot ce th doe dfferetto θ θ Wrte the lelhood fucto L( f Pge of 3

3 TA6Hdoc l θ Wrte the log lelhood fucto defed l( θ l( L( θ f ( 3 Wrte the core fucto ( θ ( θ l θ θ d olve for θ 4 Wrte the core equto 5 Chec tht the oluto the glol mmum If t the t the MLE of θ Theorem ˆ θ the MLE Ω d Ω - φ : Ω the the MLE the ew prmeterzto If ( ˆ θ ( ( ˆ θ ( φ Proof: * L ˆ θ ( ( g ˆ g ˆ ( θ φ θ f L ˆ ˆ θ θ * L( θ fθ ( gθ ( L ( θ * θ L ˆ * θ ( L θ d o ˆ ( Hece for ever Ω of the ew prmeterzto The Algorthm: The Multdmeol Ce θ the MLE I the multdmeol ce the prmeter pce Ω {( θ θ > } Wrte the lelhood fucto L (( θ θ Wrte the log lelhood fucto defed l (( θ l( L( ( θ θ l( ( θ θ 3 Wrte the core fucto ( ( θ θ 4 Wrte the core equto ( ( θ θ θ l( ( θ θ θ (( θ θ l θ θ (( d olve l θ θ θ 5 Chec tht the oluto re the glol mmum (the mtr of the ecod prtl dervtve evluted t ˆ θ ˆ mut e egtve defte or equvletl ll egevlue egtve θ TADARD ERROR AD IA uppoe θˆ the MLE; ˆ φ ( φ ˆ θ ( the etmte of φ ( θ Oe meure of ccurc commol ued ME (me qured error How relle re the etmte? Defto: Me qured Error Pge 3 of 3

4 TA6Hdoc The me qured error defed ME ˆ φ ( ˆ φ φ( θ θ E θ for ech θ Ω Theorem E θ et the ME ( ˆ θ φ vrθ Eθ ( T φ( θ E the ME ( ˆ ( T (ued If φ ( θ R d T : R uch tht ( T ote: Whe ( T φ( θ E ( T φ( θ θ θ θ φ vr θ T Defto: E θ ( T φ( θ clled the the etmte Defto: tdrd Error θ ( T vr ( T TD clled the tdrd error of the etmte θ UFFICIEC The lelhood fucto for model d dt how how the dt upport the vrou pole vlue of the prmeter It ot the ctul lelhood ut the rto of the lelhood t dfferet vlue tht re mportt Defto: tttc A tttc fucto of oe or more rdom vrle tht doe ot deped o the uow prmeter Emple tttc ut µ ot ule µ d re ow ote Although tttc doe ot deped o uow prmeter t dtruto m Defto: uffcet tttc T T L θ cl θ for ome If the tttc T uch tht ( ( ( ( c > tht m deped o ( d ( the T clled uffcet tttc for the model Emple uppoe tht { 34 } Ω { } d the prolt dtruto gve 4 d 3 4 f θ θ θ f θ 6 34 Pge 4 of 3

5 TA6Hdoc the T uffcet tttc ote tht T h ol vlue compred 34 to 4 vlue the orgl model If we defe T T Theorem: Fctorzto Theorem If the det (or prolt fucto for model fctor f ( h( g ( T ( where h d g re o-egtve the T uffcet tttc θ θ θ Proof: uppoe T ( T ( the L( θ cfθ ( ch( gθ ( T ( h( gθ ( T ( c h( gθ ( T ( h( gθ ( T ( h( c h( g ( T ( T ( T ( h( θ c f ( L( θ Hece T uffcet tttc θ Defto: Mml uffcet tttc A mml uffcet tttc T for model uffcet tttc uch tht oce we hve the lelhood fucto L ( θ for the model we c determe T ( Emple Let L e rdom mple from ( µ ( ow The lelhood fucto ( µ ( µ e e the Fctorzto Theorem T ( π ( µ uffcet tttc ecue lelhood fucto potve multple of e whch completel determed t mmum vlue Hece mml uffcet tttc determed from the lelhood fucto of the model COFIDECE ITERVAL Defto: Qurtle For p [ ] the p th qurtle p for the dtruto wth cdf F the mllet umer p uch tht p F Whe F trctl creg d cotuou the F ( p the uque umer p uch tht p F or p F ( p ( p p Prtculr Ce: F 5 5 F 5 5 F clled the med clled the frt qurtle clled the thrd qurtle Pge 5 of 3

6 TA6Hdoc Defto: Cofdece Itervl c l u A tervl ( ( ( ( α-cofdece-tervl for ( θ P ( φ( θ c( P ( l φ( θ u α for ever θ Ω θ tervl θ φ f α referred to the cofdece level of the Cofdece Itervl for µ Let e rdom mple from ( µ α% cofdece tervl for µ c here Φ α α c or Φ ( ow The c where Φ the cdf of ( µ h ( dtruto A c (t clude the prmeter µ wth prolt α; Let e rdom mple from dtruto (ot orml wth fte me µ d fte vrce (ow CLT for µ c 3 Let µ c h lmtg ( dtruto A ppromte α% cofdece tervl ; here Φ e rdom mple from ( µ α α c or Φ c where Φ the cdf of ( ( uow The µ h t dtruto A α% cofdece tervl for µ c c ; here α G c or α c G where G the cdf of t 4 Let e rdom mple from mple from dtruto (ot orml wth fte me µ d fte vrce (uow CLT µ cofdece tervl for µ c of ( h lmtg ( dtruto A ppromte α% c ; here Φ α α c or c Φ where Φ the cdf ote tht P c c µ c o µ µ c c α c µ P c c α µ µ If Z ~ ( the P ( Z c P( c Z c Φ( c Φ( c Φ( c Φ( c ( Z c α Φ( c α Φ( c α P ( Φ( c o Pge 6 of 3

7 TA6Hdoc Cofdece Itervl for p Let oml( p ~ (p uow CLT ppromte α% cofdece tervl for p α c Φ where Φ the cdf of ( p h lmtg ( dtruto A ( ( ( c c ; here Φ α c or Cofdece Itervl for Let e rdom mple from ( µ χ dtruto A α% cofdece tervl for d P ( > Let α where ~ χ e rdom mple from ( µ ( µ ow uow The µ ( µ uow χ dtruto A α% cofdece tervl for d P ( > α where ~ χ µ µ ; here P ( < uow The ( ( ; here P ( < h α h α Tetg tttcl Hpothee Defto: tttcl Hpothe A tttcl hpothe erto out the dtruto of oe or more rdom vrle( If the hpothe completel determe the dtruto t clled mpl hpothe Otherwe t clled compote hpothe Emple H : θ 75 mple hpothe H : θ 75 H : θ 75 re compote hpothe > Defto: Tet A tet of tttcl hpothe rule uch tht whe the epermetl vlue ( hve ee oted led to deco to ccept or reect the hpothe uder coderto Pge 7 of 3

8 TA6Hdoc Defto: Crtcl Rego Let C e tht uet of the mple pce whch ccordce to the precred rule of the tet led to the reecto of the hpothe uder coderto The C clled the crtcl rego Defto: Power Fucto The power fucto of tet of tttcl hpothe of H gt H the prolt of reectg the hpothe uder coderto Defto: gfcce Level The mmum vlue of the power fucto whe H true clled the gfcce level of the tet Defto: et Crtcl Rego A uet C of the mple pce clled et crtcl rego of ze α for tetg H gt H f for ever P A H we hve uet A of wth (( α P( ( C H α P( ( C H P( ( A H Theorem: em-pero Theorem L( θ If we te C ( > L θ H : θ θ gt H : θ θ the C et crtcl rego of ze α for tetg AIC TET z-tet Let : µ µ e rdom mple from H true the Z ~ ( µ where ow Whe the ull hpothe µ We reect H f the P-vlue gve µ µ µ µ Pµ µ µ P Z Φ Φ Φ µ mll If the P-vlue le th α the the reult re d to e tttcll gfct t the ( α % level eroull Model Pge 8 of 3

9 Let TA6Hdoc e rdom mple from eroull ( θ where [ ] H : θ θ Whe H true the ppromte P-vlue P Z mll θ Z θ θ h lmtg ( θ ( θ Φ ( θ θ ( θ θ uow uppoe we wt to tet θ dtruto The the We reect H whe the P-vlue Equvlece etwee z-tet d z-cofdece-itervl Let e rdom mple from ( µ where ow The cofdece tervl α z α z α z α Φ clude µ wth prolt α If we decde tht for P- vlue le th α 5 we declre the reult tttcll gfct the we ow tht the reult re tttcll gfct wheever the 95% cofdece tervl for µ doe t cot µ t-tet Let T P e rdom mple from ( µ µ ~ t ( µ T µ uppoe we wt to tet H : µ µ Whe H true We reect the ull-hpothe H whe the P-vlue gve µ G mll; here G the cdf of t mple ze Clculto We m determe the mple ze o tht the mrg of error for α % cofdece tervl for µ doe ot eceed precred vlue δ > THE METHOD OF MOMET Let e rdom mple wth pdf f ( θ θ ( θ θ r Ω r the th momet of the dtruto The um m The epectto M E[ ] clled the th momet of the mple The Method of Momet A method of pot etmto clled the method of momet c e decred follow: Pge 9 of 3

10 TA6Hdoc Equte M to the epermetl vlue m egg wth d cotug utl there re eough equto to ot θ θ r fucto of m tht θ h ( θ θ r m r e Iferece Coder rdom vrle whoe dtruto deped o θ Ω We hve prevoul loo o θ eg ome uow cott Jut we loo o pole vlue of we ow loo o θ pole vlue of the rdom Θ tht h dtruto Π o the et Ω The Pror Dtruto We hll deote the pdf (or pf of Θ π d defe π ( θ whe θ Ω ( θ Th ecue π ( θ the pdf (or pf of Θ pror to the oervto o π clled the pror pdf of Θ The Poteror Dtruto Let e rdom mple from the dtruto of d let e tttc whch fucto of We c fd the codtol pdf (or pf of gve Θ whch we deote g ( θ ot pdf (or pf of d Θ gve ( θ π ( θ g( θ gve m ( θ dθ π ( θ g( θ Thu the If Θ cotuou the the mrgl pdf of dθ If Θ dcreet the tegrto would e replced ummto I ether ce the codtol pdf (or codtol pf of gve Θ ( θ π ( θ g( θ K θ m K ( θ clled the poteror pdf (or pf of Θ Th ecue m m K θ the pdf (or pf of Θ fter the oervto o h ee mde Model Checg Oe pproch to chooe the dcrepc tttc R D le the rego of low prolt the we reect the hpothe tht the model uder vetgto true I order to compre D ( wth D we eed to compute the P-vlue P ( D > D( Th c e doe v two method: Th method requre tht D e cllr I th method we ue the codtol dtruto of D gve the vlue of uffcet tttc T It c e how tht th codtol prolt the me for ever prmeter θ D : If the oerved vlue of D Defto: Acllr A tttc whoe dtruto doe ot deped o the prmeter θ clled cllr Emple Pge of 3

11 TA6Hdoc We ume µ (µ uow ow It h ee how tht mml uffcet tttc Coder mple vlue ( d defe r r ( ( r r ( It c e how tht: R R R h dtruto tht depedet of µ Hece R rdom mple from cllr R depedet of o the codtol dtruto R Therefore the two method gree th ce Alo ( ( R ~ R ( D R oerved vlue ( R ow coder the dcrepc tttc the me the dtruto of We ow D( R ~ χ Compute ( D( R D( P > to ee f the D rego of low prolt or ot (vlue cloe to d dcte tl of the dtruto d oth ce dcte rego of low prolt If t the we reect the model uder vetgto eg true Emple We ume tht ( rdom mple from ( µ mml uffcet tttc (oth µ d uow It h ee how Coder mple vlue ( d defe r r r r It c e how tht: R R R h dtruto tht depedet of µ d Hece R cllr R depedet of o the codtol dtruto R the me the dtruto of R Therefore the two method gree th ce ow coder the dcrepc tttc R D R l To ue th tttc for model checg we eed to ow the dtruto of the tttc; th c e doe v multo The compute P ( D( R > D( to ee f the oerved vlue D ( rego of low prolt or ot (vlue cloe to d dcte tl of the dtruto d oth ce dcte rego of low prolt If t the we reect the model uder vetgto eg true CHI-QUARED GOODE OF FIT TET Let hve multoml dtruto wth prmeter d Defe Q ( p lmtg χ dtruto Hece Q ppromtel p p where p p p It c e how tht Q h p χ dtruto Pge of 3

12 TA6Hdoc Procedure Let A e the mple pce of rdom epermet A A A A A where A A Let p P( A 3 We repet the epermet tme 4 Let deote the umer of tme the outcome of the rdom epermet h multoml dtruto wth prmeter p p 5 We tet the mple hpothe other ltertve 6 If H true the the tttc ( Q c α H where Q : p p p p ( p h ppromte p A The p p re pecfed vlue gt ll χ dtruto We fd c o tht P where α the dered gfcce level of the tet (the gfcce level of the tet ppromtel equl to α 7 We reect H f the oerved vlue of Q c METHOD OF LEAT QUARE uppoe we wt to etmte E ( ed o mple vlue ( We elect the pot t ( the et of pole vlue of E ( tht mmze ( t( The etmte t ( clled the let qure etmte of ( E ote We hve ( t( (( ( t( ( ( t( ( ( t( ( ( t( ( t( ( ( t( ( t( ( o ( t( ( ( t( Th mmzed whe t( f pole vlue of E ( ; f ot we chooe pole vlue of ( etmte of t ( E tht cloet to the ut Ce of Rdom Vector Pge of 3

13 TA6Hdoc We hve d E E E( t We oerve ( R d fd t ( mmze ( t ( t { pole vlue of E( } tht Regreo Model: The mple Ler Regreo Model We tud the relto etwee the repoe vrle (depedet d the predctor vrle (depedet I the Regreo Model we th the chge the through the codtol me tht chge E chge Defto I the mple Regreo model we ume ( coeffcet E d re clled the regreo Defto: ctter Plot A ctter plot plot of dt pot ( ( d the form of the relto It how whether relto et etwee d LEAT QUARE ETIMATE PREDICTIO AD TADARD ERROR Defto: Let qure Etmte uppoe we oerve the depedet umer ( ( We hve E The lee qure etmte of the codtol me the t the et of pole vlue of the codtol me tht mmze vlue of ( The d tht mmze th clled the let qure etmte of d Theorem uppoe tht E( d we oerve the depedet vlue ( ( ( The the let qure etmte of for Pge 3 of 3

14 TA6Hdoc ( ( wheever ( ote ( ( ( o ( ( ( Theorem If ( d we oerve depedet vlue ( ( the E where ; E E where ( ( ( ( Thu d hve ued propert tht the re ued etmtor Remr A turl predctor of future vlue of whe ( ecue we do ot hve E the vlue of d we ue the etmte d for predcto Tht ue the le Theorem If E( d vr( ( ( the vr( vr( cov( ( d we oerve depedet vlue ( Corollr Pge 4 of 3

15 TA6Hdoc Pge 5 of 3 From the theorem ove we ot vr Proof: cov vr vr vr Theorem If E d vr d we oerve depedet vlue for the E where Thu ued etmte of Therefore the tdrd error of THE AOVA DECOMPOITIO AD THE F TATITIC Lemm If re uch tht the ueful decompoto of clled the regreo um of qure (R clled the error um of qure (E AOVA (Al of Vrce Tle ource DF (degree of freedom um of qure Me qure

16 TA6Hdoc Pge 6 of 3 Error Totl Reult We hve E whch equl to f d ol f I other word ued etmtor of f d ol f Proof: E E E vr Defto: The F-tttc ce ued etmte of hece the F-tttc gve E R F the rto of two ued etmte of whe : H (there ler effect due to true Therefore reect : H whe F lrge THE COEFFICIET OF DETERMIATIO AD CORRELATIO Defto: Coeffcet of Determto

17 TA6Hdoc Pge 7 of 3 Defe R the coeffcet of determto ce we hve tht R o R ote: Whe we ue the model to me predcto vlue of R er me hghl ccurte predcto whle vlue of R er me the predcto wll ot e ver ccurte Defto: Correlto We defe the correlto coeffcet D D cov corr ρ We ow tht ρ d tht c ± ± ρ Defto: mple Correlto We defe the mple correlto r where the mple covrce etmtg cov d d re mple tdrd devto of d We hve tht r d c r ± ± Theorem If re uch tht d the r R Proof: R r COFIDECE ITERVAL AD TETIG HPOTHEE Theorem

18 If dtruted ( TA6Hdoc d we oerve the depedet vlue ( ( ( the the codtol dtruto of d ~ ~ ~ ( ~ χ depedet of ( gve re: for Corollr ( 3 ~ t ( ( ( ~ t ~ t( ( ( ( 4 If F defed F R the E : H true f d ol f ~ F( F Cofdece Itervl A α% cofdece tervl for t ( A α% cofdece tervl for α α ( t t ( α t ( ( ( α Tetg Hpothe We c tet the hpothe H : computg the P-vlue P F ~ F F to ee whether or ot the oerved vlue le rego of low prolt ( Pge 8 of 3

19 TA6Hdoc otce tht we c lo tet the hpothe H computg the P-vlue : ( P T T ~ t( It c e how tht thee two P-vlue re equl AALI OF REIDUAL Model checg ed o the redul dcued erler Corollr E vr( Defto: tdrdzed Redul We defe the th tdrdzed redul ( ( ote If the codtol dtruto of orml the ( ( ppromtel true for ( ( for lrge ~ ( Th THE REIDUAL AD PROAILIT PLOT Oe pproch to model checg to ee f the vlue of the tdrdzed redul loo le mple from For th we ue the redul d prolt plot Redul Plot We defed the redul erler r ( r r ( where ( from It c e how tht ( ( R ~ µ We tdrdze R R * mple vlue d depedet of ( * The ~ ( R h me d vrce Pge 9 of 3

20 TA6Hdoc Whe * The plot of ( r uow we etmte t ppromtel dtruted ( I th ce R * ( hould ot how dcerle ptter Mot of the vlue hould le ( 33 A dcerle ptter wth everl etreme vlue evdece gt the model umpto to e correct multg orml rdom vrle d plottg t gve good de of how the plot hould loo Prolt Plot uppoe rdom mple from ( µ Φ tht the we cll ( µ Φ dcte tht f we plot the pot ( Φ e how tht the epectto of the th order tttc tfe E correpod to the order tttc E tercept µ d lope We cll uch plot prolt plot ; uppoe mple vlue The t c µ If the dt Φ the orml core of The the hould le o le wth Regreo Model: Oe Ctegorcl Predctor (Oe-W AOVA ow uppoe tht the predctor te vlue Let E( the me repoe whe te the vlue Defe We hve E( ce ol oe of the d the ret re we ot the mple ler regreo model d hece ll the reult prevoul oted hold Let qure Etmte ow uppoe we hve let qure etmte of It c e how tht vlue ( whe d ll repoe vlue re depedet The re oted mmzg ( ued etmtor of o the let qure etmte of tht E( E( mple Vrce Aumg ll hve codtol vrce vr tht vr( The ( vr ( vr( vr( ( cov whe codtol covrce d the Pge of 3

21 TA6Hdoc Pge of 3 We hve ued etmte of Cofdece Itervl d Tetg If we ume tht ~ the ~ depedet of ~ χ ow ~ d hece ( defto t T ~ ce α α < < < < P P hece α% cofdece tervl for t t α α Alo we c tet the ull hpothe : H computg the P-vlue G T P ; where G the cdf of the t dtruto IFERECE AOUT DIFFERECE OF MEA We ow tud ferece out Epectto d Vrce We hve E E E d vr vr vr ce d re depedet Cofdece Itervl ce ~ d ~ o ~ We hve ~ depedet of ~ χ Thu t T ~

22 TA6Hdoc Pge of 3 ce α α < < < < P P hece α% cofdece tervl for t t α α Alo we c tet the ull hpothe H : computg the P-vlue G T P ; where G the cdf of the t dtruto ote If te o ol vlue I th ce T clled the two-mple t-tttc the cofdece tervl clled the two-mple t-cofdece-tervl d the t-tet clled the two-mple t-tet I th ce f we coclude tht the reltohp et etwee d I geerl whe to tet the ull hpothe tht there o reltohp etwee the repoe d the predctor equvlet to H : If H true the let qure etmte of AOVA Tle We hve the compoto ource DF (degree of freedom um of qure Me qure Error Totl F-tttc To e H we ue the F-tttc Wth ormlt umpto we hve F F ~ d o we compute > F P d ee f the oerved vlue of F rego of low prolt or ot Whe th P-vlue equl to the P-vlue we oted for the two-mple t-tet

23 TA6Hdoc Model Checg To chec the model we loo t the tdrdzed redul gve efore d loo t the redul plot Pge 3 of 3