Outline. Outline. Outline. Questions 2010/9/30. Introduction The Multivariate Normal Density and Its Properties

Transcription

1 9 Multvrt orml Dstruto Shyh-Kg Jg Drtmt of Eltrl Egrg Grdut Isttut of Commuto Grdut Isttut of tworkg d Multmd Outl Itroduto Th Multvrt orml Dsty d Its Prorts Smlg from Multvrt orml Dstruto d Mmum Lklhood Estmto Th Smlg Dstruto of d S Lrg-Sml hvor of d S Outl ssssg th ssumto of ormlty Dttg Outlrs d Clg Dt Trsformtos to r ormlty Outl Itroduto Th Multvrt orml Dsty d Its Prorts Smlg from Multvrt orml Dstruto d Mmum Lklhood Estmto Th Smlg Dstruto of d S Lrg-Sml hvor of d S Qustos Wht s th uvrt orml dstruto? Wht s th multvrt orml dstruto? Why to study multvrt orml dstruto? Multvrt orml Dstruto Grlzd from uvrt orml dsty s of my multvrt lyss thus Usful romto to tru oulto dstruto Ctrl lmt dstruto of my multvrt sttsts Mthmtl trtl 6

2 9 Outl Itroduto Th Multvrt orml Dsty d Its Prorts Smlg from Multvrt orml Dstruto d Mmum Lklhood Estmto Th Smlg Dstruto of d S Lrg-Sml hvor of d S Qustos Wht s th formul for th rolty dsty futo of uvrt orml dstruto? Wht r th rolty mg of rmtrs d? How muh rolty r th trvls - + d - +? How to look u th umultd uvrt orml rolty Tl d? Qustos Wht s th Mhlos dst for uvrt orml dstruto? Wht s th Mhlos dst for multvrt orml dstruto? Wht r th symol for d th formul of th rolty dsty of -dmsol multvrt orml dstruto? Qustos Wht r th ossl shs surf dgrm of vrt orml dsty? Wht s th ostt rolty dsty otour for -dmsol multvrt orml dstruto? Wht r th gvlus d gvtors of th vrs of? Rsult. Qustos Wht s th rgo tht th totl rolty sd uls -? Wht s th rolty dstruto for lr omto of rdom vrls wth th sm multvrt-orml dstruto? Rsult. How to fd th mrgl dstruto of multvrt-orml dstruto y Rsult.? Qustos Wht s th rolty dstruto for rdom vtor otd y multlyg mtr to rdom vtor of rdom vrls wth th sm multvrt-orml dstruto? Rsult. Wht s th rolty dstruto of rdom vtor of multvrt orml dstruto lus ostt vtor? Rsult.

3 9 Qustos Gv th m d ovr mtr of multvrt rdom vtor d th rdom vtor s rttod how to fd th m d ovr mtr of th two rts of th rttod rdom vtor? Rsult. Qustos Wht r th f-d-oly-f odtos for two multvrt orml vtors d to ddt? Rsult.5 If two multvrt orml vtors d r ddt wht wll th rolty dstruto of th rdom vtor rttod to d? Rsult.5 Qustos rdom vtor s rttod to d th wht s th odtol rolty dstruto od gv =? Rsult.6 Wht s th rolty dstruto for th sur of th Mhlos dst for multvrt orml vtor? Rsult.7 Qustos How to fd th vlu of th Mhlos dst for multvrt orml vtor wh th rolty sd th orrsodg llsod s sfd? Rsult.7 Qustos Wht s th sh of h-sur dstruto urv? How to look u th umultd h-sur rolty from Tl d? Wht s th ot dstruto of two rdom vtors whh r two lr omtos of dffrt multvrt rdom vtors? Rsult.8 Uvrt orml Dstruto f [ ] 8

4 9 Tl d Tl d 9 9 Sur of Dst Sur of Dst Mhlos Mhlos dst dst -dmsol orml Dsty dmsol orml Dsty f ] [ s sml from rdom vtor Eml. vrt orml Eml. vrt orml r r E E Corr Eml. Surd Dst Eml. Surd Dst ] [ Eml. Dsty Futo Eml. Dsty Futo f ]} [ {

5 9 Eml. vrt Dstruto Eml. vrt Dstruto = = 5 = =.75 6 Cotours Costt rolty dsty otour s ll suh tht - surf of llsod trd t Rsult. ostv dft - for for ostv dft Eml. vrt Cotour Eml. Postv Corrlto vrt orml gvlus d gvtors [ [ ] ] 9 5

6 9 6 Prolty Rltd to Prolty Rltd to Surd Dst Surd Dst vlus stsfyg Sold llsod of rolty hs Prolty Rltd to Prolty Rltd to Surd Dst Surd Dst Rsult. Rsult. must for vry Eml. Mrgl Dstruto Eml. Mrgl Dstruto ] [ ] [ dstruto of Mrgl Rsult. Rsult. 5 5 d d Proof of Rsult. Prt Proof of Rsult. Prt y lr omto 6 6 vld for vry

7 9 7 Proof of Rsult. Prt Proof of Rsult. Prt d d 7 7 s rtrry d d d d Eml. Lr Comtos Eml. Lr Comtos 8 8 vrfd wth Y Y Rsult. Rsult. 9 9 Rsult. St Proof I Eml.5 Sust Dstruto Eml.5 Sust Dstruto 5 Rsult.5 Rsult Cov ddt ddt d oly f ddt f Eml.6 Idd Eml.6 Idd d lso s ddt of r ddt d ot ddt

8 9 8 Rsult.6 Rsult.6 ovr d wth m orml s gv dstruto of odtol Proof of Rsult.6 Proof of Rsult.6 I I wth ovr ot orml Proof of Rsult.6 Proof of Rsult.6 ddt r ddt d f P P P P 5 5 gv f Eml.7 Codtol vrt Eml.7 Codtol vrt 6 6 show tht f Eml. Dsty Futo Eml. Dsty Futo f 7 7 ]} [ { Eml.7 Eml.7 ] [ 8 8 f f f

9 9 9 Rsult.7 Rsult.7 Th rolty sd th sold llsod 9 9 dstruto th rtl of th th ur dots whr }s { Th rolty sd th sold llsod Dstruto Dstruto ; Z 5 5 Gmm dstruto wth frdom d.f. dgrs of f Dstruto Curvs Dstruto Curvs 5 5 Tl d Tl d 5 5 Proof of Rsult.7 Proof of Rsult.7 Z Z 5 5 Z Z I Proof of Rsult.7 Proof of Rsult.7 ssgd to th llsod y th rolty s P 5 5 dstrutd y w rdom vrl P

10 9 Rsult.8 Rsult.8 mutully ddt wth ovr mtr ot orml r d d Proof of Rsult.8 Proof of Rsult.8 ] [ I I I I I I dgoltrmsof off trms of lok dgol Eml.8 Lr Comtos Eml.8 Lr Comtos ddt dtl Eml.8 Lr Comtos Eml.8 Lr Comtos Cov Outl Itroduto Th Multvrt orml Dsty d Its Prorts Smlg from Multvrt Smlg from Multvrt orml Dstruto d Mmum Lklhood Estmto Th Smlg Dstruto of d S Lrg-Sml hvor of d S Qustos Wht r rdom smls? Wht s th lklhood? How to stmt th m d vr of uvrt orml vr of uvrt orml dstruto y th mmumlklhood thu? ot stmts Wht s th multvrt orml lklhood?

11 9 Qustos Wht s th tr of mtr? How to omut th udrt form usg th tr of th mtr? Rsult 9.9 How to rss th tr of mtr y ts gvlus? Rsult.9 Rsult. Qustos How to stmt th m d ovr mtr of multvrt orml vtor? Rsult. Wht s th vr rorty of th mmum lklhood stmts? Wht s th sufft sttsts? Mmum-lklhood Estmto Multvrt orml Lklhood s futo of lklhood rdom sml from Jot dsty of Mmum lklhood stmto Mmum lklhood stmts d for fd 6 6 kk Tr of Mtr k tr tr tr k k ; tr tr tr tr tr tr tr d tr s slr 65 k vtor Rsult.9 k k symtr mtr tr tr tr k 66

12 9 Proof of Rsult.9 m k mtr tr C tr C t tr C m k C k m mtr k m m k tr C tr tr tr tr C Proof of Rsult.9 P ΛP P P I Λ dg k tr tr P ΛP tr ΛPP tr Λ k Lklhood Futo tr L tr 69 Rsult. symmtr ostv dft mtr ostv slr tr holdg oly for for ll ostv dft wth ulty 7 tr Proof of Rsult. tr tr gvlus of tr tr hs mmum ll ostv t tr ur oud s ttd wh suh tht I 7 Rsult. Mmum Lklhood Estmtors of d ˆ ˆ rdom sml from S 7

13 9 Proof of Rsult. Eot of L tr ˆ L ˆ ˆ tr S Emls Ivr Prorty ˆ mmum lklhood stmtor of h ˆ mmum lklhood stmtor of h MLE of MLE of ˆ ˆ ˆ ˆ ˆ MLE of r 7 7 Sufft Sttsts Jot dsty of orml oulto tr through d S dds o th whol st of osrvtos d S r sufft sttsts of multvrt 75 Outl Itroduto Th Multvrt orml Dsty d Its Prorts Smlg from Multvrt orml Dstruto d Mmum Lklhood Estmto Th Smlg Dstruto of d S Lrg-Sml hvor of d S Qustos Wht s th dstruto of sml m for multvrt orml smls? Wht s th dstruto of sml ovr mtr for multvrt orml smls? Dstruto of Sml M rdom sml from Uvrt s Multvrt s f. Rsult.8 78

14 9 Smlg Dstruto of S Uvrt s rdom sml from s s Multvrt s - S Z Z Z Z Z Wshrt dstruto W - S 79 w Wshrt Dstruto ostv dft dft Prorts W W m W m m m W CC W m tr m CC CC 8 Outl Itroduto Th Multvrt orml Dsty d Its Prorts Smlg from Multvrt orml Dstruto d Mmum Lklhood Estmto Th Smlg Dstruto of d S Lrg-Sml hvor of d S Qustos Wht s th uvrt trl lmt thorm? Wht s th lw of lrg umrs for th uvrt s d th multvrt s? Rsult. Wht s th multvrt trl lmt thorm? Rsult. Qustos Wht s th lmt dstruto for th sur of sttstl dst? Uvrt Ctrl Lmt Thorm dtrmd y lrg umr of uss ddt rdom vrls hvg romtly th sm vrlty hs rly orml dstruto s lso rly orml for lrg sml sz 8

15 9 Rsult. Lw of Lrg umrs Y Y. Y ddt osrvtos from oulto my ot orml wth E Y Y Y Y Y ovrgs rolty to Tht s for y rsrd P[ -ε Y- ε] s Rsult. Multvrt Css ddt osrvtos from oulto my ot multvrt orml wth m E ovrgs rolty to S ovrgs rolty to Rsult. Ctrl Lmt Thorm ddt osrvto from oulto wth m d ft ovr s romtly for lrg sml sz ut good romto for modrt wh th rt oulto s rly orml 87 for lrg - Lmt Dstruto of Sttstl Dst rly for lrg sml sz romtly S los to wth hgh rolty wh s lrg S for lrg - romtly 88 Outl ssssg th ssumto of ormlty Dttg Outlrs d Clg Dt Trsformtos to r ormlty Qustos How to dtrm f th smls follow orml dstruto? Wht s th Q-Q lot? Why s t vld? How to msur th strghtss Q-Q lot? 5

16 9 Qustos How to us Rsult.7 to hk f th smls r tk from multvrt orml oulto? Wht s th h-sur lot? How to us t? Lt Q-Q Plot osrvtos o dstt d modrt to lrg.g. Porto of z P[ Z ] dz Plot to s f thy r romtly lr s f th dt r from orml dstruto 9 Eml.9 Eml Hstogrm of MdTrm Sors of Studts of Ths Cours 6 Q-Q Q Plot of MdTrm Sors of Studts of Ths Cours ~59 6~69 7~79 8~89 9~ = r Q =

17 9 Eml. Rdto Dt of Closd-Door Mrowv Ov Msurmt of Strghtss r Q sgf f r vlu Tl. Rt th ormlty hyothss t lvl of Q flls low th rort Tl. Q-Q Plot Corrlto Cofft Tst Eml. For dt from Eml r Q r. Q Do ot rt ormlty hyothss 99 Evlutg vrt ormlty Eml. Chk f roughly 5% of sml osrvtos l th lls gv y ll suh tht S.5 7

18 Eml. 5. S d 97.9 [6.97] d Sv out of osrvtos r wth Grtr th 5% rt vrt ormlty d.9 Howvr sml sz s too smll to rh th oluso 5 d S Ch-Sur Plot surd dst Ordr th surd dst d d d utl of th h - sur dstruto wth dgrs of frdom Grh ll d Th lot should rsml strght l through th org hvg slo ot tht Eml. Ch-Sur Plot for Eml. Eml. Ch-Sur Plot for Eml. 5 6 Ch-Sur Plot for Comutr Grtd -vrt orml Dt Outl ssssg th ssumto of ormlty Dttg Outlrs d Clg Dt Trsformtos to r ormlty 7 8

19 9 Sts for Dttg Outlrs Mk dot lot for h vrl Mk sttr lot for h r of vrls Clult th stdrdzd vlus. Em thm for lrg or smll vlus Clultd th surd sttstl dst. Em for uusully lrg vlus. I h-sur lot ths would ots frthst from th org. 9 Outl ssssg th ssumto of ormlty Dttg Outlrs d Clg Dt Trsformtos to r ormlty Qustos How to trsform sml outs roorto d orrlto suh tht th w vrl s mor r to uvrt orml dstruto? Wht s o d Co s uvrt trsformto? How to td o d Co s trsformto to th multvrt s? Qustos How to dl wth dt ludg lrg gtv vlus? Hlful Trsformto to r ormlty Orgl Sl Couts y Proortos ˆ Corrltos r Trsformd Sl y ˆ logt ˆ log ˆ r Fshrs z r log r o d Co s Uvrt Trsformtos l Choos λ to mmz l l 9

20 9 Eml.6 vs. Eml.6 Q-Q Plot 5 6 Trsformg Multvrt Osrvtos owr trsformtos for k th hrtrsts Slt to mmz k l k k k λ ˆ k ˆ ˆ ˆ ˆ ˆ ˆ l k 7 Mor Elort roh owr trsformtos for th hrtrsts Slt λ to mmz l S λ k S λ s omutd from λ k l k 8 Eml.7 Orgl Q-Q Plot for O-Door Dt Eml.7 Q-Q Plot of Trsformd O-Door Dt 9

21 9 Eml.7 Cotour Plot of for oth Rdto Dt Trsform for Dt Iludg Lrg gtv lus log log