Lecture 5. Estimation of Variance Components

Transcription

1 Lctur 5 Etmato of Varac Compot Gulhrm J. M. Roa Uvrt of Wco-Mado Mxd Modl Quattatv Gtc SISG Sattl 8 0 Sptmbr 08 Etmato of Varac Compot ANOVA Etmato Codr th data t blow rlatd to obrvato of half-b faml of urlatd r. h followg modl ca b ud to rprt th data: µ j j Sr whr j rprt th photpc trat obrvato of prog j j faml µ a ma a ffct commo to all amal havg r ad j a rdual trm

2 Etmato of Varac Compot ANOVA Etmato h r ffct quvalt to th tramttg ablt whch qual to o-half addtv gtc valu of r a o-half of t g ar radoml tramttd to ach of t prog. h rdual trm j rfr to addtoal gtc ffct uch a th ffct of dam ad vromtal compot. It aumd that d d ~0 ad j ~ 0 From th modl ttg dcud bfor w hav that E[ j ] µ ad Var[ j ] h ovrall ampl ma gv b j N j N whr N ad j ar r-pcfc ma. j h ANOVA approach cot of a orthogoal dcompoto of th total um of quar SS to btw cla or our ca r ad wth cla or rdual compot. h corrctd trm of th gral ma SS gv b: SS j j

3 B addg ad ubtractg SS ca b xprd a: SS j j [ j j ] wth th parth th j j j It that th lat part of th xpro qual to zro o that SS ca b wrtt a two compot: SSS j ad RSS j whch ar th r ad th rdual um of quar rpctvl. h SSS trm maur th varato of ach prog faml aroud th ovrall ma whl th RSS trm maur th xtra varato rlatd to ach obrvato aroud t r avrag j It ca b how that th xpctato of th um of quar trm ar: E[SSS] N N o that th ANOVA tmator of th r ad rdual varac compot ar gv b: ˆ N N [SSS ˆ I th pcfc ca of balacd data.. th am prog z for all r N/ ad th ANOVA tmator bcom: ˆ SSS ˆ ad ˆ RSS ] ad ad E[RSS] ˆ N RSS N 3

4 Appdx: Calculatg EMS Modl: j µ j wth µ fxd E[µ] µ E[µ ] µ Var[µ] 0 d ~ N0 E[ ] 0 E[ ] Var[ ] d j ~ N0 E[ j ] 0 E[ j ] Var[ j ] Cov[ ' ] Cov[ j ] Cov[ j ' j' ] 0 Sum of Squar: SSS j RSS j j j j K Expctato: E j j E ad E E j j E j E µ j j j E µ j µ µ j j j µ E[ ] E[ j ] µe[ ] µe[ j ] E[ ]E[ j ] j µ j µ 4

5 E E j j E µ E µ j j j j E µ j DP j µ 0 µ E E[ ] E j j E µ j j E µ j DP j m 0 µ 5

6 Expctd MS E[SMS] E[SSS] E E[RMS] µ µ E[RSS] E j j E µ µ Etmato of Varac Compot ANOVA approach wor wll for mpl modl uch a a o-wa tructur or balacd data uch a data from dgd xprmt wth o mg data but th ar ot dcatd for mor complx modl ad data tructur Othr propod mthod: xpctd ma quar approach of Hdro 953 ad th mmum orm quadratc ubad tmato Rao 97a 97b amog othr. Howvr maxmum llhood bad mthod ar currtl th mot popular pcall th rtrctd or rdual maxmum llhood REML approach whch attmpt to corrct for th wll-ow ba th clacal maxmum llhood ML tmato of varac compot. h two mthod ar brfl dcrbd xt. 6

7 Etmato of Varac Compot Maxmum Llhood ML Etmator Maxmum llhood tmat of th varac compot ca b obtad b maxmzg th logllhood L β G Σ wth rpct to ach lmt of G ad Σ aftr rplacg β b βˆ X V X X V Altratvl G Σ ad β ca b tmatd multaoul b maxmzg thr jot log-llhood wth rpct to th varac compot ad th fxd ffct. A a mpl xampl of maxmum llhood tmato of varac compot codr th balacd ca.. cotat prog z half-b faml data t dcud prvoul ad th lar modl: µ j j wth th am dfto a bfor but wth th addtoal aumpto of ormalt of both th r ad th rdual ffct..: d d ~ N0 ad j ~ N0 7

8 8 I matrx otato th modl ca b xprd a: whr rprt th vctor of obrvato of prog.. rlatv to r ; ad 0 rprt -dmoal colum vctor of ad 0 rpctvl; ad d th vctor of rdual aocatd wth prog µ " # " " ] [ ] [ h vctor of obrvato ha th a multvarat ormal dtr. wth ma vctor ad varac-covarac matrx gv b ad t dt fucto from whch th llhood fucto obtad ca b wrtt a: whr a matrx of ad th Krocr product ] [ µ N µ N I I / N N / p π µ I J I µ µ xp N N I J π µ µ xp N N N N J I J

9 h log-llhood fucto ca b wrtt th a: N l µ log log j B tag th drvatv ad ttg thm to 0 th followg oluto ar obtad: µ ˆ ˆ RSS ad from whch ML tmat of th varac compot ar obtad xcpt f ˆ < 0 whch ca th tmat t to zro ML tmat of varac compot ar bad dowward a th do ot ta to accout th dgr of frdom ud for tmatg th fxd ffct ˆ j SSS ˆ µ Etmato of Varac Compot Rdual Maxmum Llhood REML Etmator Rtrctd or rdual maxmum llhood approach REML: corrct th ba aocatd wth ML tmat b tag to accout th dgr of frdom ud for tmatg th fxd ffct REML maxmz th llhood fucto of a t of rror cotrat d L whr L a [ x p] full-ra matrx wth colum orthogoal to th colum of th cdc matrx X h vctor d follow a multvarat ormal dtrbuto wth ull ma vctor ad varaccovarac matrx L VL L ZGZ ΣL. Not that th dtrbuto of d do ot dpd o β. 9

10 0 h rdual llhood fucto for th varac compot th: Aothr approach for obtag th rdual llhood fucto for th varac compot b tgratg th fxd ffct out of th full llhood fucto..: a llutratd th followg xampl. π d VL L d VL L GΣ / p / xp L β GΣ β Σ G d L L Rcall th balacd half-b faml data t ad t aocatd llhood fucto: It rdual llhood th: N N L π µ µ j j xp µ µ d L L N N π µ µ d xp xp j j

11 whch qual to: L N N π λ xp xp j j λ µ λ π λ whr. B tag th drvatv wth rpct to λ ad ad b ug th varac proprt of maxmum llhood tmator th followg oluto ar obtad: ˆ RSS ad ˆ SSS ˆ whch ar th REML tmat of th varac compot xcpt f ˆ < 0.. f SSS < RSS

12 Explct form of ML ad REML tmator ar oft ot avalabl for mor complx mxd ffct modl ML ad REML tmat ar th grall obtad b tratv approach uch a th xpctato-maxmzato EM algorthm ad Nwto-Rapho-bad procdur Baa Data Aal Ifrc ug probablt modl for quatt w obrv ad for quatt about whch w wh to lar Explct u of probablt for quatfg ucrtat frc bad o tattcal data aal

13 Codtoal Probablt Ba Rul Ω A B PA B P A B PB PAPB A PB Baa Ifrc : obrvd data; ~ p θ θ: paramtr all uobrvd quatt p θ p θ p p θp θ p potror dtrbuto p θ p θp θ pror dtrbuto amplg dtrbuto 3

14 Pror Dtrbuto Iformatv ad Noformatv Propr ad Impropr Cojugat ad Nocojugat Jffr Pror Maxmum Etrop Rfrc Pror Exampl : Bomal Dtrbuto d Data: ~ B θ θ Prob Samplg modl: p θ p θ θ θ " $ # % ' θ θ & Pror: pθ Btaa b θ a θ b Potror: pθ θ a b θ θ ~ Bta a b 4

15 Exampl : Bomal Dtrbuto θ ~ Bta a b Potror ma: E[θ ] a a b Potror mod: Mod[θ ] a a b Potror varac: prct HPD tc. Fatur of th potror dtrbuto: b Var[θ ] a a b a b Exampl : Bomal Dtrbuto Sttg for xampl a ad b : Pror: Potror: pθ Uform0 pθ θ θ θ ~ Bta Not that th ca th potror mod cocd wth th maxmum llhood tmat of θ: Mod[θ ] 5

16 Exampl : Normal Dtrbuto d Data: ~ Nµ wth ow Samplg modl: p µ p µ p µ π xp $ ' % µ & & xp' π / µ * & xp π / µ ' * Exampl : Normal Dtrbuto Pror Cojugat: Jot potror: pµ µ ~ Nφ τ pµ p µ pµ πτ xp % & µ φ ' τ * & xp π / µ ' * πτ xp & ' µ φ* τ 6

17 Jot potror cot d: $ pµ xp ' % µ & xp $ ' % µ φ & τ & µ µ φ xp' * τ xp µ % % ' *µ & τ φ ' * % - '. & τ τ & # xp & $ µ µ ' % # & µ % $ τ ' $ φ ' whr ad & % τ $ τ Hc: µ ~ N τ & τ φ % $ ' & % τ φ τ ' / * 0 Mult Paramtr Modl ~ p θ θ θ p pθ θ θ p ~ pθ θ θ p p θ θ θ p Margal Potror Dtrbuto pθ pθ θ θ p dθ θ θ θ θ 7

18 Margal Potror Dtrbuto Margalzato.. tgral mult-dmoal modl ca b cumbrom ad om tm do ot hav aaltcal form A altratv th rgard: Mot Carlo mthod Mot Carlo tgrato cot of amplg from th potror dtrbuto ad th ug uch ampld valu to calculat fatur of trt o th jot or margal potror dtrbuto hr ar ma algorthm that ca b ud to ampl from a dtrbuto; om ar bad o Marov cha amog whch th Gbb amplg probabl th mot popular Gbb Samplg θ θ θ θ r pθ θ θ θ θ r θ 0 θ 0 θ 0 θ r 0 θ θ 0 θ 3 0 θ r 0 θ θ θ 3 0 θ r 0 θ r θ θ 3 θ r Bur- & Covrgc g trval & Lag corrlato Sampl z & Mot Carlo rror 8

19 Mot Carlo Approxmato Aftr covrgc ach ampld vctor a ampl from th jot potror dtrbuto ad o ach ampld lmt calar a ampl from th rpctv margal potror dtrbuto For ach paramtr.g. θ w ll hav th a r of valu: θ θ θ 3 θ N from whch fatur of t dtrbuto.g. potror ma ca b approxmatd for xampl: N E[θ ] N θ j j Exampl Iformato o photp ad gotp for a pcfc marr ba lar rgro Marr Gotp Photp 8 dvdual pr group MM Mm mm