Bootstrap Information Criteria for Linear Mixed Models

Transcription

1 Int. Statstcal Inst. Proc. 58t World Statstcal Congress, 2, Dubln (Sesson CPS53) p.578 Bootstrap Informaton Crtera for Lnear Mxed Models Ypng, Tang Graduate Scool of Scence, Cba Unversty -33 Yayo-co, Inage-ku, Cba , Japan E-mal Jnfang, Wang Department of Matematcs and Informatcs, Cba Unversty -33 Yayo-co, Inage-ku, Cba , Japan E-mal Introducton Lnear mxed models (LMM) ave receved tremendous attenton n te lterature snce te semnal paper by Lard and Ware (982) due to ter ablty to represent clustered (terefore dependent) data. Model selecton n LMM ave tradtonally been carred out usng te margnal verson of te Akake nformaton crteron (maic), wc s desgned for cross-sectonal data (Pnero and Bates (2) Ngo and Brand (22)). In an mportant paper, Vada and Blancard (25) ponted out tat wen te focus of te researc s on te clusters nstead of te populaton a more approprate crteron for selectng LMM s to use te condtonal verson of te Akake nformaton crteron (caic). Snce Vada and Blancard (25) a number of papers ave been publsed recently dscussng te use of maic and caic (see e.g. Lang, et al. (28), Greven and Kneb (2) and Srvastava and Kubokawa (2)). In ts secton we gve a revew of maic and te basc verson of caic proposed orgnally by Vada and Blancard (25), wc seems to be used n practce (Greven and Kneb, 2). Suppose tat we ave data from m clusters, n te t cluster te n -vector of response y beng modeled by y = X f + b + ffl = m were f = (f f p ) s a vector of fxed effects, b = (b b q ) s a vector of random effects, X and are te n p and n q covarate matrces for te fxed and random effects of full column ranks. Te error vectors ffl and te random effects b are ndependently and normally dstrbuted, ffl οn n ff 2 I n b οn q ff 2 K were I n s te n n dentty matrx and G a q q postve defnte matrx. It s possble to wrte te lnear mxed model (LMM) n a sngle P m equaton by stackng te vectors and matrces approprately. Specfcally, let n = n = be te total number of observatons, y = (y y m ) te n-vector of observatons, X = (X X m ) te n p covarate matrx for te fxed effects, = dag ( m ) te n mq block dagonal covarate matrx for te random effects, b =(b b m ) te mq-vector of random effects, ffl =(ffl ffl m ) te n-vector of errors, and ff 2 D = ff 2 dag (K K) te mq mq block dagonal covarance matrx of b. Ten LMM can be wrtten as () y = Xf + b + ffl ψ b ffl! οn (ψ! ψ!) ff 2 D ff 2 I n Let f c (yjb fff 2 K) = (2ßff 2 ) n=2 exp Φ jy Xf bj 2 =(2ff 2 ) Ψ denote te condtonal densty functon of y gven b. Let p(bjk) = Π m j= p(b jjk) denote te jont densty functon of b. Te margnal lkelood f m (yjfff 2 K) of y can be wrtten as ρ (2) f m (yjfff 2 K) = (2ßff 2 ) n=2 jv j exp =2 2ff (y 2 Xf) V (y Xf) were V = D + I n. Suppose tat K (terefore V ) s known. Ten te maxmum lkelood estmators of f and ff 2 can be obtaned n a stragtforward manner (3) b f =(X V X) X V y bff 2 =(y X b f) V (y X b f)=n ff

2 Int. Statstcal Inst. Proc. 58t World Statstcal Congress, 2, Dubln (Sesson CPS53) p.579 Puttng b f and bff 2 nto (2) we ave f m (yjb f bff 2 K) = (2ßbff 2 ) n=2 jv j =2 e n=2 So te margnal Akake nformaton crteron and te fnte sample corrected verson take te followng respectve forms (4) (5) maic = n log(2ßbff 2 )+n + log jv j +2(p +) maic c = n log(2ßbff 2 )+n + log jv j +2n(p +)=(n p 2) were p s te dmenson of f. In practce K s often unknown. In ts case we may stll use (4) and (5) but wt K replaced by a consstent estmator K c (Srvastava and Kubokawa, 2). Te margnal Akake nformaton crtera maic and maic c can be vewed as estmators for te margnal Akake nformaton wc s defned as mai = 2E G(y) E G(w) log f m (6) = 2 wjb f bff 2 K c log f m wjb f bff 2 K c f m (wjfff 2 K)fm c (yjfff 2 K)dwdy c Vada and Blancard (25) argue tat wen te focus of te researc s on te clusters rater tan on te populaton, a more approprate nformaton measure sould be defned n a condtonal manner. Tey defne te condtonal Akake nformaton as (7) cai = 2E G(yb) E G(wjb) log f c (wjb b f b bff 2 K) c = 2 log f c (wjb b f b bff 2 K)g(wjb)g(y c b) dwdydb were g(y b) =g(yjb)p(b) s te true jont densty functon of y and b, g(wjb) s te condtonal densty of an ndependent future observaton w wc sares te same random effect b wt y wt dstrbuton p(b). In prncple b f, bff 2 and b b n (7) can be any estmators. Popular coces for b f and bff 2 are te maxmum lkelood estmators gven by (3), andb b s te emprcal Bayes estmator gven by (8) b b = E(bj b f bff 2 y) =D V (y X b f) Wen te ff 2 and K are known, an asymptotcally unbased estmator of cai s gven by (9) caic = 2logf c (yjb b b fff 2 c K)+2(ρ +) were b b s te emprcal Bayes estmator n (8), b f s te maxmum lkelood estmator, and ρ =tr(h ) s te effectve number of degrees of freedom for model (2) wt te form of (ψ! ψ X T X X T X T X ρ = tr T X T + D T X See Vada and Blancard (25). X T T!) 2 Bootstrap Informaton Crteron Isguro et al. (997) ave proposed an extenson of AIC based on te bootstrap. Tey call t te extended nformaton crteron (EIC). A salent advantage of te bootstrap metod s tat t uses massve teratve computer calculatons rater tan analytc expressons, so t s free from troublesome analytc dervaton of te bas correcton term moreover, t can be appled to almost any type of models and estmaton procedures under very weak assumpton. In te bootstrap metods, te true dstrbuton G(y) s substtuted by an emprcal dstrbuton functon b G(y). A random sample from b G(y) s called bootstrap sample, and s denoted as y = fy y N g. A statstcal model f (yjb ) s constructed based on te bootstrap sample y, wt b = (y b ). Te bootstrap verson of te expected log-lkelood can be rewrtten as () E bg(w) [log f (W jb )] = log f (wjb )d b G(w) = N NX log f (y j jb ) `N (yjb ) j=

3 Int. Statstcal Inst. Proc. 58t World Statstcal Congress, 2, Dubln (Sesson CPS53) p.58 Te bootstrap estmator of te expected log-lkelood s defned as follows () E bg (w) [log f (W jb )] = log f (wjb )d b G (w) = N NX log f (yj j b ) `N (y jb ) were b G (w) s te emprcal dstrbuton functon based on y. Te bootstrap bas estmate s (2) b = NE bg(y ) `N (y jb ) `N (yjb ) Extract B sets of bootstrap samples of sze N, and wrte te t bootstrap sample as y () =fy ()y N ()g. Te bas estmate n (2) s usually numercally approxmated by (3) b ß B = `N (y ()jb ()) `N (y()jb ()) b B were b () s an estmate of usng y (). Te extended nformaton crteron s defned as (4) EIC = 2 N`N (yjb ) b Kons and Ktagawa (996) ave gven a teoretcal justfcaton of EIC. 2. Varance Reducton n Bootstrap Smulaton Tere are two knds of errors n te bootstrap bas estmate b B. One s caused by te randomness of te observed data, te oter s te smulaton error wc decreases as te number of bootstrap replcaton ncreases. Kons and Ktagawa (996, 28, 2) consdered an effcent resamplng metod to reduce te second type of error. Let T ( ) = (T ( )T p ( )) T 2 R p be a p-dmensonal functonal estmator. Ten te dfference between () and () can be decomposed nto tree terms as follows (5) D(y G) = were»`n (yjb ) j= log f (wjb )dg(w) = D (y G) +D 2 (y G) +D 3 (y G) (6) D (y G) = `N (yjb ) `N (yjt (G)) D 2 (y G) = `N (yjt (G)) log f (wjt (G))dG(w) D 3 (y G) = log f (wjt (G))dG(w) log f (wjb )dg(w) wt b beng te functonal estmator suc tat b = T ( G). b By takng te expectaton and te varance term by term on te rgt-and sde of (6), we observe tat te expectaton of D 2 (y G) s zero, so E G(y) [D(y G)] = E G(y) [D (y G) +D 3 (y G)] and Var[D(y G)] = O(n ) and n contrast Var[D (y G) +D 3 (y G)] = O(n 2 ) Terefore, for te bootstrap estmate we ave (7) E bg(y ) D(y G) b = E bg(y ) D (y G)+D3 b (y G) b were D (y G)=`N b (y jb ) `N (y jb ) and D3 (y G)=`N b (yjb ) `N (yjb ) To reduce te fluctuaton n te bootstrap bas estmaton of log-lkelood, we use te followng formula as a bootstrap bas estmate o nd (y () G)+D3 b (y () G) b (8) b B = B =

4 Int. Statstcal Inst. Proc. 58t World Statstcal Congress, 2, Dubln (Sesson CPS53) p.58 3 Bootstrap Informaton Crtera for te Lnear Mxed Models 3. Margnal extended nformaton crteron meic Snce te maxmzed margnal lkelood can be calculated stragtforwardly by puttngb f and bff 2 nto (2), we focus on te constructon of te bootstrap bas estmator. Because K s often unknown, we just consder te case for unknown K. By usng te maxmzed margnal lkelood, te bas of te margnal lkelood s gven by 2 X (9) b m = E G(y) 4 m 3 log f m (y j jb f bff 2 K) c E G(w) log f m (wjb f bff 2 K) c 5 j= In applyng te bootstrap metod, te second term n b m, namely te expected log-lkelood E G(w) log f m (wjb f bff 2 K) c s replaced by E bg(w) log f m (wjb f bff 2 K c ) were w s an n-dmensonal future varable sarng te same random effects wt y, and te densty functon of b G(w) s gven by (2) g m (wjb f bff 2 c K)= (2ßbff 2 ) n=2 j b V j =2 exp ρ 2bff 2 (w X b f) b V (w X b f) ff Furter te expectaton E G(y) s replaced by te bootstrap expectaton E eg(y ) of te bootstrap sample y, wc s defned as follows (2) y = X b f + b + ffl ψ! (ψ b οn ffl! ψ!) bff 2 D b bff 2 I n Consequently, te bootstrap estmate of b m s gven by (22) b m = E G(y e ) log f m (y jb f bff 2 K c ) E bg(w) log f m (wjb f bff 2 K c ) were b f = b f(y ) bff 2 = bff(y ) 2 c K = c K(y ). Usng ts bootstrap estmate of bas, we defne te bootstrap verson of te margnal nformaton crteron n LMM as follows (23) meic = 2 log f m (yjb f bff 2 c K) b m Furtermore, let us use te varance reducton metod to reduce te fluctuaton n te bootstrap bas estmaton of margnal log-lkelood. (24) b m ß B were = fd m (y ()) + D m3 (y ())g D m (y ()) = log f m (y jb f bff 2 K c ) log f m (y jb f bff 2 K) c log f m (wjb f bff 2 K) c E bg(w) log f m (wjb f bff 2 K c ) D m3 (y ()) = E bg(w) 3.2 Condtonal extended nformaton crteron ceic Wen we consder te bootstrap bas estmator for te condtonal lkelood, n addton to te estmators bfbff 2 and c K, we also need to estmate te random effect b, wc s usually estmated by te emprcal Bayes metod. Te procedure to construct a condtonal bootstrap nformaton crteron n LMM s smlar to tat for constructng meic. We use te maxmzed condtonal log-lkelood to get te bas of te condtonal lkelood as follows b c = E G(yb) log f c (yjb b b f bff 2 c K) EG(wjb) log f c (wjb b b f bff 2 c K)

5 Int. Statstcal Inst. Proc. 58t World Statstcal Congress, 2, Dubln (Sesson CPS53) p.582 In applyng te bootstrap metod, smlar to te case n constructng te margnal nformaton crteron, te expected condtonal log-lkelood, E G(wjb) log f c (wjb b f b bff 2 K) c s replaced by te condtonal expectaton E bgc log f c (wjb b f b bff 2 K c ), were b Gc as densty functon (25) g c (wjb b b f bff 2 K) =(2ßbff 2 ) n=2 exp n o jy X f b b bj 2 =(2bff 2 ) In te defnton of b c, te jont dstrbuton G(y b) of y and b s replaced by te bootstrap dstrbuton eg(y b ) of y and b usng (2). Ten we defne te bootstrap bas estmate b c of te condtonal loglkelood as follows (26) b c = E e G(y b ) log f c (y jb b f b bff 2 K d ) E bgc(w) log f c (wjb b f b bff 2 K c ) So we defne te bootstrap verson of te condtonal nformaton crteron n LMM n te form of (27) ceic = 2 log f c (yjb b b f bff 2 c K) b c As n te margnal case, to reduce te fluctuaton n te bootstrap bas estmaton of condtonal loglkelood, we also use te varance reducton metod (28) b c ß B = fd c (y ()) + D c3 (y ())g were D c (y ()) = log f c (y jb b f b bff d 2 K ) log f c (y jb b f b bff 2 K) c D c3 (y ()) = E bgc log f c (wjb b f b bff 2 K) c E bgc log f c (wjb b f b bff 2 K c ) 3.3 Hger-order bas corrected nformaton crtera In ts secton, we focus on te second-step bootstrap bas corrected estmators of te log-lkelood and propose second-order bootstrap bas corrected nformaton crtera. By now we ave avalable four types of bas estmators, n wc te bas estmators n maic and caic can be calculated analytcally wle te bas estmators n meic and ceic can only be calculated numercally. In maic, from (4) we know tat te frst-order bas estmate of te margnal log-lkelood s p +, so te second-order bootstrap bas estmate of te margnal log-lkelood n estmatng te expected margnal log-lkelood s gven by (29) n b 2m = E G(y e ) log f m (y jb f bff 2 K c ) (p +) E bg(w) log f m (wjb f bff 2 K c ) resultng te second-order bootstrap nformaton crteron (3) maic 2 = n log(2ßbff 2 )+n +logj b V j +2 p ++ n b 2m Wle n caic, from equaton (9) we know tat te frst-order bas estmate of te condtonal log-lkelood s gven by ρ +, so te second-order bootstrap bas estmate of te condtonal log-lkelood s gven by (3) n b 2c = E e G(y b ) log f c (y jb b b f bff 2 c K ) (ρ +) E bgc(w) gvng te second-order condtonal bootstrap nformaton crteron (32) caic 2 = 2 log f c (yjb b b f bff 2 c K)+2 ρ ++ n b 2c log f c (wjb b b f bff 2 c K )

6 Int. Statstcal Inst. Proc. 58t World Statstcal Congress, 2, Dubln (Sesson CPS53) p.583 Alternatvely, as n defnng meic and ceic, we may frst use te frst-order bas correcton of te loglkelood numercally by bootstrap metods. Let b m be defned as n (22). We estmate te second-order bas of te margnal log-lkelood as follows (33) n b 2m = E e G(y ) log f m (y jb f bff 2 c K ) b m E b G(w) log f m (wjb f bff 2 c K ) gvng te correspondng bootstrap nformaton crteron as (34) meic 2 = 2 log f m (yjb f bff 2 c K) b m n b 2m Smlarly to get second-order verson of ceic, we use te frst-order bootstrap bas estmate b c of te condtonal log-lkelood defned by (26). Te second-order bootstrap bas estmate of te condtonal log-lkelood s ten gven by (35) n b 2c = E e G(y b ) log f c (y jb b f b bff 2 K c ) b c E bgc(w) log f c (wjb b f b bff 2 K c ) Tus we may defne te second-order verson of ceic as follows (36) ceic 2 = 2 log f c (yjb b f b bff 2 K) c b c n b 2c References [] Greven, S. and Kneb, T. (2). On te beavour of margnal and condtonal AIC n lnear mxed models. Bometrka, 97, -7. [2] Isguro, M., Sakamoto, Y., and Ktagawa, G.(997). Bootstrappng log- lkelood and EIC, an extenson of AIC. Annals of te Insttute of Statstcal Matematcs, 49, [3] Kons, S. and Ktagawa, G. (996). Generalzed nformaton crtera n model selecton. Bometrka, 83, [4] Kons, S. and Ktagawa, G. (23). Asymptotc teory for nformaton crtera n model selecton-functonal approac. Journal of Statstcal Plannng and Inference, 4, [5] Kons, S., and Ktagawa, G. (28). Informaton Crtera and Statstcal Modelng, Sprnger New York. [6] Ktagawa, G. and Kons, S. (2). Bas and varance reducton tecnques for bootstrap nformaton crtera. Ann. Inst. Stat. Mat., 62, [7] Lard, N. M. and Ware, J. H. (982). Random-effects models for longtudnal data, Bometrcs, 38, [8] Lang, H., Wu, H. and ou, G. (28). A note on condtonal AIC for lnear mxed-effects models. Bometrka, 95, [9] Ngo, L. and Brand, R. (22). Model selecton n lnear mxed effects models usng SAS Proc Mxed. SUGI 22. [] Pnero, J. C. and Bates, D. M. (2). Mxed-Effects Models n S and S-PLUS, New York Sprnger. [] Srvastava M. S. and Kubokawa, T. (2). Condtonal nformaton crtera for selectng varables n lnear mxed models. Journal of Multvarate Analyss,, [2] Vada, F. and Blancard, S. (25). Condtonal Akake nformaton for mxed-effects models. Bometrka, 92, RÉSUMÉ (ABSTRACT) In ts paper we propose bootstrap nformaton crtera for te lnear mxed model. Tese nformaton crtera are constructed usng eter te margnal log-lkelood or te condtonal log-lkelood. Secondorder bas-corrected nformaton crtera are also consdered.