Selection Strategy for Covariance Structure of Random Effects in Linear Mixed-effects Models

Transcription

1 Scandinavian Journal of Statitic doi: /jo.1179 Publihed by Wiley Publihing Ltd. Selection Strategy for Covariance Structure of Random Effect in Linear Mixed-effect Model XINYU ZHANG Chinee Academy of Science HUA LIANG George Wahington Univerity ANNA LIU Univerity of Maachuett DAVID RUPPERT Cornell Univerity GUOHUA ZOU Chinee Academy of Science and Caital Normal Univerity ABSTRACT. Linear mixed-effect model are a owerful tool for modelling longitudinal data and are widely ued in ractice. For a given et of covariate in a linear mixed-effect model, electing the covariance tructure of random effect i an imortant roblem. In thi aer, we develo a joint likelihood-baed election criterion. Our criterion i the aroximately unbiaed etimator of the exected Kullback Leibler information. Thi criterion i alo aymtotically otimal in the ene that for large amle, etimate baed on the covariance matrix elected by the criterion minimize the aroximate Kullback Leibler information. Finite amle erformance of the rooed method i aeed by imulation exeriment. A an illutration, the criterion i alied to a data et from an AIDS clinical trial. Key word: aymtotic otimality, covariance tructure, Kullback Leibler information, longitudinal data 1. Introduction In clinical trial, a well a other biological and biomedical alication, ubject are often meaured reeatedly over a given eriod. Meaurement obtained from one ubject are generally correlated, while meaurement obtained from different ubject can be indeendent. One ueful tool for uch longitudinal data i mixed-effect modelling, in which within-ubject and between-ubject variation are both conidered. Linear mixed-effect (LME) model (Laird & Ware, 198) are a owerful technique for the analyi of longitudinal data and have been tudied and alied widely (Pinheiro & Bate, ; Verbeke & Molenbergh, 9; Voneh & Chinchilli, 1996). A in conventional linear regreion, tatitical analyi of longitudinal data involve model election, and electing the mot deirable model i tyically the firt roblem encountered in data analyi. The Akaike information criteria (AIC) (and ome other related criteria uch a Bayeian information criterion (BIC)) have been ued for model election in LME model (Ngo & Brand, ; Pinheiro & Bate, ). However, it i well known that for mall amle, AIC can be highly biaed in etimating exected Kullback Leibler information under linear model (Hurvich & Tai, 1989), and imilar biae of AIC have been found in LME model when the marginal likelihood function i ued (Greven & Kneib, 1). To reduce thee biae, a natural idea i to aly Hurvich & Tai (1989) AIC c to the LME model by conidering the marginal likelihood function, that i, the marginal denity of the oberved

2 X. Zhang et al. Scand J Statit data with the unoberved random effect integrated out. Thi lead one to ue the marginal AIC c in the LME model election. The marginal AIC (maic) and marginal AIC c generally aly to the cae of a oulation focu where interet lie redominantly in etimation of the oulation arameter, that i, the fixed effect and variance comonent. On the other hand, for the LME model, we are often intereted in rediction of the cluter-ecific random effect, not jut the oulation arameter (Vaida & Blanchard, 5). Vaida & Blanchard (5) oberved in the analyi of a cadralazine tudy that had a cluter focu; the maic and marginal AIC c are not aroriate for electing an LME model. Intead, Vaida & Blanchard (5) rooed uing the conditional likelihood function, given the random effect, which lead to the conditional AIC (caic) and conditional AIC c ; Liang et al. (8), Greven & Kneib (1) and Yu & Yau (1) alo tudy caic. Both the fixed effect and random effect are etimated/redicted by the emirical bet linear unbiaed rediction (BLUP), which are the BLUP with the unknown covariance matrix of the random effect relaced by it etimate. Therefore, a good etimate for thi matrix i imortant both for a oulation focu and for a cluter focu. In thi aer, we are concerned with election of a model for thi covariance matrix. For election of thi model, the caic (or conditional AIC c ) i not aroriate, becaue the conditional likelihood doe not exlicitly deend on the variance comonent, a mentioned by Vaida & Blanchard (5). Marginal likelihood may be ued for identifying the covariance matrix of the random effect, becaue thi matrix i art of the marginal covariance matrix of the reone. However, a dicued by Vaida & Blanchard (5), marginal likelihood i not aroriate for a cluter focu; thi oint i dicued further at the end of Section.1. For identifying the covariance tructure of the random effect, we find that the joint likelihood of the data and the random effect i an aroriate tool. We develo a election criterion baed on the joint likelihood and call it joint AIC (jaic). Becaue the joint denity i the roduct of the conditional denity of the data given the random effect and the marginal denity of the latter, jaic ue the ame conditional likelihood that i the bai of caic, and o jaic retain the cluter focu of caic. Moreover, becaue jaic include the marginal denity of the random effect, it i better than caic at electing a model for the random effect. In our imulation tudy, both maic and jaic greatly outerform caic when electing the covariance model. To develo our election criterion, we utilize a imilar method to that of Hurvich & Tai (1989) linear model criterion, that i, deriving an (aroximately) unbiaed etimator of the exected Kullback Leibler information. Technically, our derivation i by no mean traightforward for the following reaon: (i) in Hurvich and Tai aroach, the (aroximate) ditribution of the reidual um of quare (RSS) and the variance etimator hould be known. For the LME model, uch ditribution may not be eay to find, becaue cloed form for variance etimator are not available. (ii) The RSS and the variance etimator are required to be indeendent in Hurvich and Tai aroach. Thi cannot be true for the LME model. Thi aer i organized a follow. In Section, we derive aroximately unbiaed etimator of the exected Kullback Leibler information and then rooe a jaic for the election of the covariance matrix of the random effect. We rove aymtotic otimality of jaic in the ene that the etimator baed on the elected covariance matrix that minimize our rooed criterion alo minimize aroximate Kullback Leibler information in large amle. In Section 3, we reent Monte Carlo imulation reult to illutrate the erformance of the rooed criterion. We aly our criterion to a real data et in Section 4. Some dicuion are given in Section 5. The detail of derivation and roof are reented in Aendix and Suorting Information.

3 Scand J Statit Selection trategy for covariance 3. Joint AIC for LME model.1. The LME model The general form of the LME model i y i D X i ˇ C Z i b i C " i ;id 1;:::;n with b i N.; D/ and " i N.; I mi /, where ˇ i a 1 vector of fixed regreion coefficient; b i i a k 1 vector of random coefficient ecific to the ubject i; y i D.y i1 ;:::;y imi /, X i.m i / and Z i.m i k/ denote the reone variable, known covariate matrice for the fixed and random effect of full column rank, reectively, of the ith ubject; " i.m i 1/ i an error vector indeendent of b i ; i an unknown arameter; and I mi i an m i m i identity matrix. The covariance matrix D may have ecial tructure. Let N D P n id1 m i be the total number of obervation and be the vector of arameter in the model, including ˇ, and the arameter in D. Clearly, the reviou LME model can be written a y D Xˇ C Zb C "; b N.; G/; (1) where y i an N 1 vector of obervation, X D.X 1 ;:::;X n / i an N matrix of rank, Z D diag.z 1 ;:::;Z n / i an N r block-diagonal matrix of rank r D nk, b D.b 1 ;:::;b n /, " D." 1 ;:::;" n / and G D diag.d;:::;d/ i an r r block-diagonal matrix. Suoe that the data y come from the following true LME model: y D Xˇ C Z b C " ; b N.; G /; " N.; I N /; () where ˇ i a 1 vector of fixed effect, b i an r 1 vector of random effect, X and Z are the N and N r matrice, reectively, " i the N 1 diturbance, and G D diag.d ;:::;D / i an r r block-diagonal matrix. In alication, we ue the model (1) to fit the oberved data y. The uroe of thi aer i to chooe the aroriate matrix G for random effect given covariate matrice X and Z. The etu of model (1) and () imlie that when we conider the choice of covariance tructure, the fitting covariate matrice X and Z are allowed to be different from the true covariate matrice X and Z, which i a more ractical and general framework. Let D I N C ZGZ. Under model (1), for given G and, the maximum likelihood (ML) etimator of ˇ and b aregivenby bˇ D.X 1 X/ 1 X 1 y (3) and bb D GZ 1.y Xbˇ/; (4) reectively (e.g. Laird & Ware, 198). The unknown arameter in G and can be etimated by uing the ML or retricted ML method (Davidian & Giltinan, 1995). Denote their etimator by b G and. In the following content, when uing bˇ and bb, b G and have been lugged in. Let b be the etimator of.

4 4 X. Zhang et al. Scand J Statit.. Joint Akaike information and election criterion For avoiding the imact of rearameterization on the criterion that we will rooe, 1 we firt cale the random effect b by defining b D G 1= b. Let b D G 1= b, and then b i a vector of indeendent tandard normal variable. A ome tudie on electing random effect uch a Chen & Dunon (3) and Kinney & Dunon (7), we focu on b intead of b when conidering likelihood function. Denote the denity of y under the true model by () by f true.y; b jˇ; G /, the conditional denity function of y by f.yj; b / and the denity of b by.b j/. Rewrite the etimator b, G b and bb a b.y/, G.y/ b andbb.y/, reectively, and letbb D bb.y/ D G.y/ b 1= bb.y/ and b D G 1= b. Then we define the joint Akaike information a follow. Definition 1. The joint Akaike information (or exected Kullback Leibler information) i defined to be h jai D E y;b f true.y; b jˇ; G / E y;b E ey; log.f eb. ey jb.y/;bb.y//.bb ±i.y/jb.y/// ; where ey and eb have the ame ditribution a y and b, and are indeendent of y and b, reectively. Here, we conider the dicreancy between the joint denity function of the data and random effect (which are, of coure, unobervable) under the candidate and true model. Clearly, if we invetigate the deviation of the likelihood under the candidate model from that under the true model by making ue of the marginal or conditional denity function, then the maic or caic will be reulted in. Thi quantity ha been ued in literature (e.g. Li et al., ; Wu & Zhang, 6,. 19). Uing model (1), the ditinction between cluter and oulation focu can be tated more analytically. The cluter focu catured by caic i on the model Xˇ C Zb for the conditional mean of y. The oulation focu catured by maic i on the marginal mean Xˇ and marginal covariance matrix D I N C ZGZ of y. In contrat, jaic focue on the conditional mean of y and the marginal covariance matrix of b. Thu, jaic meaure both the goodne-of-fit of Xˇ C Zb to y and how well the covariance matrix of b i fit by G, while caic doe only the former. One remie of thi aer i that it i often bet to meaure the goodne-of-fit of the conditional, rather than the marginal, model for the mean of y; therefore, we refer jaic to maic. Another remie i that one hould not neglect the model for the ditribution of the random effect, which lead u to refer jaic to caic. Define Z D Z b G 1=.WeueE to rereent E y;b for imlicity. Noting that we obtain log f..eyjb.y/;bb.y//.bb.y/jb.y/// D N log./ N log 1 b key X bˇ Z bb k r log./ 1. bb / bb D N log./ N log 1 key X bˇ Zbbk r log./ 1 b b b G 1 bb log g b.ey;bb/; 1 If we do not cale the random effect, there will be an additional term log j b Gj in our criterion, which i related to rearameterization of Z (i.e., relace Z by multile of Z).

5 Scand J Statit Selection trategy for covariance 5 E f true.y; b jˇ; G / he ey; eb log f.eyjb.y/;bb.y//.bb ±i.y/jb.y// D N log C 1 ± kx ˇ Xbˇ Zbbk C tr.z G Z / C N Cbb b G 1 bb C.r r / log./ N r.b;bb/; where kak D a a and the notation tr mean taking trace. It i een that jai D E.b;bb/.For the convenience in the ubequent analyi, we rewrite.b;bb/ a.b;bb/ D N log C 1 kx ˇ C Z b Xbˇ Zbbk C 1 tr.z G Z / b Z Z b b Z.X ˇ Xbˇ Zbb/ ± (5) C N C bb b G 1 bb C.r r / log./ N r : Oberving that E Œb Z Z b C b Z.X ˇ Xbˇ.ey/ Zbb.ey// D tr.z G Z /, we ee that under the true model, the right-hand ide of (5) can be aroximated by N log C 1 b kx ˇ C Z b Xbˇ Zbbk C N C bb b G 1 bb C.r r / log./ N r.b;bb/: (6) Therefore, a reaonable meaure rereenting the dicreancy between the candidate and true model would be E.b;bb/. Next, we derive an (aroximately) unbiaed etimator of E.b;bb/ becaue uch an etimator can be ued to define a feaible election criterion for the covariance tructure of random effect. Define b D Xbˇ C Zbb, ˆ.y/ D b =@y C.b /=@y,.y/ D /=.@y@y / and jaic D log g b.y;bb/c ˆ.y/C 4.y/ N log N log N r r log : (7) From derivation in Aendix A.1, we know jaic i an unbiaed etimator of E.b;bb/, that i, E.b;bb/ D E jaic : (8) Note that jaic till deend on unknown. In Section S.1 of Suorting Information, we derive an exactly unbiaed etimator of under ome condition. Here, for imlicity, we etimate by. In Section S.3 of Suorting Information, we can ee that the election reult by uing and are very imilar. Thu, we rooe to elect the bet covariance matrix of random effect that minimize the following quantity: jaic D log g b.y;bb/ C ˆ.y/ C b 4.y/ D log g b.y;bb/ C.y/ C 1.y/; (9) where.y/ D =@y and 1.y/ D.b /=@y C b 4 /=.@y@y / : It i intereting to oberve that the exectation of.y/ in (9), conditional on b, i jut the generalized degree of freedom defined by Ye (1998) for the LME model. Alo, the enalty term.y/ i exactly the ame a that in caic with the known error variance (Liang et al.,

6 6 X. Zhang et al. Scand J Statit 8). The third term, 1.y/ in (9), i an extra enalty due to the variability of etimating the unknown error variance. Comared with the caic of Liang et al. (8) and Greven & Kneib (1), our jaic utilize joint likelihood function that contain the ditribution information of the random effect b, but the caic doe not directly deend on the ditribution of b (of coure, when etimating, the ditribution information of b i ued) and thu the etimated covariance bg. A a reult, a new term bb b G 1 bb that meaure the goodne-of-fit related to the random effect aear in the jaic. In addition, our imulation reult in Section 3 alo how that the caic i not aroriate in electing covariance tructure of random effect..3. Aymtotic otimality Our foregoing dicuion focue on the finite amle jutification of the rooed criterion. We now conider the large amle aymtotic otimality of our aroach. Noting that.b;bb/ can be regarded a aroximate Kullback Leibler information between the candidate model and true model, in thi ubection, we will illutrate that the etimator baed on the elected covariance tructure of random effect that minimize our rooed criterion alo minimize.b;bb/ in large amle. Let ¹1;:::;Sº be the index et denoting the candidate LME model that have the ame form a (1) but different covariance tructure of random effect, and D, G, and be the value of D, G, and under the th candidate model, reectively. We further write bb and b a the correonding verion of bb and b, reectively. Denote! a the unknown arameter vector in G, and b! and b a the etimator of! and, reectively. Let D.! ; / with finite J element, and b D.b! ;b / that i obviouly a art of b. Denote b D.b / D b I N CZG.b! /Z D b I N CZG b Z, bv D b 1= X.X b 1 X/ 1 X b 1=, and bp D I N b b 1=.I N bv /b 1=. Then from (3) and (4), we oberve that under the th candidate model, the etimator of i given by b D Xbˇ C Zbb D bp y: Recalling (6) and (9), the value of.b;bb/ and jaic under the th candidate model can be written a and. b ;bb / D N c C log c! 1 C b kx ˇ C Z b b k Cbb b G 1 b b C.r r / log./ r jaic D N log./ C N log c C b.y @y@y C b ky b k C c ; C r log Cbb b G 1 b reectively. Letb D arg min jaic ; the model elected by minimizing the criterion jaic. Under ome ¹1;:::;Sº reaonable condition, the following theorem how the aymtotic otimality of the jaic in the ene of making. b ;bb / a mall a oible. Theorem 1. Under condition 1 8 in Aendix A., we have min ¹1;:::;Sº b. b b ;bb b /. b ;bb / ± The roof i given in Aendix A.. (1) (11)! 1 a N!1: (1)

7 Scand J Statit Selection trategy for covariance 7.4. Imlementation of jaic In fact, the enalty term in jaic,.y/ C 1.y/, i the ame a ˆ1 of Greven & Kneib (1) excet that e i relaced by it etimate (ee formula (9) of Greven & Kneib, 1). When i known, the enalty term can be imlified to.y/, that i, ˆ in Liang et al. (8) and Greven & Kneib (1). But the calculation of ˆ and ˆ1 require additional N and N model fit, reectively, which lead to large calculation burden. Exhilaratingly, Greven & Kneib (1) develo an analytic rereentation of ˆ (ee their theorem 3) and rovide an R ackage for imlementation. They alo how the cloe agreement between uing ˆ1 and ˆ C 1 for model election by imulation. Therefore, for the imlementation of jaic to be convenient, we rooe to ue jaic D log g b.y;bb/ C ˆ; (13) where ˆ D b C P j D1 e b B 1b j G ba bw ;j ba y and the definition of b,, e j, bb, G b, ba and bw ;j can be found in theorem 3 of Greven & Kneib (1). 3. Simulation tudy In thi ection, we invetigate the finite amle erformance of the rooed rocedure and comare it with ome exiting method by Monte Carlo imulation. We imulate data from the following LME model: y ij D.1; t j /ˇ C.1; t j /b i C " ij ; i D 1;:::;1; j D 1;:::;; with b i N.; D/, t j D 5.j 1/,.ˇ;ˇ1/ D.3; :/, and " ij are normally ditributed with mean and variance :5. That i, we have 1 ubject with a oitive definite covariance matrix D. In the imulation, we vary a comlexity arameter ( or ); when thi arameter increae from it baeline value, the tructure of D diverge from a imle model. A in Greven & Kneib (1), we reort the robability of electing the more comlex model by the variou criteria. The quared Frobeniu norm difference between the true and etimated covariance matrice, which are defined by tr¹.d bd/.d bd/º, are alo reented. Four cae are conidered a follow. Cae I: In thi cae, the choice i between a multile of the identity! tructure ( D 1) and a general diagonal tructure ( >1)forD, where D D :6. We fit the imulated data :6 uing the R function lme with the otion that the random effect covariance matrix would be multile of the identity or a diagonal matrix. We then obtain the maic and the marginal BIC (mbic) value under thee two covariance tructure. Secifically, the maic and mbic are defined by loglik C and loglik C log.n /, reectively, where loglik and are the marginal log-likelihood and the number of unknown arameter in the th candidate model, reectively. Additionally, we alo include the adjuted BIC (abic) rooed by Delattre et al. (14) in thi imulation, which i defined by loglik C log.n/ R; C log.n / F;, where F; i the number of fixed effect and R; i the um of the number of random effect and the number of unknown arameter in D. Baed on 1 relication, we comute the roortion electing the diagonal tructure and the average Frobeniu norm difference between the true and etimated covariance matrix. The reult are reented in Fig. 1A and B. Firt, when D 1, which mean that D i exactly a multile of the identity, mbic lead to the mallet roortion of electing the diagonal tructure and all AIC correond to about.3 etimated robability of wrongly chooing the diagonal tructure. A the value of increae

8 8 X. Zhang et al. Scand J Statit Selection frequency Selection frequency Selection frequency Selection frequency 1 (A).5 jaic caic maic mbic abic ν 1.5 (C) ν 1.5 (E) δ 1.5 (G) δ Squared Forbeniu norm Squared Forbeniu norm Squared Forbeniu norm Squared Forbeniu norm 4 (B) ν 4 (D) ν (F) δ (H) δ Fig. 1. Left column: roortion of 1 relication where the more comlex model i elected. Right column: the value of the average Frobeniu ditance between etimated and true covariance matrice. Joint Akaike information criteria (jaic): olid line; conditional AIC (caic): dahed line; marginal AIC (maic): dotted line; marginal BIC (mbic): dahed-dotted line; and adjuted BIC (abic): olid line with +. Cae I IV are in row 1 4, reectively. from 1, that i, the tructure of D move away from being a multile of the identity to a general diagonal tructure, the robability of each criterion chooing a diagonal tructure increae a well, excet that caic lead firt to a decreaing robability before increaing. However, caic and mbic have relatively low robabilitie of electing the general diagonal tructure when

9 Scand J Statit Selection trategy for covariance 9 >1; their robabilitie are even le than.5 when 4, which imlie very different diagonal element. It i worth noting that the robability of electing the general diagonal tructure i alway omewhat higher for jaic than for maic, which mean that jaic tend to favour a more comlex tructure, but the maic tend to elect a imler tructure. Thi erformance can be artly exlained by the finding in theorem 1 of Greven & Kneib (1) that maic i not an aymtotically unbiaed etimator of the Akaike information and favour maller model without random effect. abic erform between maic and mbic and cloely to maic. Figure 1B how that when i mall, all election criteria etimate D with imilar accuracy, but when i larger, jaic erform the bet. In a high roortion of circumtance, jaic i found to be the referred criterion. Cae II: We elect between comound-ymmetry (! D 1) and general oitive definite (general PD) ( >1) tructure of D. We et D D :6 :4 :4 :6. The reult hown in Fig. 1C and D are imilar to thoe in Cae I. Cae III: In thi cae, we elect between a multile! of the identity tructure and a comoundymmetry tructure of D. We et D D :6, o the correlation coefficient ı D =:6. :6 Figure 1E dilay the roortion of electing comound-ymmetry tructure of all criteria. When ı i very mall (ay < :1), which mean that the multile of the identity tructure i aroriate, the mbic and caic obviouly uort thi tructure (the etimated robability of electing it i above.9), and jaic, maic and abic lead to omewhat maller robabilitie (about.7.8). When ı i big (ay.6), that i, the comound-ymmetry tructure i more aroriate, jaic lead to the highet etimated robability of electing the comoundymmetry tructure, while the robabilitie for caic and mbic are till lower than.5. Figure 1F how that when ı i mall (ay < :3), the criteria that favour a imler tructure rovide better etimate of the covariance matrix. In contrat, when ı i moderate to large (ay > :4), the criteria that favour a more comlex tructure rovide better etimate of the covariance matrix.! Cae IV: We elect between diagonal and general PD tructure of D. We et D D :6. :3 The correlation coefficient ı D = :18. The reult hown in Fig. 1G and H are imilar to thoe in Cae III. In concluion, jaic and maic are referred to caic in election for covariance tructure. Comared with maic, mbic and abic, jaic tend to favour a more comlex tructure regardle of the value of or ı; a a reult, jaic lead to a better etimate of covariance matrix of the random effect when or ı i large, but a wore etimate when or ı i mall. In mot ituation we conidered, jaic erform bet. All reult reented in Fig. 1 are baed on ML etimation. The reult baed on retricted ML etimation are imilar to thoe we have hown and omitted to ave ace but available uon requet from the author. 4. Examle: decay rate of viral reone An undertanding of the athogenei of HIV infection lay an imortant role in the evaluation of antiviral theraie for AIDS/HIV. Recent reearch indicate that the decay rate of the firt hae of the viral reone (number of coie of HIV RNA in the lama or viral load)

10 1 X. Zhang et al. Scand J Statit i a ueful marker for antiviral otency (Wei et al., 1995; Ho et al., 1995). LME ha become a tandard tool for etimation of the firt decay rate (Wu & Ding, 1999). In thi ection, we reent an analyi of a ubet of an AIDS clinical trial grou (ACTG 315) tudy. In thi tudy, viral load wa cheduled to be meaured on day, 7, 1, 14, 1 and 8 and week 8, 1, 4 and 48 after initiation of an antiviral theray. We ued the data from the firt week, becaue week i the time when the econd decay aear. Our data et i comried of 36 atient, with the number of obervation er atient varying from 3 to 5. We reent the catterlot of thee obervation in Fig.. See Lederman et al. (1998) for the detail about thi tudy. We now fit the following model to the data: y ij D.1; t j /ˇ C.1; t j /b i C " ij ; where i D 1;:::;36, j D 1;:::;m i and y ij i the viral load (log cale) of atient i at meaurement time t ij. We conider the following four covariance tructure: (a) multile of an identity, (b) diagonal, (c) general PD, (d) comound-ymmetry, becaue the ackage nlme in R imlement for thee four commonly ued tructure. All of the AIC and BIC criteria difavour tructure (a) and (d) but give ambiguou reult for tructure (b) and (c). Therefore, in what follow, we reent the detail for tructure (b) and (c). It i een in Table 1 that caic refer to general PD tructure. On the other hand, the reult baed on other criteria indicate 14 1 viral load (coie/ml) Time (day) 1 Fig.. The catter lot of viral load (log cale) againt time for 36 AIDS atient from the AIDS Clinical Trial Grou 315 tudy. Table 1. The AIC and BIC value of the variou criteria for ACTG 315 data Structure abic mbic maic caic jaic Diagonal General PD AIC, Akaike information criteria; BIC, Bayeian information criterion; ACTG, AIDS Clinical Trial Grou; abic, adjuted BIC; mbic, marginal BIC; maic, marginal AIC; caic, conditional AIC; jaic, joint AIC; PD, oitive definite.

11 Scand J Statit Selection trategy for covariance 11 Table. The etimated value and e of the intercet and loe uing LME with the diagonal and general PD covariance tructure for ACTG 315 data Structure Intercet (e) Sloe (e) Mean rediction error (e) Diagonal (.189).91(.117).6518 (.844) General PD (.1).916(.14).798 (.149) e, tandard error; LME, linear mixed-effect; PD, oitive definite; ACTG, AIDS Clinical Trial Grou. general PD Diagonal Fig. 3. The random etimate of the intercet (left anel) and loe (right anel) with the diagonal (+) and general oitive definite (PD) (ı) covariance matrice from a linear mixed-effect fit of the AIDS data. The number in the left margin are id of atient. that the diagonal tructure i referable. Recalling the imulation erformance that how that the caic i not aroriate in election of covariance tructure of random effect, we ugget that the diagonal tructure be elected. Now let u ee the etimated value of the intercet and loe along thee two tructure, which we reent in the middle two column of Table. The etimated value with thee two tructure are imilar, but the tandard error baed on the diagonal tructure are maller than thoe baed on the general PD tructure, which indicate that the fixed etimate baed on the diagonal covariance matrix are more efficient than thoe baed on the general PD tructure. Furthermore, it i intereting to oberve that the redictor of random intercet and loe baed on the diagonal tructure are motly cloer to zero than thoe baed on the general PD tructure (hown in Fig. 3). Lat, we examine the redictive ower of the two model with diagonal and general PD random effect covariance matrice, reectively. Secifically, we exclude the lat obervation of each atient from the original amle a the teting amle and do etimation baed on the left obervation. The lat column of Table how the mean rediction error over the teting amle and their tandard error, from which we ee that the model with diagonal tructure ha more recie rediction than the model with general PD tructure. Thi uort the choice of diagonal tructure in thi examle again.

12 1 X. Zhang et al. Scand J Statit 5. Dicuion and ummary To obtain a more aroriate election criterion for the covariance tructure of random effect in the LME model, we have generalized Hurvich & Tai (1989) aroach and develoed a new criterion jaic baed on the joint likelihood of data and random effect. The jaic take the variance comonent of the random effect fully into account and can well hel ditinguih between model with different covariance tructure of the random effect. Our criterion i nearly unbiaed for etimating the exected Kullback Leibler information and ha aymtotic otimality. The rooed method ha alo been hown to be romiing by a imulation tudy. In that tudy, we found that jaic i much better than caic at electing a model for the covariance matrix of the random effect. It i worth noting that the jaic i develoed to elect tructure of random effect but cannot be ued to tet the ignificance of random effect. A in Liang et al. (8), we made ue of the integration by art technique, which ha been utilized to obtain rik-unbiaed etimator before (Lu & Berger, 1989; Stein, 1981), to derive the election criterion for the covariance matrix of random effect. We exect that our method i alicable to model election in other context uch a non-arametric regreion (Hurvich et al., 1998) and ingle-index model (Naik & Tai, 1). For generalized mixed-effect model, the jaic can be develoed uing the technique of Saefken et al. (14). Thee warrant our future reearch. Variable election for the LME model i alo an imortant toic. When focuing on the choice of random effect, the jaic can be regarded a the caic with the additional enalty term bb b G 1 bb and thu hould have the tendency to chooe model with le random effect than the caic. Note that Greven & Kneib (1) mentioned that maic chooe to incororate random effect rarely. So it i exected that jaic erform between caic and maic when focuing on the choice of random effect. Preently, we are alying the idea in thi aer to addre the variable election for the LME model. Lat, extending our method to LME model with miing data i an intereting roblem and remain our further reearch. Acknowledgement We thank the editor Holger Rootzen, the aociate editor and two anonymou referee for their contructive comment and uggetion that greatly imroved the original manucrit. Thi reearch wa artially uorted by the National Natural Science Foundation of China (NNSFC) (grant no and ). Zhang work wa artially uorted by the NNSFC (grant no and 7154). Liang work wa artially uorted by the NSF grant DMS and DMS Zou work wa artially uorted by the NNSFC (grant no and ) and a grant from the Beijing High-level Talent Program. Aendix A.1. Derivation of formula (8) The following lemma will be ued in the roof of formula (8). Lemma (Stein, 1981): Let a be an N.; 1/ random variable and g W R to R be an indefinite integral of the Lebegue meaurable function g, eentially the derivative of g. Suoe alo that Ejg.a/j < 1. Then EŒg.a/ D E.ag.a//.

13 Scand J Statit Selection trategy for covariance 13 We firt write D X ˇ C Z b and b D.y b/=. Recall b D Xbˇ C Zbb. From formula (6), we have μ E.b;bb/ D E N log C k bk C N b b C r log./ C bb G b 1 bb (A.1) N log r log r N: A commonly ued aroach to obtain an (aroximately) unbiaed etimator of E.b;bb/ i to make ue of the (aroximate) ditribution of k bk = and. See, for examle, Hurvich & Tai (1989, 1995), Hurvich et al. (1998) and Vaida & Blanchard (5). For the LME model, however, uch ditribution might not be eay to find. Intead, we rooe to ue the following integration by art method. It i clear that. b/. E b/ D E ky bk ky bk D E NX id1 NX id1.yi i /.Y i b i / E C E b E yjb.yi i /.Y i b i / NX id1 C.Yi i / E NX id1.yi i / E b E yjb ; where Y i, i and b i denote the ith comonent of y; and b, reectively. Note that conditionally given b, y follow the ditribution of N. ; I N /. Auming that b i D.Y i b i /= i a continuou function with iecewie continuou artial derivative with reect to y, it can be hown from Stein lemma that ".Yi i /.Y i b i / E yjb D E yjb h.y i i / b i i D b i i rovided the exectation on the right-hand ide exit. Similarly, auming /=@Y i (i D 1;:::;N) are continuou function with iecewie continuou artial derivative and the correonding exectation exit, we have " #.Y i i / E yjb " 1 D E yjb C 4 b E yjb / i # ; So, " # " k E bk ky bk D E NX b i C N C 4 # / : i Subtituting thi formula in (A.1) and after ome calculation, we obtain E.b;bb/ D E N log./ C N log ky bk C C ˆ.y/ C 4.y/ μ Cr log./ Cbb G b 1 bb N log N log N r r log : In a conequence, formula (8) follow. (A.)

14 14 X. Zhang et al. Scand J Statit A.. Condition and roof of theorem 1 In what follow, max (min), max (min) and max (min) indicate the maximization (minimization) over i ¹1;:::;nº, ¹1;:::;Sº and j ¹1;:::;J º, reectively. Aume that a i i j j N!1, b! and the limit of b i >. Denote b;j and a the j th element ;j of b and, reectively. For D 1;:::;S and j D 1;:::;J, we can ;j =@y D bt ;j y almot urely, where bt ;j can be random and deend on y. Write b b =.@y@y / and D I N C Z G Z that i the covariance matrix of y. Let bm D I N bp, V have the ame form a bv excet that the notationb i removed and V D V j D. Similarly, we can define P, M, G,, P and M. Further, for j ¹1;:::;J º, denote W ;j where! ;j i the j th element of!, bw ;j D W ;j j!db!, D P y, H D E k k, and D minh. max.a/ and min.a/ denote the maximum and minimum ingular value for a matrix A. All limiting rocee dicued here are with reect to N! 1. c, Nc, Qc, c and c? are all generic oitive contant. The following condition are imoed to obtain theorem 1. A..1. Condition SP Condition 1. H 1!. D1 Condition. r 1!,. C r/ 1!, and N!. Condition 3. kyk D O.N /. Condition 4. N 1 max max.p bp /!. Condition 5. N 1 max max max.bt ;j /! and N 1 max max.b /!. Condition 6. max j max j ¹1;:::;J 1º max.bw ;j / Nc<1. Condition 7. min min. b G / Qc>or max max.zz / c? < 1. Condition 8. max max. 1= 1= / c < 1. Condition 1 i a tandard condition for aymtotic otimality of model election, and the imilar condition are ued in the literature like (A.3) in Li (1987) and (.6) in Shao (1997). Condition contain retriction on the increaing rate of the number of fixed effect and random effect and a N!1. Retriction imilar to them can be found in Shibata (198) and Newey (1997). Condition 3, which concern the um of y with i ¹1; ::; N º, i quite i common and reaonable. Condition 4 require that b converge to at a rate uch that 1 max max.p bp / converge to quicker than N!1.For ¹1;:::;Sº and j 1 ;j ¹1;:::;J º, we define a an N N matrix with the i 1 i th element ;i1;i D P J P J j 1D1 j.d1 ;j 1 ;j 1 /.b ;j ;j /@ P ;i1;i =.@ ;j / j De i 1 ;i, where P ;i1;i i the i 1 i th element of P and e i1;i i a J 1 vector between b and. From the formula (S.13) and (S.5) (S.7) in the roof of

15 Scand J Statit Selection trategy for covariance 15 theorem 1 in Suorting Information, we ee that when condition, 6 and 7 hold, condition 4 i imlied by max. / D O.N 1= / and N.b ;j ;j / D O.1/, which are the common convergence rate. Condition 5 lace contraint on the robutne of the etimator b. Take the lat element of b, b, a an examle. In the roce of ML etimation, b i calculated by b D y M M y=n. Then the retriction related to b in condition 5 are obviouly imlied by condition and 3. Condition 6 i on the derivative of G b. Conider a very common cae with! ;j directly being the element of G. In thi @! ;j i finite, and o condition 6 hold. The firt art of condition 7 exclude degenerate etimated ditribution of b, which i analogou to condition (A. ) of Andrew (1995) and condition (A.1) of Hanen & Racine (1). The econd art of condition 7 hold, for examle, in a ituation with m i being bounded and max.z i Z i / D O.m i / uniformly for i ¹1;:::;nº and ¹1;:::;Sº. Condition 8 lace contraint on the relation between and the true covariance matrix of y. Analogou condition have been imoed by Yang (4) and Yuan & Yang (5) for linear regreion. A... Proof of theorem 1. Let R D b H, D R C N c C log c! 1 Cbb b G 1 b C r log ; U D b.y @y@y C c ; and jaic D jaic N log N log N r log r ; where N log N log N r log r ha nothing to do with. So, b D arg min jaic : From (1) and (11), we have ¹1;:::;Sº. b ;bb / jaic D N log C N b N log N r log r C b k b k b k b C " k (A.3) C N log C N log C N C r log C r D b.n k" k / b. b " U and. b ;bb / D b k b k R r.1 C log /: (A.4) From condition 1 8, we can how that (ee Section S. of Suorting Information for detailed roof) max r!; (A.5)

16 16 X. Zhang et al. Scand J Statit max ˇ b ˇN k" k ˇˇ!; (A.6) max max b ˇ b j. b / " j!; ˇk b k H ˇˇ!; (A.7) (A.8) and max max ju j!; jtr.@b =@y/j!: (A.9) (A.1) Now, by (A.3) (A.1), we have max j. b ;bb / jaic j! and max j. b ;bb / j!; (A.11) which, along with the roof of theorem.1 in Li (1987), imly (1). Reference Akaike, H. (1974). A new look at the tatitical model identification. IEEE Tran, on Automatic Control AC 19, Andrew, D. W. K. (1995). Aymtotic otimality of generalized c l, cro-validation, and generalized cro-validation in regreion with heterokedatic error. Journal of Econometric 4, Chen, Z. & Dunon, D. (3). Random effect election in linear mixed model. Biometric 59, Davidian, M. & Giltinan, D. M. (1995). Nonlinear model for reeated meaurement data, Chaman and Hall, New York. Delattre, M., Lavielle, M. & Pourat, M. A. (14). A note on BIC in mixed-effect model. Electronic Journal of Statitic 8, Greven, S. & Kneib, T. (1). On the behaviour of marginal and conditional AIC in linear mixed model. Biometrika 97, Hanen, B. & Racine, J. (1). Jackknife model averaging. Journal of Econometric 167, Ho, D., Neumann, A., Perelon, A., Chen, W., Leonard, J. & Markowitz, M. (1995). Raid turnover of lama virion and CD4 lymhocyte in HIV-1 infection. Nature 373, Hurvich, C. & Tai, C. (1989). Regreion and time erie model election in mall amle. Biometrika 76, Hurvich, C. & Tai, C. (1995). Model election for extended quai-likelihood model in mall amle. Biometric 51, Hurvich, C. M., Simonoff, J. S. & Tai, C. L. (1998). Smoothing arameter election in nonarametric regreion uing an imroved Akaike information criterion. Journal of the Royal Statitical Society, Serie B 6, Kinney, S. & Dunon, D. (7). Fixed and random effect election in linear and logitic model. Biometric 63, Laird, N. M. & Ware, J. H. (198). Random-effect model for longitudinal data. Biometric 38, Lederman, M., Connick, E., Landay, A., Kuritzke, D., Sritzke, J., St Clair, M., Kotzin, B., Fox, L., Chiozzi, M., Leonard, J., Roueau, F., Wade, M., Roe, J., Martinez, A. & Keler, H. (1998). Immunologic reone aociated with 1 week of combination antiretroviral theray coniting of zidovudine, lamivudine, and ritonavir: reult of AIDS clinical trial grou rotocol 315. The Journal of Infectiou Dieae 178, 7 79.

17 Scand J Statit Selection trategy for covariance 17 Li, K. C. (1987). Aymtotic otimality for c ;c l, cro-validation and generalized cro-validation: dicrete index et. The Annal of Statitic 15, Li, L., Brown, M., Lee, K. & Guta, S. (). Etimation and inference for a line-enhanced nonlinear oulation harmacokinetic model. Biometric 58, Liang, H., Wu, H. L. & Zou, G. H. (8). A note on conditional AIC for linear mixed-effect model. Biometrika 95, Lu, K. L. & Berger, J. O. (1989). Etimation of normal mean: frequentit etimation of lo. The Annal of Statitic 17, Naik, P. & Tai, C. (1). Single-index model election. Biometrika 61, Newey, W. (1997). Convergence rate and aymtotic normality for erie etimator. Journal of Econometric 79, Ngo, L. & Brand, R. (). Model election in linear mixed effect model uing SAS Proc Mixed. SAS Global Forum. Pinheiro, J. & Bate, D. (). Mixed-effect model in S and S-PLUS, Sringer Science, New York. Saefken, B., Kneib, T., van Waveren, C. & Greven, S. (14). A unifying aroach to the etimation of the conditional Akaike information in generalized linear mixed model. Electronic Journal of Statitic 8, 1 5. Shao, J. (1997). An aymtotic theory for linear model election. Statitica Sinica 7, Shibata, R. (198). Aymtotically efficient election of the order of the model for etimating arameter of a linear roce. The Annal of Statitic 8, Stein, C. (1981). Etimation of the mean of a multivariate normal ditribution. The Annal of Statitic 9, Vaida, F. & Blanchard, S. (5). Conditional Akaike information for mixed-effect model. Biometrika 9, Verbeke, G. & Molenbergh, G. (9). Linear mixed model for longitudinal data, Sringer, New York. Voneh, E. & Chinchilli, V. (1996). Linear and nonlinear model for the analyi of reeated meaurement, Marcel Dekker, New York. Wei, X., Ghoh, S. K., Taylor, M. E., Johnon, V. A., Emini, E. A., Deutch, P., Lifonarallel, J. D., Bonhoeffer, S., Nowak, M. A., Hahn, B. H., Saag, M. & Shaw, G. M. (1995). Viral dynamic in human immunodeficiency viru tye 1 infection. Nature 373, Whittle, P. (196). Bound for the moment of linear and quadratic form in indeendent variable. Theory of Probability and It Alication 5, Wu, H. & Ding, A. (1999). Poulation HIV-1 dynamic in vivo: alicable model and inferential tool for virological data from AIDS clinical trial. Biometric 55, Wu, H. & Zhang, J. T. (6). Nonarametric regreion method for longitudinal data analyi. Wiley Serie in Probability and Statitic. Hoboken, Wiley-Intercience [John Wiley & Son], NJ. Yang, Y. (4). Combining forecating rocedure: ome theoretical reult. Econometric Theory, 176. Ye, J. M. (1998). On meauring and correcting the effect of data mining and model election. Journal of the American Statitical Aociation 93, Yu, D. & Yau, K. K. W. (1). Conditional Akaike information criterion for generalized linear mixed model. Comutational Statitic & Data Analyi 56, Yuan, Z. & Yang, Y. (5). Combining linear regreion model: when and how? Journal of the American Statitical Aociation 1, Received Setember 14, in final form July 15 Xinyu Zhang, Academy of Mathematic and Sytem Science, Chinee Academy of Science, Beijing 119, China. xinyu@am.ac.cn Suorting information Additional uorting information may be found in the online verion of thi article at the ubliher web ite.