Advances in Longitudinal Methods in the Social and Behavioral Sciences. Finite Mixtures of Nonlinear Mixed-Effects Models.

Transcription

1 Advances n Longtudnal Methods n the Socal and Behavoral Scences Fnte Mxtures of Nonlnear Mxed-Effects Models Jeff Harrng Department of Measurement, Statstcs and Evaluaton The Center for Integrated Latent Varable Research Unversty of Maryland, College Park harrng@umd.edu Advances n Longtudnal Methods Conference J. Harrng Overvew Mxture modelng unvarate and multvarate applcatons Characterstcs of repeated measures learnng data Nonlnear mxed-effects models model descrpton and analyss Nonlnear mxed-effects mxture (NLMM) model model descrpton Advances n Longtudnal Methods Conference J. Harrng 2

2 Overvew Learnng data example revsted analytc decson ponts Issues, challenges & consderatons Advances n Longtudnal Methods Conference J. Harrng 3 Fnte Mxture Models Karl Pearson (894) Prmary purposes model the densty of complex dstrbutons model populaton heterogenety Modelng heterogenety mxture of dstrbutons from the same parametrc famly Inferental goals Advances n Longtudnal Methods Conference J. Harrng 4

3 Applcatons of Fnte Mxture Models Unvarate mxtures Advances n Longtudnal Methods Conference J. Harrng 5 Applcatons of Fnte Mxture Models Regresson mxtures y x e 0k k k Regresson mxture modelng nvolves estmatng separate regresson coeffcents and error for each latent class Advances n Longtudnal Methods Conference J. Harrng 6

4 Applcatons of Fnte Mxture Models Multvarate normal mxtures MVN mxtures cluster analyss Advances n Longtudnal Methods Conference J. Harrng 7 Applcatons of Fnte Mxture Models Multvarate normal mxtures f K f( y, ) f ( y ) k k k k p 2 ( y ) (2 ) exp ( y ) ( y ) k k k k k k 2 2 Advances n Longtudnal Methods Conference J. Harrng 8

5 Applcatons of Fnte Mxture Models Multvarate normal mxtures f K f( y, ) f ( y ) k k k k p 2 ( y ) (2 ) exp ( y ) ( y ) k k k k k k 2 2 Advances n Longtudnal Methods Conference J. Harrng 9 Applcatons of Fnte Mxture Models Multvarate normal mxtures f K f( y, ) f ( y ) k k k k p 2 ( y ) (2 ) exp ( y ) ( y ) k k k k k k 2 2 Advances n Longtudnal Methods Conference J. Harrng 0

6 Applcatons of Fnte Mxture Models Multvarate normal mxtures f K f( y, ) f ( y ) k k k k p 2 ( y ) (2 ) exp ( y ) ( y ) k k k k k k 2 2 Advances n Longtudnal Methods Conference J. Harrng Applcatons of Fnte Mxture Models Growth mxture modelng Advances n Longtudnal Methods Conference J. Harrng 2

7 Applcatons of Fnte Mxture Models Growth mxture modelng y e k k k k f p 2 ( y ) (2 ) exp ( y ) ( y ) k k k k k k 2 k k k k k 2 Advances n Longtudnal Methods Conference J. Harrng 3 Applcatons of Fnte Mxture Models Advances n Longtudnal Methods Conference J. Harrng 4

8 Applcatons of Fnte Mxture Models Handle nonlnear data wth a nonlnear functon Modelng potental populaton heterogenety Advances n Longtudnal Methods Conference J. Harrng 5 Applcatons of Fnte Mxture Models Intrnscally Nonlnear Functons Nonlnear Mxed- Effects Model Modelng Populaton Heterogenety Fnte Mxture Model Advances n Longtudnal Methods Conference J. Harrng 6

9 Applcatons of Fnte Mxture Models Intrnscally Nonlnear Functons Nonlnear Mxed- Effects Model + Modelng Populaton Heterogenety Fnte Mxture Model Nonlnear Mxed- Effects Mxture Model Advances n Longtudnal Methods Conference J. Harrng 7 Nonlnear Learnng Data Speech Errors Burchnal & Appelbaum (99) Repeated measures data are recorded speech errors on a standard passage of text from an nstrument of language profcency A sample of 43 young chldren rangng n ages from 2 to 8 years Advances n Longtudnal Methods Conference J. Harrng 8

10 Nonlnear Learnng Data Speech Errors Burchnal & Appelbaum (99) A maxmum number of 6 repeated measures were taken Ages at tme of testngs dffered for each chld ncomplete cases were observed ndependent ratng of overall speech ntellgblty used as a covarate Advances n Longtudnal Methods Conference J. Harrng 9 Model for Nonlnear Learnng Data The nonlnear mxed-effects (NLME) model s well- suted to handle both ntrnscally nonlnear functons as well as key data and desgn characterstcs contnuous response compellngly strong ndvdual dfference n trajectores substantal varablty n tme-response across subjects dstnct measurement occasons for each subject nterestng nonlnear change y f(,,, x ) e j p j j j,, n;,, m Advances n Longtudnal Methods Conference J. Harrng 20

11 NLME Model for Burchnal & Appelbaum Data Followng Davdan & Gltnan (2003) : two-stage herarchy A subject-specfc deceleratng, decreasng exponental functon s proposed Stage : Indvdual-Level Model y exp ( x 3) e j 2 j j : the average number of speech errors at age 3 : rate parameter that governs functonal declne 2 Advances n Longtudnal Methods Conference J. Harrng 2 NLME Model for Burchnal & Appelbaum Data Stage 2: Populaton-Level Model b b uncondtonal z b z b condtonal Advances n Longtudnal Methods Conference J. Harrng 22

12 NLME Model for Burchnal & Appelbaum Data Dstrbutonal assumptons b N( 0, ) e N(, 0 ()) cov( e, b ) 0 cov( e, e ) 0 cov( b, b ) 0 cov( b ) var( b ) cov( b2, b ) var( b2) Advances n Longtudnal Methods Conference J. Harrng 23 NLME Model for Burchnal & Appelbaum Data Dstrbutonal assumptons b N( 0, ) e N(, 0 ()) cov( e, b ) 0 cov( e, e ) 0 cov( b, b ) 0 2 I n Advances n Longtudnal Methods Conference J. Harrng 24

13 Inference NLME Model : Maxmum Lkelhood The margnal dstrbuton of y h( y ) p( y, b ) db p( y b ) ( b ) db p Let,, vech( ) Parameter estmaton s carred out by maxmzng the log-lkelhood AGHQ (Pnhero & Bates), GHQ (Davdan & Gallant), Lnearzaton (Lndstrom & Bates), GTS (Davdan & Gltnan), Bayesan l( ) ln L( y) m ln h( y ) Advances n Longtudnal Methods Conference J. Harrng 25 Analyss The uncondtonal model was ftted usng SAS PROC NLMIXED (Gaussan Quadrature 30 ponts) ˆ. ˆ 8 48 ˆ ˆ ˆ ln L227.0 BIC Advances n Longtudnal Methods Conference J. Harrng 26

14 Analyss The condtonal model was ftted wth mean-centered covarate SAS PROC NLMIXED (GH - 30 ponts) ˆ ˆ 8.22 ˆ ˆ ˆ 2.76 ˆ ˆ ˆ ln L27.2 BIC Advances n Longtudnal Methods Conference J. Harrng 27 Potentally Clustered Data In subject-specfc models, lke the NLME model, regresson parameters are allowed to vary across ndvduals resultng n dfferng wthn-subject profles E( b ) 0 & E( e ) 0 assumpton all subjects were sampled from a sngle populatons wth common parameters Fnte mxture models relax the sngle populaton dstrbutonal assumpton of the random effects and the condtonal dstrbuton for the data to allow for parameter dfferences across unobserved populatons Verbeke & Lesaffre, 996, Verbeke & Molenberghs, 2000 Muthén & Shedden, 999; Muthén, 200, 2003, 2004 Hall & Wang, 2005 Advances n Longtudnal Methods Conference J. Harrng 28

15 NLMM Model The exponental NLME model can be extended to a NLMM model for K latent classes where n class k (k =, 2,, K ) y 2 j exp ( x 3) e e N ( 0, ) k z b b N( 0, ) k k and k 2k k s the probablty of belongng to class k k k k 0 k Advances n Longtudnal Methods Conference J. Harrng 29 NLMM Model Gettng Started Ft a seres of models suppressng random effects : 0 k & k (LCGM Nagn (999)) Method of dervng startng values for the mean structure of the NLMM model Use NLME output to suggest startng values for and Advances n Longtudnal Methods Conference J. Harrng 30

16 NLMM Model How Many Latent Classes? Several statstcs have been proposed and recommended n practce: AIC, BIC, SBIC, CLC, LMR-LRT (Lo, Mendell & Rubn, 200), BLRT, Multvarate skewness and kurtoss ndces Compute and plot BIC or SBIC values aganst number of classes Advances n Longtudnal Methods Conference J. Harrng 3 NLMM Model How Many Latent Classes? No Wthn-Class Varaton LCGM Wthn-Class Varaton NLMM Advances n Longtudnal Methods Conference J. Harrng 32

17 NLMM Model How Many Latent Classes? Decson based on theoretcal defensblty model ft parsmony class separaton and class ncdence Expertse of the substantve researcher Do the classes represent substantvely meanngful groups? Advances n Longtudnal Methods Conference J. Harrng 33 NLMM Model Analyss of Learnng Data Parameter estmates for the two-class model NLME Estmates ˆ Class Means ˆ k Class-Specfc Covarate Estmates ˆ k ˆ 033. ˆ ˆ ˆ ˆ ˆ ˆ 276. * ˆ ˆ Advances n Longtudnal Methods Conference J. Harrng 34

18 NLMM Model Analyss of Learnng Data Parameter estmates for the two-class model NLME Estmate Covarance Matrx ˆ Class-Specfc Covarance Matrx ˆ ˆ k NLME Estmate Resdual Varance 2 ˆ Class-Specfc Resdual Varance 2 2 ˆ ˆ k * 2.73* Advances n Longtudnal Methods Conference J. Harrng 35 NLMM Model Assessng Model Ft? The qualty of the mxture based on the precson of the classfcaton classfcaton s based on estmated posteror probabltes For K 2, average posteror probabltes can be computed. A K K matrx should have hgh dagonal and low offdagonal values ndcatng good classfcaton qualty Average Latent Class Probabltes for Most Lkely Latent Class Membershp (Row) by Latent Class (Column) Advances n Longtudnal Methods Conference J. Harrng 36

19 NLMM Model Assessng Model Ft? Entropy a summary measure of the classfcaton based on ndvduals estmated posteror probabltes can be computed wth values close to ndcatng near perfect classfcaton E K m K k ( ˆ ln ˆ ) k mln K k E 0.73 K 2 Advances n Longtudnal Methods Conference J. Harrng 37 Graphcal Summares Plot predcted random coeffcents on a contour plot or surface plot Advances n Longtudnal Methods Conference J. Harrng 38

20 Graphcal Summares Plot predcted random coeffcents on a contour plot or surface plot Good Separaton Modest Separaton Advances n Longtudnal Methods Conference J. Harrng 39 Graphcal Summares Two Latent Classes: Mean Trajectores Advances n Longtudnal Methods Conference J. Harrng 40

21 Graphcal Summares Ftted Curves for 4 Indvduals Ftted Curves for 4 Indvduals n Class n Class 2 Advances n Longtudnal Methods Conference J. Harrng 4 Practcal Issues Estmaton Computng standard errors Advances n Longtudnal Methods Conference J. Harrng 42

22 Log-lkelhood Let (,, ) K Let (, vech( ), vech( )) k k k Let (, ) all model parameters, then the log-lkelhood l( ) ln L( y) m ln h ( y ) m K k K ln h ( y ) k k k k k where h ( y ) p ( y b ) p ( b ) db k k k Advances n Longtudnal Methods Conference J. Harrng 43 Estmaton If the nonlnear regresson coeffcents are fxed across ndvduals : Mplus (however does not handle unque measurement occasons) y exp ( x 3) e j 2 j j Drectly maxmze the log-lkelhood usng gradent methods lke N-R or Quas-Newton Use EM (Expectaton Maxmzaton) algorthm treatng class membershp as mssng data Advances n Longtudnal Methods Conference J. Harrng 44

23 Standard Errors Drect maxmzaton uses the dagonal elements of the Hessan matrx at convergence for a model-based estmate of the standard errors. EM algorthm no standard errors are computed as a byproduct of the algorthm At convergence, use a drect maxmzaton step to produce SE Advances n Longtudnal Methods Conference J. Harrng 45 More Practcal Issues and Future Consderatons Evdence of local extrema Covarates predctng ndvdual coeffcents & class membershp Does the method of handlng the ntractable ntegraton nfluence the number of latent classes? Normalty of the random effects dstrbuton: nonparametrc alternatves? Advances n Longtudnal Methods Conference J. Harrng 46

24 Advances n Longtudnal Methods n the Socal and Behavoral Scences Fnte Mxtures of Nonlnear Mxed-Effects Models Jeff Harrng Department of Measurement, Statstcs and Evaluaton The Center for Integrated Latent Varable Research Unversty of Maryland, College Park harrng@umd.edu Advances n Longtudnal Methods Conference J. Harrng 47