Snger & Wllett, October, Dong Data Analyss n n the the Multlevel Model for for Change Judy Snger & John Wllett Harvard Unversty Graduate School of Educaton What What we we wll wll cover? cover? Composte specfcaton of the multlevel model for change Frst steps: uncondtonal means model and uncondtonal growth model Intraclass correlaton Quantfyng proporton of explaned outcome varaton Practcal model buldng strateges Developng and fttng a taxonomy of models Dsplayng prototypcal change trajectores Recenterng to mprove nterpretaton Comparng models Usng devance statstcs Usng nformaton crtera (AIC and BIC) Model-based (Emprcal Bayes) estmates of the ndvdual growth trajectores...5.6.9 p.8 p.9 p. p.6 p. Harvard Unversty
Snger & Wllett, October, What Data Data Example Wll Wll We We Use? Use? Change n n Adolescent Alcohol Use Use by by Parental Alchoholsm Research Queston: Do Do ndvdual trajectores of of alcohol alcohol use use durng durng adolescence dffer dffer by: by: () () parent parent alcoholsm; and and peer peer alcohol alcohol use? use? Ctaton: Curran, Curran, Stce, Stce, and andchassn (997). (997). Sample: 8 8 adolescents Desgn: Fully Fully balanced wth wth equally equally spaced spaced waves: waves: At At ages ages,, 5, 5, and and 6, 6, teenagers ndcated ther ther alcohol alcohol consumpton durng durng the the prevous year year Alcohol use use was was assessed usng usng an an 8-pont 8-pont scale scale for for tems tems (drank (drank beer/wne, hard hard lquor, lquor, 5 or or more more drnks drnks n n a a row, row, and and got got drunk). drunk). Each Each tem tem scored scored on on an an 8 pont pont scale scale (= not (= not at at all all to to 7= every day ) day ) for for a a maxmum of of.. Varables n n our our analyss analyss, s s square square root root of of the the sum sum of of the the tems tems COA, COA, s s the the teen teen a a Chld Chld Of Of an an Alcoholc parent? parent? PEER, a a measure measure of of peer peer alcohol alcohol use use at at age age.. What s an an Approprate Functonal Form for for Level- Submodel? Emprcal growth growth plots plots wth wth supermposed OLS OLS trajectores d ID # d ID # d ID # d ID # - - - - 5 6 7 5 6 7 5 6 7 5 6 7 d ID # d ID #56 d ID #65 d ID #8 - - - - 5 6 7 5 6 7 5 6 7 5 6 7 = π + π ( ) + ε features of these plots. Approxmately lnear, often (but not always ncreasng over tme). Some OLS trajectores ft well (,, 56, 65). Some OLS trajectores show more scatter (,,, 8) whereε ~ N(, σ ε ) Y = + π π TIME + ε s true ntal status (e, when TIME=) s true rate of change per unt of TIME porton of s outcome unexplaned on occason j Harvard Unversty
Snger & Wllett, October, What What Do Do the the Level- Submodels for for Indvdual Dfferences n n Change Look Look Lke? Lke? COA = COA = Level- ntercepts Populaton average ntal status and rate of change for a non-coa - 5 6 7 Low PEER - 5 6 7 Hgh PEER π π = γ = γ + γ + γ COA COA Level- slopes Effect of COA on ntal status and rate of change + ζ + ζ - 5 6 7-5 6 7 Examnng varaton n OLS-ftted level- trajectores by:. COA: COAs have hgher ntercepts but no steeper slopes.. PEER (splt at mean): Teens whose frends at age drnk more have hgher ntercepts but shallower slopes Level- resduals Devatons of ndvdual change trajectores around predcted averages ζ ζ σ ~ N, σ σ σ Developng the composte specfcaton of the multlevel model for change Key dea: It s mportant to realze that, nstead of a level-/level- specfcaton for the multlevel model, you can develop a sngle composte model, by substtutng the level- submodels nto the level- model ( γ + γ + ) ( γ + γ + ) COA ζ COA ζ Y π TIME + ε = + π ( γ + γ + ) Y = COA ζ ( γ + γ COA + ζ ) + ε + TIME Y = [ γ + γ TIME + [ ζ + γ + ζ COA + γ TIME + ε ( COA TIME )] ] The composte specfcaton shows how the outcome,, depends smultaneously on: The level- predctor TIME The level- predctor COA The cross-level nteracton, COA TIME. Ths tells us that the effect of one predctor (TIME) dffers by levels of another predctor (COA) It s also the specfcaton used n most computer software for multlevel modelng. Harvard Unversty
Snger & Wllett, October, Model A: A: Uncondtonal Means Model: Parttonng the the Outcome Varaton Usually, t s a good dea to begn any knd of multlevel analyss by fttng an uncondtonal means model Level- Model: Level- Model: Y = π + ε, where ε ~ N (, σ ε π = γ + ζ, where ζ ~ N(, σ ) ) Composte Model: Y γ ζ + ε = + Grand mean Person-specfc mean Wthn-person devaton Results of fttng Model A usng SAS PROC MIXED Covarance Parameter Estmates Standard Z Cov Parm Subject Estmate Error Value Pr Z Intercept ID.569.9.7 <. Resdual.567.6 9.6 <. Soluton for Fxed Effects Standard Effect Estmate Error DF t Value Pr > t Intercept.9.957 8 9.6 <. ˆ σ =.56 *** Estmated between-person varance ˆ =.56*** σ ε Estmated wthn-person varance γˆ =.9*** Estmated grand mean across occasons and ndvduals How How Do Do You You Estmate the theintraclass Correlaton? Comparng the the wthn-person and and between-person varance components An nterestng statstc derved from the estmated varance components n the uncondtonal means model s: the ntra-class correlaton coeffcent, ρ. It quantfes the proporton of total outcome varaton that les between people. ˆ σ =.56*** Estmated between-person varance ˆ =.56*** σ ε Estmated wthn-person varance ρ = σ σ + σ ε.56 ρˆ = =.5.56 +.56 An estmated 5% of the total varaton n alcohol use s attrbutable to dfferences among adolescents Harvard Unversty
Snger & Wllett, October, How How Do Do You You Specfy Specfy the the Uncondtonal Growth Growth Model Model (Model (Model B)? B)? Then, we usually ft an uncondtonal growth model: Level- Model: Level- Model: Composte Model: Y π π Y = π + π TIME + ε, where ε ~ N (, σ ε = γ = γ + ζ + ζ w here ζ ζ σ ~ N, σ σ σ = γ TIME TIME + + γ + [ ζ + ζ ε ] ) Average ntal status Average rate of change Composte resdual Estmates for fxed effects n Model B usng SAS PROC MIXED Soluton for Fxed Effects Standard Effect Estmate Error DF t Value Pr > t Intercept.65.5 8 6. <. _.77.65 8. <. ˆ =.65+.7( ) γˆ =.65*** Estmated average true ntal status (at ) γˆ =.7*** Estmated average true annual rate of change 5 6 7 How How Do Do You You Interpret Varance Components n n the the Uncondtonal Growth Model? Model? SAS PROC MIXED output contnued Covarance Parameter Estmates Standard Z Cov Parm Subject Estmate Error Value Pr Z UN(,) ID.6.8. <. UN(,) ID -.68.78 -.98.88 UN(,) ID.5.567.68.7 Resdual.7.568 6. <..6***.68.5* * Level- (wthn person): ˆ =.7 *** σ ε Estmated wthn-person resdual varance Level- (between-persons): σˆ Estmated between-person =.6*** resdual varance n ntal status Estmated between-person σˆ =.5*** resdual varance n rate of change Estmated covarance between σˆ =.68 ntal status and rate of change How How do do we we quantfy quantfy the the effect effect of of addng addng TIME TIMEto to the the uncondtonal uncondtonal means means model, model, to to produce produce the the uncondtonal uncondtonal growth growth model? model? Next slde Harvard Unversty 5
Snger & Wllett, October, How How Do Do You You Quantfy the the Proporton of of Outcome Varaton Explaned? R ε = Proporton al reducton n the Level - varance component.56.7 = =..56 R Y r ( ˆ ˆ ) = (.). ˆ = =, Y Y, Y % of the varaton n s assocated wth lnear tme.% of the total varaton n s assocated wth lnear tme Extendng Extendng the the dea dea of of proportonal proportonal reducton reducton n n varance varancecomponents components to to Level-: Level-: σˆ ( ) ˆ ( ζ Uncondtonal Growth Model σ ζ Subsequent Model) Pseudo Rζ = σˆ ( Uncondtonal Growth Model) ζ [ Careful [ Careful: : Don t Don t do do ths ths comparson comparson wth wth the the uncondtonal uncondtonal means means model.] model.] How How To To Ft Ft a Taxonomy of of Multlevel Models, Beyond Beyond The The Uncondtonal Growth Model? Model? Use all the ntuton and skll that you can brng from the cross-sectonal world: Examne the effect of each predctor separately. Prortze the predctors, Focusng on queston predctors, Include nterestng control predctors. Progress towards a fnal model whose nterpretaton addresses your RQs. Captalze on the the unque features of longtudnal data: Remember there are multple level- outcomes (the ndvdual growth parameters). Each can be related separately to the predctors Remember that there are two knds of effects beng modeled: Fxed effects and varance components. Not not all effects are requred n every model. In the Alcohol use data, research nterest focuses on the effect of COA, and PEER was a control. Model C: COA predcts both ntal status and rate of change. Model D: Adds PEER to both Level- sub-models n Model C. Model E: Smplfes Model D by removng the non-sgnfcant effect of COA on change. Harvard Unversty 6
Snger & Wllett, October, How How Do Do You You Assess Assess the the Uncontrolled Effects Effects of of the the Queston Predctor (Model (Model C)? C)? Est. ntal value of for non-coas s.6 (p<.) Est. dfferental n ntal between COAs and non-coas s.7 (p<.) Est. annual rate of change n for non-coas s.9 (p<.) Est. dfferental n annual rate of change between COAs and non-coas s.9 (ns) Identcal to B s because no level- predctors were added Down % from B COA explans % of var n ntal status (also stll stat sg suggestng need for level- predctors) Unchanged from B, COA explans no var n rate of change (but also stll stat sg suggestng need for level- predctors) Next step? Remove COA? Let s not just yet because t s our queston predctor. Add PEER? Yes, so we examne the controlled effects of COA. How How Do Do You You Assess Assess the the Controlled Effects Effects of of the the Queston Predctor (Model (Model D)? D)? Effects of COA Est. dff. n between COAs and non- COAs, controllng for PEER, s.579 (p<.) No sgnfcant dfference n rate of change. Effects of PEER Teens whose peers drnk more at also drnk more at (ntal status). Modest negatve effect on rate of change (p<.). Unchanged (as expected) Taken together, PEER and COA explan: 6.% of the varaton n ntal status, 7.9% of the varaton n rates of change. Next step? If we had other predctors we d add them, because the varance components are stll stat sg. Smplfy the model? Snce COA s not assocated wth rate of change, remove ths term from the model? Harvard Unversty 7
Snger & Wllett, October, What s What s the the Fnal Fnal Model Model That That Descrbes the the Relatonshp Between Adolescent Alcohol Alcohol Use Use and and the the Presence of of Alcoholc Parents? Effects of COA Controllng for PEER, the estmated dfferental n between COAs and non-coas s.57 (p<.) Effects of PEER Controllng for COA, for each -pt dfference n PEER, ntal s.695 hgher (p<.) but rate of change n s.5 lower (p<.) Varance Components are unchanged, suggestng lttle s lost by elmnatng the man effect of COA on rate of change (although there s stll level- varance left to be predcted by other varables.) Partal Covarance After controllng for PEER and COA, ntal status and rate of change are unrelated. How How Do Do You You Dsplay Dsplay the the Fndngs Prototypcal Ftted Ftted Plots? Plots? Let s start smply, by nterpretng Model C, whch contans the man effects of COA on both ntal status and rate of change at level- π$ =. 6 + 7. COA π$ =. 9. 9 COA Substtute observed values for COA ( and ) COA = When COA = When COA = π$ =. 6 + 7. ( ) =. 6 π$ = 9. 9. ( ) = 9. π$ =. 6 + 7. ( ) = 59. π$ = 9. 9. ( ) =. COA = Substtute the estmated growth parameters nto the ndvdual growth model: When COA = : Yˆ =.59 +. TIME When COA = : Yˆ =.6 +. 9TIME 5 6 7 Plot the ftted prototypcal growth trajectores. Harvard Unversty 8
Snger & Wllett, October, How How Do Do You You Dsplay Dsplay Prototypcal Change Change Trajectores When When the the Predctors Aren t Aren t All All Dchotomes? Substtute nterestng values of the predctors nto the ftted model and plot prototypcal trajectores: Choose substantvely nterestng values (e.g.,, & 6 years of educaton). Use a sensble range of percentles (e.g., th th /5 th th /9 th th ). Use the sample mean ±.5 (or ) standard devaton. Use just the sample mean f you want to smply control for a predctor s mpact nstead of dsplayng ts effect. COA = Hgh PEER COA = Low Let s nterpret Model E, whch contans the man effects of COA and PEER. In the sample, PEER = 8., sdpeer =. 76 Low PEER = 8.. 5(. 76 ) =. 655 Hgh PEER = 8. +. 5(. 76 ) = 8. Hgh PEER Low 5 6 7 What What Effect Effect Does Does Re-Centerng the the Predctors Have? Have? Re-centerng TIME at level- s often benefcal: Ensures that the ndvdual ntercepts are easly nterpretable, correspondng to status at a specfc age. Often ntal status but as we ll see, we can center TIME on any sensble value. Many estmates are unaffected by centerng You can extend re-centerng to predctors at Level-, by subtractng from each: The sample mean. Ths causes the level- ntercepts to represent average ftted values (mean PEER=.8; mean COA=.5). Another other meanngful value, e.g., yrs of ed, for IQ scores, etc. Centerng changes the level- ntercepts: In Model F, the ntercepts descrbe an average non-coa (wth a mean level of PEER) In Model G, the ntercepts descrbe an average teen (mean PEER and mean COA). Here, we prefer Model F, because t s easer to nterpret the effects of the dchotomous queston predctor, COA, that s un-centered. Harvard Unversty 9
Snger & Wllett, October, How How Should Should You You Conduct Hypothess Tests Tests When When Fttng Fttng Multlevel Models Models for for Change? Controversy about sngle parameter hypothess tests: Smple to conduct & easy to nterpret, but statstcans dsagree about ther nature, form, and effectveness. Dsagreement s so strong that some software (e.g., MLwN) don t routnely output them. Especally problematc for tests on varance components. Convenent & useful n hands-on data analyss. Alternatve Approach: Devancebased hypothess testng Superor statstcal propertes. Permts jont tests on several parameters smultaneously. Devance = [ LL current model LLsaturated model ] Devance quantfes how much worse the current model s, compared to a saturated model (the best model possble ): Model wth small devance s nearly as good as possble. Model wth large devance s much worse. Because a saturated model fts perfectly, the second term drops out: Devance = LL current model You can use the devance statstc to compare two ftted multlevel models f: Both models were ftted to the same data beware mssng data One model s nested wthn the other the less complex model s specfed by mposng constrants on one or more parameters (usually settng them to ) n the complex model. Devance ~ χ (asymptotcally), wth df = # of ndependent constrants How How Do Do You You Compare Nested Nested Models Models Usng Usng Devance Statstcs? We We obtan obtan Model Model Afrom Model Model Bby by nvokng nvokng constrants: constrants: H : γ =, σ =, σ = Dfference n Devance: (67.6-66.6) =.55 ( df, p<.) Reject H Uncondtonal growth model (B) fts better than uncondtonal means model (A). Harvard Unversty
Snger & Wllett, October, How How Do Do You You Use Use Devance Statstcs to to Smultaneously Evaluate the the Effects Effects of of Addng Addng Predctors to to Both Both Level- Level- Models? Comparng Model C (whch ncludes effects of COA on both ntal status and growth rates) to Model B. We obtan Model B from Model C by nvokng ndependent constrants (now only on the fxed effects): H : γ =, γ = Dfference n Devance 66.6-6.=5. ( df, p<.) Condtonal growth model provdes a better ft than the uncondtonal growth model Note that the pooled test does not mply that each level- slope s on ts own statstcally sgnfcant How How Can Can You You Compare Non-nested Multlevel Models? You can (supposedly) compare non-nested mult-level models usng the nformaton crtera Informaton Crtera: AIC and BIC Each nformaton crteron penalzes the log-lkelhood statstc for excesses n the structure of the current model The AIC penalty accounts for the number of parameters n the model. The BIC penalty goes further and also accounts for sample sze. Models need not be nested, but datasets must be the same. Smaller values of AIC and BIC ndcate that the ft s better. Here s the taxonomy of multlevel models that we ended up fttng, n the example.. Model E has the lowest AIC and BIC statstcs Interpretng dfferences n BIC across models (Raftery, 995): -: Weak evdence -6: Postve evdence 6-: Strong evdence >: Very strong Careful: Gelman & Rubn (995) declare these statstcs and crtera to be off-target and only by serendpty manage to ht the target Harvard Unversty
Snger & Wllett, October, Model-based (Emprcal Bayes) Bayes) Estmates of of the the Indvdual Growth Trajectores Three ways to estmate the ndvdual growth trajectores: OLS estmates are straghtforward, but neffcent Populaton average estmates are derved by substtutng n observed predctor values nto the ftted model for each ndvdual Emprcal Bayes estmates are a weghted average of the two: When OLS estmates are precse they have greater weght When OLS estmates are mprecse, they have less weght Advantages of model based estmates: They shrnk the OLS estmates towards the peer averages Requre estmaton of fewer parameters. More precse. But, they are based (OLS estmates are unbased) Across panels, the OLS trajectores are the most dfferent, the PA trajectores are most stable, and the EB trajectores le between (by defnton). Wthn panels, the estmates for some teens are very dfferent (esp and ) OLS EB Pop Av d ID # d ID # d ID # d ID # - - - - 5 6 7 5 6 7 5 6 7 5 6 7 d ID # ID d #56 d ID #65 d ID #8 - - - - 5 6 7 5 6 7 5 6 7 5 6 7 Wthn panels, the trajectores for some teens are relatvely consstent (, 56 & ) Harvard Unversty