An R implementation of bootstrap procedures for mixed models

Transcription

1 The R User Conference 2009 July 8-10, Agrocampus-Ouest, Rennes, France An R mplementaton of bootstrap procedures for mxed models José A. Sánchez-Espgares Unverstat Poltècnca de Catalunya Jord Ocaña Unverstat de Barcelona

2 Outlne Introducton and motvaton Bootstrap methods for Mxed Models Implementaton detals Some examples Conclusons

3 (Generalzed) Lnear Mxed Models Repeated measures or Longtudnal data: Response vector Y for th subject Y = ( Y 1,..., Yn Observatons on the same unt can be correlated )' Condtonal / Herarchcal approach: Between-subject varablty explaned by Random-effects b usually wth Normal dstrbuton ~ N(0, D) E( Y g( µ ) b ) = µ = θ = X b β + Z b

4 Estmaton n (G)LMM Random-effects are not drectly observed Estmaton of parameters based on Margnal Lkelhood, after ntegraton of Random-effects MLE: L( β, D, φ Y ) = n n = 1 j= 1 f ( Y n = 1 β, b, φ) f ( Y β, D, φ) = f ( b D) db Analytc soluton n the Normal case (Lnear Mxed Models) Approxmatons are needed n the general case. lme4 package: common framework for L-GL-NL/MM Fast and effcent estmaton for ML and REML crtera va Laplace Approxmaton/Adaptatve Gaussan Quadrature for GLMM.

5 Inference (G)LMM Wald-type and F-tests (summary) Asymptotc standard errors for the fxed effects parameters Lkelhood rato test (anova) Comparson of lkelhood of two models Bayesan Inference (mcmcsamp) MCMC samplng procedure for posterors on parameters Some drawbacks: Asymptotc results Degrees of freedom of the reference dstrbuton n F-test Lkelhood Rato test can be conservatve under some condtons Tests on Varance components close to the boundary of the parameter space.

6 Motvaton Inference based on bootstrap for LMM and GLMM Inference on functons of parameters.e. confdence ntervals and hypothess test for rato of varance components Robust approaches.e. n presence of nfluental data and outlers Effect of msspecfcaton.e. non-gaussan random effects and/or resduals

7 Extenson of the package lmer merboot provdes methods for Monte Carlo and bootstrap technques n generalzed and lnear mxedeffects models The mplementaton s object-orented It takes proft of specfctes of the appled algorthms to enhance effcency, usng less tme and memory. It has a flexble nterface to desgn complex experments.

8 Bootstrap n lnear models For (Generalzed) lnear models (wthout random effects) there s only one random component generaton of the response varable accordng to the condtonal mean. µ = g Resdual resamplng: µ = Xβ Y ~ N( µ, σ) 1 ( Xβ ) Y ~ F Estmate parameters for the systematc part of the model Resample random part of the model (parametrc or emprcal) Some varants to deal wth heterocedastcty (Wld bootstrap) µ

9 Bootstrap n Mxed Models In Mxed models, the systematc part has a random component generaton of the response varable n two steps: Bootstrap of the condtonal mean (functon of the lnear predctor) Bootstrap of the response varable µ = Xβ + Zb b~ N(0, θ) Y ~ N( µ, σ) µ = g 1 ( Xβ + Zb) b~ N(0, θ) Y ~ F Two objects n the merboot mplementaton: BGP: Set-up for the Bootstrap Generaton Process merboot: Coeffcents for the resamples and methods for analyss µ

10 Implementaton detals Step I Step II model specfcatons BGP generaton of samples model fttng object merboot number of samples

11 Bootstrap Generaton Process Fxed parameters β Lnear predctor level Desgn matrces X Z Random effects generator (b ) Parametrc: generatng b from a multvarate gaussan dstrbuton Semparametrc/Emprcal (from a ftted object): samplng b from wth replacement. User-defned: any other dstrbuton/crtera to generate b bˆ η = X β + Z b

12 Bootstrap Generaton Process Response level Famly (dstrbuton F + lnk functon g) Response generator (Y ) Parametrc: µ = g -1 (η ιj ), sample Y ~F(.; µ ) Semparametrc/Emprcal (from a ftted object): Resdual-based: bulds Y lke n lnear heterocedastc models, dependng on type of resduals Y = µ + ε Dstrbuton-based: resamples estmated quantles

13 Resduals n GLM Raw resduals: Y µˆ Pearson resduals: Standardzed Pearson resduals: e r = = Y a ˆ µ V ( ˆ µ ) e 1 h( ˆ µ ) Standardzed resduals on the lnear predctor scale: l = a g( Y g' ( ˆ µ ) ) 2 g( ˆ µ ) V ( ˆ µ )(1 h ) Devance resduals: r = sgn( Y ˆ µ ) d

14 Emprcal resdual-based Standardzed Pearson resduals: Resample e from centered e ˆ Calculate Y φ = µ + V ( µ ) e a Standardzed Pearson resduals on the lnear predctor scale: Resample l from l ˆ Calculate φ Y 1 + g η g' ( η ) V ( µ a l = )

15 Emprcal dstrbuton-based Randomzed Quantle resduals (Dunn & Smyth,1996): nvertng the estmated dstrbuton functon for each observaton 1 to obtan exactly unform (q ) or standard normal resduals ( Φ ( for dagnostcs. Randomzaton needed for dscrete dstrbutons. Resamplng scheme wth Quantles Resduals for (G)LMM: q )) Calculate q ˆ = F( ; ˆ µ ) Y Sample q wth replacement from qˆ Generate 1 Y = F ( ; ˆ µ ) q

16 Response generaton For Normal famly and dentty lnk functon, all three strateges (pearson, lnear predctor and quantle resduals) are the same. In all the schemes, response s rounded to the nearest vald value, accordng to the famly consdered. For dscrete varables, randomzaton of the quantles allows for contnuous unform resduals. Transformaton of the random effects n order to have the frst and second moments equal to the parameters (adjusted bootstrap). For all the schemes, f resample of resduals/quantles s restrcted to the subject obtaned n the lnear predctor level, a nested bootstrap s performed.

17 Bootstrap Generaton Process merbgp -model:mer merbgpparam -dstranef:functon -dstresd:functon merbgpsempar -ranef:data.frame -resd:lst -adjusted:logcal merbgpsemparnested merbgpsemparwld -dstwldranef:functon -dstwldresd:functon merbgpsemparnestedwld

18 generatelnpred BGPparam: BGPsempar: BGPsemparWld: BGP Methods generate (lmer) generatelnpred +Resdual-based BGPparam: ε ~ Fθ Y = µ + ε BGPsempar: ε ˆ ~ Fn (., e ) Y = µ + ε BGPsemparWld: ε ˆ ~ Fn (., e ) w ~ W Y = µ + w ε BGPsemparNested ε ˆ ~ Fn (., e j ) Y = µ + ε generate (glmer) generatelnpred +Dstrbuton-based BGPparam: BGPsempar: BGPsemparWld: BGPSemparNested b F η = X β + Z ~ ˆ θ b ˆ η = β + b ˆ + ~ Fn (., b ) X Zb ~ Fn (., b ) w ~ W η = X β Zw b q ~ F µ Y 1 = F µ ( q ) b

19 Object merboot Step II: Extracton of coeffcents pseudo-data set coeffcents model structure model fttng reft other statstcs

20 Bootstrap method for (g)lmer object dstref dstres adj nest wld B model2 Model specfcaton to generate pseudo-data set (glme object or a lst). It contans the formula descrbng the structure, the parameters (β,d) and the desgn matrx (X,Z) Parameters to ndcate how to generate varance components (random-effects) and response. Strngs (for pre-mplemented optons) or functons (userdefned methods) Number of samples. Alternatve model specfcaton used to ft the pseudo-data. Default s same as object

21 Example: sleepstudy model=lmer(reacton~days+(1 Subject) (0+Days Subject),sleepstudy) Average reacton tme (ms) Lnear mxed model ft by REML Formula: form Data: sleepstudy AIC BIC loglk devance REMLdev Random effects: Groups Name Varance Std.Dev. Subject (Intercept) Subject Days Resdual Number of obs: 180, groups: Subject, 18 Fxed effects: Estmate Std. Error t value (Intercept) Days Days of sleep deprvaton Correlaton of Fxed Effects: (Intr) Days

22 Methods merboot: prnt > sleep.boot=bootstrap(model,b=1000) > prnt(sleep.boot) lmer(formula = lmer(reacton~days+(1 Subject)+(0+Days Subject), data = sleepstudy) Resamplng Method: BGPparam Bootstrap Statstcs : orgnal bas mean std. error (Intercept) Days Subject.(Intercept) Subject.Days sgmareml

23 Methods merboot: ntervals > ntervals(sleep.boot) $norm lower upper (Intercept) Days Subject.(Intercept) Subject.Days sgmareml $basc lower upper (Intercept) Days Subject.(Intercept) Subject.Days sgmareml $perc lower upper (Intercept) Days Subject.(Intercept) Subject.Days sgmareml > HPDnterval(mcmcsamp(model,n=1000)) $fxef lower upper (Intercept) Days attr(,"probablty") [1] 0.95 $ST lower upper [1,] [2,] attr(,"probablty") [1] 0.95 $sgma lower upper [1,] attr(,"probablty") [1] 0.95

24 (Intercept) Methods merboot: plot Subject.(Intercept) Quantles of Standard normal Densty (Intercept) Densty Days sgmareml Quantles of Standard normal Densty Days Densty Subject.Days Densty Subject.(Intercept) Densty sgmareml/subject.days Quantles of Standard normal Subject.Days Quantles of Standard normal Quantles of Standard normal sgmareml β 0 β 1 σ 0 σ σ 1 σ /σ 1 Quantles of Standard normal sgmareml/subject.days

25 Methods merboot: LR & pars Devance ML Day s Subject.(Intercept) Subject.Day s sgmareml Densty 230 (Intercept)

26 Conclusons (G)LMM provde a framework to model longtudnal data for a wde range of stuatons (contnuous, count and bnary among others) but approxmate estmaton methods mply weaker nference. Monte-Carlo smulaton and bootstrap can enhance nference provdng emprcal p-values and bootstrap estmators. The BGP/merBoot objects developed allow mplementaton of dfferent bootstrap optons and evaluaton of the technques va smulaton.