Censored Quantile Regression and Survival Models

Transcription

1 Censred Quantile Regressin and Survival Mdels Rger Kenker University f Illinis, Urbana-Champaign University f Minh June 2017 Rger Kenker (UIUC) CRQ Redux Braga / 34

2 Quantile Regressin fr Duratin (Survival) Mdels A wide variety f survival analysis mdels, fllwing Dksum and Gask (1990), may be written as, h(t i ) = x i β + u i where h is a mntne transfrmatin, and T i is an bserved survival time, x i is a vectr f cvariates, β is an unknwn parameter vectr {u i } are iid with df F. Rger Kenker (UIUC) CRQ Redux Braga / 34

3 The Cx Mdel Fr the prprtinal hazard mdel with lg λ(t x) = lg λ 0 (t) x β the cnditinal survival functin in terms f the integrated baseline hazard Λ 0 (t) = t 0 λ 0(s)ds as, lg( lg(s(t x))) = lg Λ 0 (t) x β s, evaluating at t = T i, we have the mdel, fr u i iid with df F 0 (u) = 1 e eu. lg Λ 0 (T) = x β + u Rger Kenker (UIUC) CRQ Redux Braga / 34

4 The Bennett (Prprtinal-Odds) Mdel Fr the prprtinal dds mdel, where the cnditinal dds f death Γ(t x) = F(t x)/(1 F(t x)) are written as, we have, similarly, lg Γ(t x) = lg Γ 0 (t) x β, lg Γ 0 (T) = x β + u fr u iid lgistic with F 0 (u) = (1 + e u ) 1. Rger Kenker (UIUC) CRQ Redux Braga / 34

5 Accelerated Failure Time Mdel In the accelerated failure time mdel we have s lg(t i ) = x i β + u i P(T > t) = P(e u > te xβ ) = 1 F 0 (te xβ ) where F 0 ( ) dentes the df f e u, and thus, λ(t x) = λ 0 (te xβ )e xβ where λ 0 ( ) dentes the hazard functin crrespnding t F 0. In effect, the cvariates act t rescale time in the baseline hazard. Rger Kenker (UIUC) CRQ Redux Braga / 34

6 Beynd the Transfrmatin Mdel The cmmn feature f all these mdels is that after transfrmatin f the bserved survival times we have: a pure lcatin-shift, iid-errr regressin mdel cvariate effects shift the center f the distributin f h(t), but cvariates cannt affect scale, r shape f this distributin Rger Kenker (UIUC) CRQ Redux Braga / 34

7 An Applicatin: Lngevity f Mediterrean Fruit Flies In the early 1990 s there were a series f experiments designed t study the survival distributin f lwer animals. One f the mst influential f these was: Carey, J.R., Lied, P., Orzc, D. and Vaupel, J.W. (1992) Slwing f mrtality rates at lder ages in large Medfly chrts, Science, 258, ,203,646 medflies survival times recrded in days Sex was recrded n day f death Pupae were initially srted int ne f five size classes 167 aluminum mesh cages cntaining rughly 7200 flies Adults were given a diet f sugar and water ad libitum Rger Kenker (UIUC) CRQ Redux Braga / 34

8 Majr Cnclusins f the Medfly Experiment Mrtality rates declined at the ldest bserved ages. cntradicting the traditinal view that aging is an inevitable, mntne prcess f senescence. The right tail f the survival distributin was, at least by human standards, remarkably lng. There was strng evidence fr a crssver in gender specific mrtality rates. Rger Kenker (UIUC) CRQ Redux Braga / 34

9 Lifetable Hazard Estimates by Gender mrtality rate M F days Smthed mrtality rates fr males and females. Rger Kenker (UIUC) CRQ Redux Braga / 34

10 Medfly Survival Prspects Lifespan Percentage Number (in days) Surviving Surviving , , Initial Ppulatin f 1,203,646 Rger Kenker (UIUC) CRQ Redux Braga / 34

11 Medfly Survival Prspects Lifespan Percentage Number (in days) Surviving Surviving , , Initial Ppulatin f 1,203,646 Human Survival Prspects Lifespan Percentage Number (in years) Surviving Surviving , , , , Estimated Thatcher (1999) Mdel Rger Kenker (UIUC) CRQ Redux Braga / 34

12 Quantile Regressin Mdel (Geling and K (JASA,2001)) Criticism f the Carey et al paper revlved arund whether declining hazard rates were a result f cnfunding factrs f cage density and initial pupal size. Our basic QR mdel included the fllwing cvariates: Q lg(ti )(τ x i ) = β 0 (τ) + β 1 (τ)sex + β 2 (τ)size + β 3 (τ)density + β 4 (τ)%male SEX Gender SIZE Pupal Size in mm DENSITY Initial Density f Cage %MALE Initial Prprtin f Males Rger Kenker (UIUC) CRQ Redux Braga / 34

13 Base Mdel Results with AFT Fit Intercept Gender Effect Size Effect Quantile Quantile Quantile Density Effect %Male Effect Quantile Quantile Rger Kenker (UIUC) CRQ Redux Braga / 34

14 Base Mdel Results with Cx PH Fit Intercept Gender Effect Size Effect Quantile Quantile Quantile Density Effect %Male Effect Quantile Quantile Rger Kenker (UIUC) CRQ Redux Braga / 34

15 What Abut Censring? There are currently 3 appraches t handling censred survival data within the quantile regressin framewrk: Pwell (1986) Fixed Censring Prtny (2003) Randm Censring, Kaplan-Meier Analgue Peng/Huang (2008) Randm Censring, Nelsn-Aalen Analgue Available fr R in the package quantreg. Rger Kenker (UIUC) CRQ Redux Braga / 34

16 Pwell s Apprach fr Fixed Censring Ratinale Quantiles are equivariant t mntne transfrmatin: Q h(y) (τ) = h(q Y (τ)) fr h Mdel Y i = T i C i min{t i, C i } Q Yi x i (τ x i ) = x i β(τ) C i Data Censring times are knwn fr all bservatins {Y i, C i, x i : i = 1,, n} Estimatr Cnditinal quantile functins are nnlinear in parameters: ˆβ(τ) = argmin ρ τ (Y i x i β C i) Rger Kenker (UIUC) CRQ Redux Braga / 34

17 Prtny s Apprach fr Randm Censring I Ratinale Efrn s (1967) interpretatin f Kaplan-Meier as shifting mass f censred bservatins t the right: Algrithm Until we encunter a censred bservatin KM quantiles can be cmputed by slving, starting at τ = 0, ˆξ(τ) = argmin ξ n i=1 ρ τ (Y i ξ) Once we encunter a censred bservatin, i.e. when ˆξ(τ i ) = y i fr sme y i with δ i = 0, we split y i int tw parts: y (1) i = y i with weight w i = (τ τ i )/(1 τ i ) y (2) i = y = with weight 1 w i. Then denting the index set f censred bservatins encuntered up t τ by K(τ) we can slve min ρ τ (Y i ξ)+ [w i (τ)ρ τ (Y i ξ)+(1 w i (τ))ρ τ (y ξ)]. i/ K(τ) i K(τ) Rger Kenker (UIUC) CRQ Redux Braga / 34

18 Prtny s Apprach fr Randm Censring II When we have cvariates we can replace ξ by the inner prduct x i β and slve: min ρ τ (Y i x i β)+ [w i (τ)ρ τ (Y i x i β)+(1 w i (τ))ρ τ (y x i β)]. i/ K(τ) i K(τ) At each τ this is a simple, weighted linear quantile regressin prblem. Rger Kenker (UIUC) CRQ Redux Braga / 34

19 Prtny s Apprach fr Randm Censring II When we have cvariates we can replace ξ by the inner prduct x i β and slve: min ρ τ (Y i x i β)+ [w i (τ)ρ τ (Y i x i β)+(1 w i (τ))ρ τ (y x i β)]. i/ K(τ) i K(τ) At each τ this is a simple, weighted linear quantile regressin prblem. The fllwing R cde fragment replicates an analysis in Prtny (2003): require(quantreg) data(uis) fit <- crq(surv(lg(time), CENSOR) ~ ND1 + ND2 + IV3 + TREAT + FRAC + RACE + AGE * SITE, data = uis, methd = "Pr") Sfit <- summary(fit,1:19/20) PHit <- cxph(surv(time, CENSOR) ~ ND1 + ND2 + IV3 + TREAT + FRAC + RACE + AGE * SITE, data = uis) plt(sfit, CxPHit = PHit) Rger Kenker (UIUC) CRQ Redux Braga / 34

20 Reanalysis f the Hsmer-Lemeshw Drug Relapse Data ND1 ND2 IV TREAT FRAC RACE AGE SITE AGE:SITE Rger Kenker (UIUC) CRQ Redux Braga / 34

21 Peng and Huang s Apprach fr Randm Censring I Ratinale Extend the martingale representatin f the Nelsn-Aalen estimatr f the cumulative hazard functin t prduce an estimating equatin fr cnditinal quantiles. Mdel AFT frm f the quantile regressin mdel: Prb(lg T i x i β(τ)) = τ Data {(Y i, δ i ) : i = 1,, n} Y i = T i C i, δ i = I(T i < C i ) Martingale We have EM i (t) = 0 fr t 0, where: M i (t) = N i (t) Λ i (t Y i x i ) N i (t) = I({Y i t}, {δ i = 1}) Λ i (t) = lg(1 F i (t x i )) F i (t) = Prb(T i t x i ) Rger Kenker (UIUC) CRQ Redux Braga / 34

22 Peng and Huang s Apprach fr Randm Censring II The estimating equatin becmes, En 1/2 x i [N i (exp(x i β(τ))) τ 0 I(Y i exp(x i β(u)))dh(u) = 0. where H(u) = lg(1 u) fr u [0, 1), after rewriting: Λ i (exp(x i β(τ)) Y i x i )) = H(τ) H(F i (Y i x i )) = τ 0 I(Y i exp(x i β(u)))dh(u), Rger Kenker (UIUC) CRQ Redux Braga / 34

23 Peng and Huang s Apprach fr Randm Censring III Apprximating the integral n a grid, 0 = τ 0 < τ 1 < < τ J < 1 yields a simple linear prgramming frmulatin t be slved at the gridpints, j 1 α i (τ j ) = I(Y i exp(x ˆβ(τ i k )))(H(τ k+1 ) H(τ k )), k=0 yielding Peng and Huang s final estimating equatin, n 1/2 x i [N i (exp(x i β(τ))) α i(τ)] = 0. Setting r i (b) = lg(y i ) x i b, this cnvex functin fr the Peng and Huang prblem takes the frm R(b, τ j ) = n r i (b)(α i (τ j ) I(r i (b) < 0)δ i ) = min! i=1 Rger Kenker (UIUC) CRQ Redux Braga / 34

24 Prtny vs. Peng-Huang Peng Huang Prtny Rger Kenker (UIUC) CRQ Redux Braga / 34

25 Sme One Sample Asympttics Suppse that we have a randm sample f pairs, {(T i, C i ) : i = 1,, n} with T i F, C i G, and T i and C i independent. Let Y i = min{t i, C i }, as usual, and δ i = I(T i < C i ). In this setting the Pwell estimatr f θ = F 1 (τ), is asympttically nrmal, ˆθ P = argmin θ n i=1 ρ τ (Y i min{θ, C i }). n(ˆθ P θ) N(0, τ(1 τ)/(f 2 (θ)(1 G(θ)))). Rger Kenker (UIUC) CRQ Redux Braga / 34

26 One Sample Asympttics In cntrast, the asympttic thery f the quantiles f the Kaplan-Meier estimatr is slightly mre cmplicated. Using the δ-methd ne can shw, n(ˆθ KM θ) N(0, Avar(Ŝ(θ))/f 2 (θ)) where, see e.g. Andersn et al, Avar(Ŝ(t)) = S 2 (t) t 0 (1 H(u)) 2 d F(u) and 1 H(u) = (1 F(u))(1 G(u)) and F(u) = t 0 (1 G(u))dF(u). Since the Pwell estimatr makes use f mre sample infrmatin than des the Kaplan Meier estimatr it might be thught that it wuld be mre efficient. But this isn t true. Rger Kenker (UIUC) CRQ Redux Braga / 34

27 Kaplan Meier vs Pwell Prpsitin Avar(ˆθ KM ) Avar(ˆθ P ). Prf: θ f 2 (θ)avar(ˆθ KM ) = S(θ) 2 (1 H(s)) 2 d F(s) 0 θ = S(θ) 2 (1 G(s)) 1 (1 F(s)) 2 df(s) 0 = = = = S(θ) 2 1 G(θ) S(θ) 2 1 G(θ) θ (1 F(s)) 2 df(s) 0 θ 1 1 F(s) 0 S(θ) 2 1 G(θ) F(θ) 1 F(θ) F(θ)(1 F(θ)) (1 G(θ)) τ(1 τ) (1 G(θ)). Rger Kenker (UIUC) CRQ Redux Braga / 34

28 Alice in Asymptpia Leurgans (1987) cnsidered the weighted estimatr f the censred survival functin, Ŝ L (t) = I(Yi > t)i(c i > t), I(Ci > t) that uses all the C i s. Cnditining n the C i s, it can be shwn that E(Ŝ L (t) C) = S(t), and that the cnditinal variance is Var(Ŝ L (t) C) = F(t)(1 F(t)). 1 Ĝ(t) Averaging this expressin gives the uncnditinal variance which cnverges t F(t)(1 F(t)) Avar(Ŝ L (t) C) =, 1 G(t) and cnsequently quantiles based n Leurgan s estimatr behave (asympttically) just like thse prduced by the Pwell estimatr. Rger Kenker (UIUC) CRQ Redux Braga / 34

29 Alice in Asymptpia It might be thught that the Pwell estimatr wuld be mre efficient than the Prtny and Peng-Huang estimatrs given that it impses mre stringent data requirements. Cmparing asympttic behavir and finite sample perfrmance in the simplest ne-sample setting indicates therwise. median Kaplan-Meier Nelsn-Aalen Pwell Leurgans Ĝ Leurgans G n= n= n= n= n= Scaled MSE fr Several Estimatrs f the Median: Mean squared errr estimates are scaled by sample size t cnfrm t asympttic variance cmputatins. Here, T i is standard lgnrmal, and C i is expnential with rate parameter.25, s the prprtin f censred bservatins is rughly 30 percent replicatins. Rger Kenker (UIUC) CRQ Redux Braga / 34

30 Simulatin Settings I x Y x Y Rger Kenker (UIUC) CRQ Redux Braga / 34

31 Simulatins I-A Intercept Slpe Bias MAE RMSE Bias MAE RMSE Prtny n = n = n = Peng-Huang n = n = n = Pwell n = n = n = GMLE n = n = n = Cmparisn f Perfrmance fr the iid Errr, Cnstant Censring Cnfiguratin Rger Kenker (UIUC) CRQ Redux Braga / 34

32 Simulatins I-B Intercept Slpe Bias MAE RMSE Bias MAE RMSE Prtny n = n = n = Peng-Huang n = n = n = Pwell n = n = n = GMLE n = n = n = Cmparisn f Perfrmance fr the iid Errr, Variable Censring Cnfiguratin Rger Kenker (UIUC) CRQ Redux Braga / 34

33 Simulatin Settings II Y Y x x Rger Kenker (UIUC) CRQ Redux Braga / 34

34 Simulatins II-A Intercept Slpe Bias MAE RMSE Bias MAE RMSE Prtny L n = n = n = Prtny Q n = n = n = Peng-Huang L n = n = n = Peng-Huang Q n = n = n = Pwell n = n = n = GMLE n = n = n = Cmparisn f Perfrmance fr the Cnstant Censring, Heterscedastic Cnfiguratin Rger Kenker (UIUC) CRQ Redux Braga / 34

35 Simulatins II-B Intercept Slpe Bias MAE RMSE Bias MAE RMSE Prtny L n = n = n = Prtny Q n = n = n = Peng-Huang L n = n = n = Peng-Huang Q n = n = n = Pwell n = n = n = GMLE n = n = n = Cmparisn f Perfrmance fr the Variable Censring, Heterscedastic Cnfiguratin Rger Kenker (UIUC) CRQ Redux Braga / 34

36 Cnclusins Simulatin evidence cnfirms the asympttic cnclusin that the Prtny and Peng-Huang estimatrs are quite similar. Rger Kenker (UIUC) CRQ Redux Braga / 34

37 Cnclusins Simulatin evidence cnfirms the asympttic cnclusin that the Prtny and Peng-Huang estimatrs are quite similar. The martingale representatin f the Peng-Huang estimatr yields a mre cmplete asympttic thery than is currently available fr the Prtny estimatr. Rger Kenker (UIUC) CRQ Redux Braga / 34

38 Cnclusins Simulatin evidence cnfirms the asympttic cnclusin that the Prtny and Peng-Huang estimatrs are quite similar. The martingale representatin f the Peng-Huang estimatr yields a mre cmplete asympttic thery than is currently available fr the Prtny estimatr. The Pwell estimatr, althugh cnceptually attractive, suffers frm sme serius cmputatinal difficulties, impses strng data requirements, and has an inherent asympttic efficiency disadvantage. Rger Kenker (UIUC) CRQ Redux Braga / 34

39 Cnclusins Simulatin evidence cnfirms the asympttic cnclusin that the Prtny and Peng-Huang estimatrs are quite similar. The martingale representatin f the Peng-Huang estimatr yields a mre cmplete asympttic thery than is currently available fr the Prtny estimatr. The Pwell estimatr, althugh cnceptually attractive, suffers frm sme serius cmputatinal difficulties, impses strng data requirements, and has an inherent asympttic efficiency disadvantage. Quantile regressin prvides a flexible cmplement t classical survival analysis methds, and is nw well equipped t handle censring. Rger Kenker (UIUC) CRQ Redux Braga / 34