Survival Analysis. STAT 526 Professor Olga Vitek

Size: px

Start display at page:

Download "Survival Analysis. STAT 526 Professor Olga Vitek"

Aron Neal
6 years ago
Views:

1 Survival Analysis STAT 526 Professor Olga Vitek May 4,

2 Survival Data and Survival Functions Statistical analysis of time-to-event data Lifetime of machines and/or parts (called failure time analysis in engineering) Time to default on bonds or credit card (called duration analysis in economics) Patients survival time under different treatment (called survival analysis in clinical trial) Event-history analysis (sociology) Why a special topic on survival analysis? Non-normal and skewed distribution Need to answer questions related to P (X > t + t 0 X t 0 ) Censored/truncated data Here we only focus on right-censored data 9-1

3 Survival Function Continuous survival time T its probability density function is f(t) its cumulative probability function is F (t) F (t) = P (T t) = t 0 f(s)ds The survival function of T is S(t) = P (T > t) = 1 F (t) also called survival rate steeper curve shorter survival S (t) = f(t) Mean survival time 0 0 µ = E{T } = tf(t) dt = 0 0 [ t ] = dx d[1 S(t)] = = 0 S(x) dx t df (t) = t d[1 S(t)] [ 0 ] d[1 S(t)] dx 0 x 9-2

4 Hazard Function The hazard function of T is λ(t) = lim t 0 P (t T < t + t T t) t proportion of subjects with the event per unit time, around time t; λ(t) 0 measure of proneness to the even as function of time λ(t) f(t): λ(t) is conditional on survival to t Relates to the survival function F (t + t) F (t)/ f(t) λ(t) = lim S(t) = t 0 t S(t) = (t) S S(t) = d dt log S(t) (negative slope of log-survival) The cumulative hazard function of T is defined as Λ(t) = t 0 λ(s)ds Λ(t) = log S(t) S(t) = exp{ Λ(t)} λ(t), Λ(t) or S(t) define the distribution 9-3

5 Parametric Survival Models Exponential distribution: λ(t) = ρ, where ρ > 0 is a constant, and t > 0 S(t) = e ρt, f(t) = S (t) = ρe ρt Weibull distribution: λ(t) = λp(λt) p 1 ; λ, p > 0 are constants, t > 0 S(t) = e (λt)p, f(t) = S (t) = e (λt)p λp(λt) p 1 is the Exponential distribution when p = 1 Gompertz distribution: λ(t) = αe βt ; α, β > 0 are constants, t > 0 is the Exponential distribution when β = 0 Gompertz-Makeham distribution: λ(t) = λ + αe βt ; α, β > 0 are constants, t > 0 adds an initial fixed component to the hazard 9-4

6 Non-Parametric Survival Models Approximate time by discrete intervals f(x i ) = { P {T = xi } 0 otherwise S(x i ) = j: x j t f(x j) λ(x i ) = f(x i ) / S(x i ) After substitution: f(x i ) = λ(x i ) i 1 j=1 (1 λ j) S(x i ) = i 1 j=1 (1 λ j) S(t) = j:x j <t (1 λ j), t > 0 9-5

7 Right-Censored Data Survival time of i-th subject T i, i = 1,, n. Censoring time of i-th subject C i, i = 1,, n. Observed event for ith subject: Y i = min(t i, C i ), δ i = 1 {Ti C i } = { 1, if Ti C i 0, if T i > C i Data are reported in pairs (Y i, δ i ) = (observedtime, hasevent) Assumption: C i are predetermined and fixed, or C i are random, mutually independent, and independent of T i 9-6

8 Non-parametric Estimation of Survival Function And Comparison Between Groups With Censoring 9-7

9 Example: Remission Times of Leukaemia Patients 21 leukemia patients treated with drug (6-mercaptopurine) 21 matched controls, no covariates. > library(mass) #Described in Venable & Ripley, Ch.13 > data(gehan) > gehan pair time cens treat control MP control MP control MP control MP control MP control MP control MP control MP control

10 Kaplan-Meier Estimator of Survival Function Also called product limit estimator Re-organize the data Distinct Event Times t 1 t i t k # of Events d 1 d i d k # Survivors Right Before t i n 1 n i n k Kaplan-Meier estimator Ŝ(t) : Estimate P (T > t 1 ) by ˆp 1 = (n 1 d 1 )/n 1 Estimate P (T > t i T > t i 1 ) by ˆp i = (n i d i )/n i, i = 2,, k If t i 1 < t < t i, S(t) = P (T > t) = P (T > t 1 )P (T > t T > t 1 ) = i 1 P (T > t 1 ){ P (T > t m T > t m 1 )}P (T > t T > t i 1 ) m=2 Estimate S(t) by Ŝ(t) = Ŝ(t i 1 ) n i d i n i (Kaplan & Meier, 1958, JASA) = i:t i t n i d i n i, 9-9

11 Non-Parametric Fit of Survival Curves Represent data in the censored survival form create a data structure Surv + represents censored time > library(survival) > Surv(gehan$time, gehan$cens) K-M estimation of survival function subset of results for the treatment group: > fit <- survfit(surv(time, cens) ~ treat, data=gehan) > summary(fit) treat=6-mp time n.risk n.event survival std.err lower95%ci upper95%ci

12 Interval K-M Estimator of Survival Function Direct interval estimation of S(t) The variance of Ŝ(t) can be estimated by var(ŝ(t)) = [Ŝ(t)] 2 d i n i (n i d i ) i:t i t This is Greenwood s formula (Greenwood, 1926) CI for S(t) is Ŝ(t) ± 1.96 var[ŝ(t)], S(t) [0, 1], but var(ŝ(t)) may be large enough that the usual CI will exceed this interval Interval estimation using log S(t) (preferred) The variance of log Ŝ(t) can be estimated as d i var[ log Ŝ(t) ] = n i (n i d i ) i:t i t Construct the CI for log S(t) [L, U] = log Ŝ(t) ± 1.96 var[log Ŝ(t)], For CI of S(t), transform the limits with [e L, e U ]. 9-11

13 Estimate the Cumulative Hazard Function Λ(t) = log S(t) Using the Kaplan-Meier estimator of Ŝ(t): ˆΛ(t) = log Ŝ(t) Alternatively, the Nelson-Aalen Estimator: Λ(t) = V ar( Λ(t) = { 0, if t t1 i:t i t d i n 2 i:t i t i d i n i, if t t i better small-sample-size performance than based on the K-M procedure useful in comparing the fit of a parametric model to its non-parametric alternative 9-12

14 Non-Parametric Fit of Survival Curves >plot(fit, conf.int=true, lty=3:2, log=true, xlab="remission (weeks)", ylab="log-survival", main="gehan") Plot curves and CI; default CI are on the log scale log=true plots y axis (i.e. S(t)) on the log scale (i.e. y axis shows the hegative cumulative hazard) Gehan Survival control 6 MP Remission (weeks) 9-13

15 Log-Rank Test for Homogeneity Non-parametric test Compare two populations with hazard functions λ i (t), i = 1, 2. Collect two samples from each population. Construct a pooled sample with k distinct event times Distinct Failure t 1 t i t k Time Pool # of Failures d 1 d i d k Sample # survivors n 1 n i n k right before t i Sample # of Failures d 11 d 1i d 1k 1 # survivors t i n 11 n 1i n 1k right before t i Sample # of Failures d 21 d 2i d 2k 2 # survivors n 21 n 2i n 2k right before t i Note d i = d 1i + d 2i, n i = n 1i + n 2i, i = 1,, k Test the hypotheses vs. H 0 : λ 1 (t) = λ 2 (t), t τ H a : λ 1 (t) λ 2 (t) for some t τ 9-14

16 Log-Rank Test for Homogeneity: Procedure Consider [t i, t i + ) for small Sample Sample Pooled 1 2 Sample # failures d 1i d 2i d i # survivors at t i n 1i d 1i n 2i d 2i n i d i # of survivors t i n 1i n 2i n i right before t i If λ 1 (t) λ 2 (t) around t i, there should be association between sample and event in this 2 2 table d 1i (n i, d i, n 1i ) H 0 Hypergeometric(n i, d i, n 1i ): ( ) ( di ni d i x n 1i x P (d 1i = x n i, d i, n 1i ) = = E[d 1i n i, d i, n 1i ] = d in 1i n i Define U = [ ] k i=1 d 1i d in 1i n i ( ni n 1i ) ) E[U] = 0, var(u) = k i=1 U/ var(u) 1/2 H 0, asy N(0, 1) d i n 1i (n i d i )(n i n 1i ) n 2 i (n i 1) 9-15

17 Log-Rank Test in R (Venable & Ripley Sec.13.2) >?survdiff > survdiff(surv(time,cens)~treat,data=gehan) N Observed Expected (O-E)^2/E (O-E)^2/V treat=6-mp treat=control Chisq= 16.8 on 1 degrees of freedom, p= 4.17e-05 Conclusion: Reject H 0 Warning: this test does not adjust for covariates, and may often be inappropriate Appropriate for this study, since the individuals are matched pairs 9-16

18 Parametric Estimation of Survival Function and Comparison Between Groups 9-17

19 The Likelihood Function The likelihood of i-th observation (y i, δ i, x i ): { f(yi X i ), if δ i = 1, (not censored) S(y i X i ), if δ i = 0, (right censored) The likelihood of the data is L(β) n i=1 { [f(yi X i )] δ i[s(y i X i )] 1 δ i} The inference approaches based on the likelihood function can be applied. See chapter by Lee on ML parameter estimation Use AIC or BIC to compare nested models. Graphical check of model adequacy Plot Λ(t) vs. t for exponential models; Plot log( Λ) vs. log(t) for Weibull models; Can also plot deviance residuals. 9-18

20 Accounting for Covariates: Models for Hazard Function General form: λ(t) = λ 0 (t)e x β λ 0 (t): the baseline hazard e x β : multiplicative effect, independent of time Implications for the Survival Function: S(t) = e Λ(t) = e t 0 λ(s) ds = e Λ 0 (t)e x β log Λ(t) = log[ log S(t)] = log Λ 0 + x β If the model is appropriate, a plot of log[ log S(t) KM ] for different groups yields roughly parallel lines 9-19

21 Accounting for Covariates: Models for Hazard Function Exponential distribution, no predictors: λ(t) = ρ - constant hazard function S(t) = e ρ t - survival function Exponential distribution, one predictor: λ(t) = e β 0+β 1 X = e β 0 eβ 1X, e β 0 plays the role of λ 0, i.e. the constant baseline hazard Weibull distribution, no predictors: λ(t) = λ p p t p 1 - hazard function S(t) = e (λ t)p - survival function Weibull distribution, one predictor: λ(t) = p t p 1 e (β 0+β 1 X) p = p t p 1 e βp 0 e pβ 1 X e β 0+β 1 X plays the role of λ in overall hazard p t p 1 e βp 0 is the baseline hazard e β 0 plays the role of λ in the baseline hazard 9-20

22 More on Proportional Hazard Assumption of proportional hazard implies: The hazard ratio for two subjects i and j with different covariates, at a gven time is λ i (t) λ j (t) = λ 0(t) λ 0 (t) eβ0+β1xi e β 0+β 1 X j = e (X i X j ) β constant over time only function of covariates Can graphically verify the assumption of proportional hazard. Check for parallel lines of Ŝ KM (t) on the complementary log-log scale For the gehan dataset: > plot(fit, lty=3:4, col=2:3, fun="cloglog", xlab="remission (weeks)", ylab="log Lambda(t)") > legend("topleft", c("control", "6-MP"), lty=4:3, col=3:2) 9-21

23 Verify the Assumption of Proportional Hazard: Gehan Study log Lambda(t) control 6 MP Remission (weeks) Good agreement with the additive structure on the log-log scale The assumption of proportional hazard is plausible 9-22

24 Accounting for Covariates: Models fot Survival Function Also called Accelerated Failure Models S(t) = S 0 (t e x β ) S 0 is the baseline survival function Covariates accelerate or contract the time to event Survival time T 0 = T e x β has a fixed distribution log T 0 = log T + X β, log T = log T 0 X β Weibull (and its special case Exponential) are the only distributions that can be simultaneously (and equivalently) specified as proportional hazard and accelerated failure models. 9-23

25 Accelerated Failure: Exponential Exponential distribution: λ(t) = ρ, S(t) = e ρ t S 0 (t) = e t - survival of the standard exponential Effect of the covariates: S(t) = S 0 (t e β 0+β 1 X ), e β 0+β 1 X = ρ If T 0 exp(1), and T are the observed times T 0 = T e β 0+β 1 X log T 0 = log T + β 0 + β 1 X, and log T = β 0 β 1 X + log T 0 In R: log T = β 0 + β 1 X + σ log ε, σ is a scale parameter fixed at σ = 1 ε Exp(1) use opposite sign of ˆβ to estimate survival: Ŝ(t) = S 0 (t e ˆβ 0 ˆβ X 1 ) 9-24

26 Accelerated Failure: Exponential > fit.exponential <- survreg(surv(time,cens) ~ treat, data=gehan, dist="exponential") > summary(fit.exponential) Value Std. Error z p (Intercept) e-28 treatcontrol e-04 Scale fixed at 1 Exponential distribution Loglik(model)= Loglik(intercept only)= Chisq= on 1 degrees of freedom, p= 4.9e-05 Parameter interpretation: β treatcontrol = 1.53 > 0 time until remission is longer for controls Predicted survival above T = 10 for trt: exp(-10*exp(-3.69))= comparable to ŜKM trt (10) =

27 Accelerated Failure: Weibull Weibull distribution: λ(t) = λ p p t p 1 - hazard function S(t) = e (λ t)p - survival function Can be viewed as the survival function of the exponential random variable T = T p Exp(λ p ) Effect of the covariates: Identical to the exponential: S(t ) = S 0 (t e β 0+β 1 X ), e β 0+β 1 X = λ p If T 0 exp(1), and T are the observed times T 0 = T e β 0+β 1 X log T 0 = log T + β 0 + β 1 X Returning to the original notation T = T p : log T 0 = p logt + β 0 + β 1 X log T = 1 p β 0 1 p β 1X + 1 p log T

28 Accelerated Failure: Weibull In R Weibull is the default distribution: log T = β 0 + β 1 X + σ log ε, σ is a scale parameter, σ = 1 p, ε Exp(1) use ˆβ/ˆσ to estimate survival: Ŝ(t) = S 0 (t 1/σ e ˆβ 0 /σ ˆβ /σx 1 ) > fit.weibull <- survreg(surv(time,cens)~treat, data=gehan) > summary(fit.weibull) Value Std. Error z p (Intercept) e-44 treatcontrol e-05 Log(scale) e-02 Scale= Weibull distribution Loglik(model)= Loglik(intercept only)= Chisq= on 1 degrees of freedom, p= 9.3e-06 Predicted survival above T = 10 for trt: exp(-10^(1/0.732)*exp( /0.732))= comparable to ŜKM trt (10) =

29 Variable Selection and Prediction Compare models with and without pair as blocking factor anova( survreg(surv(time,cens) ~ treat, data=gehan), survreg(surv(time,cens)~factor(pair)+treat, data=gehan)) Terms Res.Df -2*LL TestDf Deviance P(> Chi ) treat NA NA NA (pair)+treat (pair) Prediction for the median survival On the linear predictor scale > fit.weibull.nopairs <- survreg(surv(time,cens)~treat, data=gehan) >?predict.survreg > pred.contr <- predict(fit.weibull.nopairs, data.frame(treat= control ), type="uquantile", p=0.5, se=true) On the survival function scale > exp( c(l=pred.contr$fit - 2*pred.contr$se.fit, U=pred.contr$fit + 2*pred.contr$se.fit) ) L.1 U

30 Cox Proportional Hazards Model A semi-parametric approach 9-29

31 Cox Proportional Hazards (Relative Risk) Model Assume: C i and T i conditionally independent, given X i. Observe k distinct exact failure times (i.e. δ i = 1); t 1 < t 2 < < t k. No ties in observed exact failure times (i.e., if δ i = δ j = 1, then y i y j for i j). Define the risk set at time t R(t) = {i : y i t} = {individuals alive right before time t} Cox (1975) considered the conditional probability Probability of event in a small interval around t j, given that the individual is in the risk set P (ind. i failed at [t j, t j + ) i R(t j )) λ(t j ) k R(t j ) λ(t j) 9-30

32 Partial Likelihood Approach to Estimate β (Cox, 1975) With covariate X i (t), assume a proportional hazards model λ(t) = λ 0 (t) exp X i (t) β(t) where λ 0 (t) is a baseline hazard and e X i β(t) is the relative risk. Treating λ 0 (t) as a nuisance parameter, one may estimate β(t) by maximizing the partial likelihood L(β) = j exp X i (t j ) β(t j ) l R(t j ) exp X l(t) β(t j ), where the i(j)th item fails at t j and R(t j ) is the risk set at t j. Conditional on the history up to t j and the fact that one item fails at t j, each term within the product is proportional to the likelihood of a multinomial model. λ 0 (t) is not specified, which is the non-parametric part of the model. X l (t) β(t j ) is the parametric part of the model. 9-31

33 Partial Likelihood Approach to Estimate β (Cox, 1975) The partial likelihood function by Cox (1975) is { } n obs e X β i n L(β) = k R(t j ) β = e X β δi i ex k i=1 i=1 k R(y i ) ex k β Cox argued that the partial likelihood has all properties of an usual likelihood, e.g., maximizing it for an optimal β = maximum partial likelihood estimator (MPLE) ˆβ Breslow s Estimator of the Baseline Cumulative Hazard Rate: ˆΛ 1 0 (t) = i R(t) ˆβ = ex j j:t j t i:y i t δ i j R(y i ) ex j ˆβ There are different ways to relax Assumption 3 to handle tied failure times, e.g., exact method, Efron s method, and Breslow s method. Method R Options Comment Exact method="exact" accurate, long Efron s method="efron" approximate, better Breslow s method="breslow" approximate 9-32

34 Cox Proportional Hazard Use exact method to account for ties ˆβ > 0 as expected > fit.cox1 <-coxph(surv(time,cens)~treat, method="exact", data=gehan) > summary(fit.cox1) n= 42, number of events= 30 coef exp(coef) se(coef) z Pr(> z ) treatcontrol *** exp(coef) exp(-coef) lower.95 upper.95 treatcontrol Rsquare= (max possible= 0.98 ) Likelihood ratio test= on 1 df, p=5.544e-05 Wald test = on 1 df, p= Score (logrank) test = on 1 df, p=4.169e

35 Cox Proportional Hazard Add pair as block > fit.cox2 <- coxph(surv(time,cens)~treat+factor(pair), method="exact", data=gehan) > summary(fit.cox2) coef exp(coef) se(coef) z Pr(> z ) treatcontrol e-06 *** factor(pair) ** factor(pair) *... Rsquare= (max possible= 0.98 ) Likelihood ratio test= on 21 df, p= Wald test = on 21 df, p= Score (logrank) test = on 21 df, p= LR test compares log(partial likelihoods), but the test has similar properties > anova(fit.cox1, fit.cox2, test="chisq") Analysis of Deviance Table Cox model: response is Surv(time, cens) Model 1: ~ treat Model 2: ~ treat + factor(pair) loglik Chisq Df P(> Chi )

36 Visualize Model Fit Survival curve and CI for an individual with average covariates K-M curves refer to unadjusted populations, Cox curves refer to an average patient can make survival curves for the Cox model for specific values of covariates by providing newdata >plot(survfit(fit.cox1),lty=2:3,xlab="remission (weeks)", ylab="survival", main="gehan", cex=1.5) Gehan Survival Remission (weeks) 9-35

37 Model Diagnostics Cox-Snell residuals are most useful for examining the overall fit of a model r c,i = log[ŝ(y i x i )] = ˆΛ(y i x i ) = ˆΛ 0 (y i )e X i ˆβ Λ 0 (t) is estimated by the Breslow s estimator. If the estimates ˆΛ 0 (t) and ˆβ were accurate, {(r c,i, δ i ), i = 1,, n} are right-censored observations of Exponential(1). For the right-censored data {(r c,i, δ i ), i = 1,, n}, construct the Nelson-Aalen estimator Λ(t) and plot Λ(t) vs. t. Martingale residuals can be used to determine the functional form of a covariate r m,i = δ j r c,j Fit the Cox model with all covariates except the one of interest, and plot the martingale residuals against the covariate of interest. Deviance residuals are approximately symmetrically distributed about zero and large values may indicate outliers. r d,i = sign(r m,i ) 2[r m,i + δ i log(r c,j )] 9-36

Survival Analysis. Stat 526. April 13, 2018

Survival Analysis. Stat 526. April 13, 2018 Survival Analysis Stat 526 April 13, 2018 1 Functions of Survival Time Let T be the survival time for a subject Then P [T < 0] = 0 and T is a continuous random variable The Survival function is defined