CONTINUOUS TIME MULTI-STATE MODELS FOR SURVIVAL ANALYSIS. Erika Lynn Gibson

Transcription

1 CONTINUOUS TIME MULTI-STATE MODELS FOR SURVIVAL ANALYSIS By Erika Lynn Gibson A Thesis Submitted to the Graduate Faculty of WAKE FOREST UNIVERSITY in Partial Fulfillment of the Requirements for the Degree of MASTER OF ARTS in the Department of Mathematics December 2008 Winston-Salem, North Carolina Approved By: James L. Norris, Ph.D., Advisor Examining Committee: Robert Plemmons, Ph.D., Chairperson Kenneth Berenhaut, Ph.D.

2 Table of Contents Acknowledgments Abstract ii iii Chapter 1 Introduction Chapter 2 General Survival Analysis Chapter 3 Multi-State Review Chapter 4 Homogeneous Markov Models Chapter 5 Homogeneous Semi-Markov Models Chapter 6 Non-Homogeneous Markov Models Chapter 7 More General Multi-State Models Chapter 8 Conclusion Appendix A Data Bibliography Vita i

3 Acknowledgments I would like to thank Dr. Norris for his time, patience, and guidance through this process. Also thank you to Dr. Berenhaut and Dr. Plemmons for reading my thesis. A special thanks to my husband Will for all the sacrifices he made to allow me to go to school again. Thanks to my two boys, Colin and Owen, for the many times putting up with a stressed out mommy. Last of all, I would like to thank Wake Forest for allowing me this opportunity and letting me go part-time so I could take care of my family. ii

4 Abstract Erika L. Gibson Continuous Time Multi-State Models for Survival Analysis Thesis under the direction of James L. Norris, Ph.D. The purpose of this paper is to discuss the survival analysis techniques for continuous time multi-state models. It explores homogeneous Markov models, homogeneous semi-markov models, non-homogeneous Markov models, and general models. It also discusses the likelihoods, hazards, and probabilities for each model. This paper examines techniques that deal with right and interval censoring data in continuous situations. Last of all, this paper deals with the issue of observable transitions and non-observable transitions. iii

5 Chapter 1: Introduction In medical research, the development of survival analysis has had tremendous effect on the understanding of data. The issues with right censored data were immediately apparent. Kaplan-Meier and Nelson-Aalen developed non-parametric maximum likelihood estimators of the survival function. Also, work was done on discrete time survival analysis. However, most medical studies do not adhere to these strict assumptions. Most medical events do not occur in discrete time and because the subject is often not being observed constantly, the event times may only be known to happen in a certain interval. Thus work in continuous time models with interval censoring was developed. The aim of this paper is to consider continuous time multi-state models in survival analysis. Multi-state models are a sub-discipline of survival analysis. They are the most common models for describing longitudinal failure time data. Multi-state models are models with definitive states that an individual in a study may visit. The most convenient model in multi-state models is a Markov model. This paper will discuss the developments in homogeneous Markov models, semi-homogeneous Markov models, non-homogeneous Markov models, and more general multi-state models. Homogeneous Markov models are the most desirable models since the relationship between transition probabilities and transition intensities are easy to determine. When considering different multi-state models, there are various related tools that are used to evaluate the data. Likelihoods, intensities or hazards, transition probabilities, covariates, and distributions are a few of the tools that are used to model the data that are discussed in this paper. This paper will first give general information on 1

6 2 survival analysis, then a brief multi-state review before exploring multi-state models in survival analysis. There are several ways a set of data can be evaluated. The way in which the study is structured, called state structure, determines how to analyze the data. Below are a set of examples that can be set up in different structures and evaluated using the procedures in this paper: heart data that evaluates whether a heart transplant extends a patient s life twin data that determines if the death of one twin increases the probability of the death of the second twin pregnancy data that determines the probability of a woman having another baby multiple sclerosis data that models the survival rate of person as they progress in the disease HIV data that tracks the effects of a certain drug on the progression of the disease. In this paper, the Stanford Heart data and the Albuminuria progression data are used as illustrations in chapters 5 and 7 respectively. The Stanford Heart data can be found in Appendix A.

7 Chapter 2: General Survival Analysis Survival analysis is the study of the time to some event, usually failure time. This ranges from machine failure, heart attack, death, onset of a disease, or birth of a child. Distribution of the survival times, sometimes called event times or failure times, is studied, but most often it is the effect of explanatory variables on the survival that is the focus [10]. Some individuals will not reach a failure before the end of the study, may leave the study early, or be lost to followup. In these cases, they will be right censored. When an individual is right censored at time t, it is only known that their time to an event lies in the interval (t, ). Let there be n subjects, each of which might be right censored, then let T ı and C ı be the survival time and censoring time for the ıth subject. Then the observed data can be represented by X ı = min(t ı, C ı ) and V ı = 1(T ı C ı ). Let the risk set R(t) = {ı : X ı t} be the set of individuals at risk of failure at time t, and r(t) = card{r(t)} be the number of these individuals. The survival function is the probability of surviving beyond time t and can be denoted as S(t) = P (T > t) (2.1) where T is the total lifetime. The survival function is the complement of the cumulative distribution function, S(t) = 1 F (t) = 1 P (T t). Let T be a continuous random variable, then the survival function is also the integral of the probability density function, S(t) = P (T > t) = t f(u)du. (2.2) 3

8 4 Therefore, f(t) = ds(t). (2.3) dt Then The hazard function, is the derivative of F conditioned on T > t namely, λ(t) = lim 0+ λ(t) = f(t) S(t) The cumulative integrated hazard function is then Then we have Λ(t) = t 0 S(t) = exp[ Λ(t)] = exp[ P {T t + T > t}. (2.4) d ln[s(t)] =. (2.5) dt λ(u)du = ln[s(t)]. (2.6) t 0 λ(u)du]. (2.7) The Kaplan-Meier estimator, called the product-limit estimator, is the nonparametric maximum likelihood estimator for the survival function under right censored data [10]: S(t) ˆ = t (j) t ( 1 d ) j, (2.8) r j where the t (j) are the jth ordered observed failure time, d j = #{ı : T ı = t (j), V ı = 1} is the number of individuals that failed at t (j), and r j = card{r(t (j) )}. Thus r j+1 = r j d j w j, where w j is the number of individuals censored between t (j) and t (j+1). Another estimator is the Nelson-Aalen estimator of the cumulative hazard function which is ˆΛ(t) = t (j) t d j r j. (2.9) The Kaplan-Meier estimator and the Nelson-Aalen estimator are closely related. The negative log of the Kaplan-Meier estimator approximates the Nelson-Aalen estimator

9 5 as follows, ln(ŝ(t)) = t (j) t ( ln 1 d ) j d j = r j r ˆΛ(t) (2.10) t (j) t j Let λ ı (t) be the hazard function of survival time T ı, and assume that S ı (t) is a continuous survival function. Then Cox s proportional hazard model incorporating time-dependent covariates vector z ı is λ ı (t) = exp{β T z ı (t)}λ 0 (t). (2.11) λ 0 (t) is the baseline hazard (for z ı = 0) and β is the vector of regression parameters describing the direction and magnitude of the dependence of the distribution of T ı on z ı [10]. The partial likelihood that Cox then proposed is lik(β) = exp{β T z ı (t ı )} [ k R(t ı) exp{βt z k (t ı )} ] (2.12) for inference about β and when λ 0 (t) is not known. The product of the partial likelihood is over all uncensored failure times t ı, and each term denotes the conditional probability that individual ı fails at time t ı, given that one individual from the risk set at time t ı fails at t ı [10]. Another model that is often used to estimate the transition probabilities in survival analysis is the accelerated life model. The accelerated life model is often used to compare two or more sets of data. In the accelerated life model the time scale is influenced by the covariates z ı where z ı does not depend on t [10]. Let ϕ ı = exp{β T z ı } then the accelerated life model is represented by λ ı (t) = ϕ ı λ 0 (ϕ ı t). (2.13) It is called the accelerated life model because if ϕ ı > 1 (i.e. β T z i > 0) then ϕ ı t > t, which means time is accelerated for person i. If ψ 0 = E(log T z ı = 0) and by taking the logs of both sides, we now have log T ı = β T z ı + ψ 0 + ε, (2.14)

10 6 where ε is a random variable with zero mean whose distribution is not dependent upon z [2]. A competing risks problem is where an individual in a study may encounter one of many possible failures and/or experience many types of failures [10]. For example, an individual s cause of death could be heart failure, cancer, automobile accident, etc. Obviously the individual can only experience one of these possible failures, but studying the rates of these occurrences is often valuable. Let s be the number of independent risks an individual may have, with hazard functions P (T t + t, D = ı T t) λ ı (t) = lim t 0 t (2.15) for ı = 1,..., s and with D being the specific failure type of the individual. The total hazard is then λ(t) = λ ı (t). Thus, the subdensity function of the improper distribution of the time to a failure type ı is [10] f ı (t) = exp{ t 0 λ(u)du}[λ ı (t)], (2.16) which is the probability of survival to time t times the hazard of failing by type i at time t. In most medical studies, interval censoring tends to be one of the most natural forms of censoring. Interval censoring is when the failure time of an individual occurs between two consecutive observation times t ı and t ı+1, but the specific time is not known. When the observation times are the same for all individuals in the study, then grouped survival data can be used [10]. When the observation times are not all the same, which is usually the case, then other more labor-intensive methods are needed and the survival function is generally not able to be compiled in closed form.

11 Chapter 3: Multi-State Review For each study, there may be multiple state structures possible. Choosing the best state structure can simplify calculations, make assumptions more visible, and ideally make it a Markov process [7]. There are six standard state structures: alternating, bivariate, competing risks, disability, mortality, and recurrent events (see figures below). The mortality model consists of two states: alive and dead. The competing risk models contains the states alive and dead by cause n, n = 1,..., s. In the competing risk model, we can only describe the state that the event occurs in. The alternating model is used in a case where a person can get ill and well repeatedly (often called reversible diseases). Irreversible diseases, like MS, can be modeled by the disability model. This model allows forward movement (often called progressive models) in the states only. The bivariate model describes bivariate parallel data that contain the following four states: where both are healthy, individual 1 is the only one infected, individual 2 is the only one infected, and both are infected. Recurrent events is a model that has states that are the number of times the event has been repeated for example, the number of children birthed by a woman. Competing Risks 7

12 8 Mortality Alternating Recurrent Bivariate Disability A multi-state model is a stochastic process model that has a finite set S of states: {0,1,...,s}, such that at any given time t, it is in one of the given states. Transitions (or events) are the changing from one state to another. An absorbing state is a state that once entered cannot be left, for example, death. A state structure determines which states there are and which transitions are able to occur. Movement within the model is determined by the state structure. Uni-directional models only allow for

13 9 forward transitions. Once an individual leaves a state, it is not able to return to that state. In bi-directional models, an individual can return to a state provided that it does not go to an absorbing state. Figures of the state structure are often helpful. Multi-state models can be used to describe failure times of longitudinal data. These models are often seen in medical research. Healthy, infected, and death are a few possible states in a medical study. Collection of data on the occupation of the states can be at times incomplete. Many times observation are taken at certain intervals; thus observations of the time when an event occurs may not be precise. Then transition time is interval censored. Errors in measurement may also occur, which may cause misclassification errors. Much work has been done in the area of probability with Markov models. Markov models are helpful models that allow one to make long term predictions more easily. A Markov model uses only the present state to make predictions. Thus the future is conditionally independent in the hazard for transitions from the past given the present state. Hence for a Markov model, the intensity for transitioning from time t s state to state l is λ l (t X t ) = lim t 0 P {X t+ t = l X t }. (3.1) t For a homogeneous Markov process, this hazard function only depends on t through its state X t. A non-homogeneous Markov process depends on t beyond X t, but does not depend on past history. Past history can be states visited, how often a state has been visited, or the times to transitions. Thus the hazard function is independent of the time spent in the current state or even previous states. Markov extension models allow for the hazard functions to depend on the time spent in the current state. For Markov and non-markov processes, hazard functions allow for the prediction of the next transition. They do not allow for prediction of more than one transition at a time. To predict for more than one transition ahead, we must look at the transition

14 10 probabilities. The general transition probability function of being in state l at time v given the history through time t is P l (t, v) = P (X v = l X u, u [0, t]), (3.2) for v t. The transition probabilities depend on the state structure, possible transitions, and the forms of the transitional hazards [7]. In the simplest case, the probability of X being in state l at time v can be defined as P Il (v) = P (X v = l X 0 = I), (3.3) where X 0 = I denotes the initial state, I. Transition probabilities are used for Markov and non-markov processes, but are more easily determined for Markov processes. Markov transition probabilities do not depend on the past given the present. Evaluating at present time t, we have P l (t, v) = P r(x v = l X u, u [0, t]) = P r(x v = l X t ) (3.4) for v t. This equation becomes more clear when one considers a specific state, say m, at the present time. Then the transition probabilities become P ml (t, v) = P (X v = l X t = m) (3.5) for v t. This is simply Equation (3.4) when X t = m. Let Q(t) be the transition intensity matrix such that the off-diagonal elements are P ml (t, t + t) Q ml (t) = lim t 0 t λ ml (t) (3.6) which is Equation (3.1) with X t = m, while the diagonal elements are Q mm = l m λ ml λ mm. (3.7)

15 11 Equation (3.7) is the negative of the total hazard of moving from state m. An important relation is the Chapman-Kolmogrov relation, which in Markov processes yields P ml (t + t) = k S P mk (t)p kl (t, t + t). (3.8) This says that to be in state l at time t + t, you can be in any state k at time t and then be in state k at t in the future. Subtracting P ml (t) from both sides and dividing by t then letting t 0 and substituting, we now have dp ml (t) dt = P ml (t)( λ ll (t)) + k l P mk (t)(λ kl (t)). (3.9) These are forward difference differential equations for a fixed m [1]. We can get the backward difference differential equations for a fixed l by doing similar methods. The matrix version of Equation (3.9) is dp (t)/dt = P (t)[q(t)], (3.10) where P (t) is the s s probability transition matrix. Since the coefficients of Q(t) might not be a constant matrix, this equation is generally not solvable, though it is solvable in the homogeneous cases, where Q does not depend on t. For homogeneity, it is easy to show that P ml (t, v) = P ml (0, v t), (3.11) and Equation (3.9) can then be represented by the matrix exponential function, P (t, v) = exp{(v t)q} = Q r (v t) r r=0 r! P (v t), (3.12) where exp{(v t)q} is from the vector version of first-order differential equations and r=0 Q r (v t) r r! is a Taylor series expansion. Although estimating transitions hazards do have some advantages, estimating transition probabilities tend to be more useful because they can predict more than one

16 12 transition ahead at a time. In a parametric model, estimating transition probabilities involve substituting the estimated parameters. The Aalen-Johansen estimator (a generalization of the Kaplan-Meier estimator) can be used for non-parametric Markov models that do not have covariates. See chapters 4-7 for further discussion. How to deal with right censoring is a major issue that must be established within each study. The method that is generally the most accepted is called processdependent censoring. It is based on the history of the individual. Thus you can set a limit as to how many transitions must be made before it can be right censored or have it be censored after a determined amount of time with no events occurring. One drawback to this method is that at certain times, some of the hazard functions are not able to be estimated [7]. See chapters 4-7 for further discussion. Checking the validity of a multi-state model can be done in a number of ways. If the proposed model s fit to the data is not significantly different from that of a more general model, then the proposed model may be accepted over the more general one. Examples of generalizations include using a Weibull hazard to generalize a homogeneous Markov model, and using a Markov extension model to generalize a non-homogeneous Markov model[7]. See chapters 4-7 for further discussion.

17 Chapter 4: Homogeneous Markov Models Our homogeneous Markov multi-state models are time-continuous models for which the transition probability only depends upon the current state and has constant transition hazards for the state. In medical research, discrete observation times are generally used. The state that the patient is in at the observation is the only thing known. The researcher may know that there has been a transition to a new state, but does not know when in that interval of time it occurred. Thus we consider homogeneous Markov models with interval censoring. Suppose that there are k processes that move independently within states {1,..., s} in a homogeneous Markov process. Since the model is homogeneous, q ml (t) = q ml because it is independent of t. Let d 1, d 2,..., d s be the eigenvalues of Q and A be the matrix such that it s lth column is the eigenvector associated with the eigenvalue d l [11]. Thus we can now represent Equation (3.12) as a matrix extension singular value decomposition [8]: P (v t) = A diag(e d 1(v t), e d 2(v t),..., e ds(v t )) A 1. (4.1) Suppose the study design only examines the states that processes (e.g. patients) occupy at certain times. Specifically, let x j,1, x j,2,..., x j,n be the states that individual j occupies at times t j,1, t j,2,..., t j,n. Then the likelihood over all k processes (patients) is k n P xj,ı 1,x jı (t j,ı t j,ı 1 ). (4.2) j=1 ı=2 Oftentimes in application of these methods, there are measured covariates on each individual patient. Studying the effect of these covariates on intensities q ml 13

18 14 has garnered much attention [9]. Incorporating covariates, the transition intensities, based on Cox s proportinal hazards, can now be expressed as Q ml (z) = Q ml0 exp(β mlz) (4.3) for m l, where z is a w-dimensional vector of covariates, β ml is a vector of w regression parameters relating the instantaneous rate of transitions from state m to state l to the covariates z and Q ml0 represents the baseline intensity relating to the transition from state m to state l [11]. We can substitute Q ml (z) forq ml (t) in Equations (3.6) and (3.7) and P ml (t z) for P ml (v, t) in Equation (3.5). Covariates can also be time-dependent covariates. We can modify P ml (t v z) with P ml (t v z(v)) by assuming that the time-dependent covariates are constant between consecutive times v and t [11]. Therefore, giving us Q ml (t z(t)) = Q ml0 exp(β mlz(t)) (4.4) for m l. Assessing the fit of a model can be done in several ways. The assumption of time homogeneity can be tested by fitting another model with piecewise constant hazard and then comparing by the likelihood ratio test or by using local score tests [12]. To assess the sufficiency of a model, a goodness-of-fit test is generally used. Comparing the observed and expected values is the usual way of testing for goodness-of-fit [11]. This requires observing each individual at the same time points. In medical research, this can rarely be achieved. To get around this problem, we approximate observed and expected counts. We assume that the patient is in the initial state at the first observation and that if the patient is not observed at time t, then they are still in the initial state I. Patients who have entered absorbing states are still counted in the observed counts until the end of the study. The number of patients in state l at time t is the observed count for state l, O l (t). E Il (t) is the

19 15 expected counts for state l from initial state I, which is the product of the number of individuals under observation at time t whose initial state was I and the transition probability ˆP Il (t)(i = 1,..., s)[11]. The expected counts is then Ê l (t) = s Ê Il (t). (4.5) I=1 Note that Êl(t) is the product of the number of patients that were in the initial state and the transition probability p ml (t). Then the statistic is M(t) = s (O l (t) Êl(t)) 2, (4.6) Ê l (t) l=1 which has an asymptotic Chi-Square distribution with s 1 degrees of freedom.

20 Chapter 5: Homogeneous Semi-Markov Models It is often useful for a model to include the duration of time spent in a certain state before it transitions to another state. General Markov models are not appropriate since they are memoryless. In other words, they do not record the time spent in a certain state. Semi-Markov multi-state models are Markov extension models which allow for dependence on the time spent in the current state [5]. This dependence is often times called duration dependence. In this chapter, we examine homogeneous Semi-Markov models where the transition intensities do not depend on time. Let t be the time at which an individual enters a state m and let v be the time at which the individual leaves that state. Then d = v t and D is its corresponding random variable which does not depend on t by homogeneity. Then the pdf of the duration of time spent in state m before the individual directly transitions into state l is f ml (d) = lim d 0 Therefore, the duration survival function can be denoted and the hazard function [5] is P (d D < d + d X v = l, X t = m). (5.1) d S ml (d) = P (D > d X v = l, X t = m) = d f ml (u)du (5.2) λ ml (d) = P (d D < d + d D > d, X v = l, X t = m) lim d 0 d (5.3) = f ml(d) S ml (d) (5.4) = ln[s ml(d)]. d (5.5) 16

21 17 Distribution of dependence can be modeled in many different ways. We look at three parametric models, namely the exponential, Weibull, and generalized Weibull. In the exponential, (σ ml ) distribution the hazard functions are constant. Thus the hazard function of the duration dependence [6] is λ ml (d) = 1 σ ml, d 0, σ ml > 0. (5.6) The Weibull distribution has two parameters: let them be (σ ml, δ ml ). The hazard function of the Weibull distribution can be represented by ( ) δml 1 λ ml (d) = δ ml d δml 1, d 0, σ ml > 0, δ ml > 0. (5.7) σ ml For the generalized Weibull distribution, we want the hazard function to be able to fit a U shape. Thus with parameters (σ ml, δ ml, θ l ), the hazard function is ( λ ml (d) = 1 ( ) ) 1 δml 1 θ d ml ( ) δml 1 δml d 1 +, (5.8) θ ml σ ml σ ml σ ml d 0 σ ml > 0, δ ml > 0, θ ml > 0. The advantage of these distributions is that they are nested, and therefore, the Likelihood Ratio Statistics can be used to determine if it is advisable to use a larger number of parameters [6]. To be able to take covariates into consideration, the proportionality of duration hazard functions is commonly assumed [5]. Let z ml be the vector of covariates for the transition m l. Then the duration hazard function with covariates is a modification of Equation (2.10), namely λ ml (d z ml ) = λ 0,ml (d) exp{β T mlz ml (d)} (5.9) where λ 0,ml is the baseline hazard function for the transition from m l and β ml is the vector of regression parameters associated with z ml [6].

22 18 Parameter Variable ˆβ(SE) p β 1 Age (years) 0.031(0.015) < 0.05 β 2 Indicator 0.015(0.336) 0.96 β 3 Time since transplant (days).00014( ) 0.93 Table 5.1: Estimations of mortality of heart transplant data Due to interval censoring, let d 0 t j,i be the last duration observed before D tj,i and d 1 t j,i be the first duration observed after D tj,i i.e.(0 d tj,i 0 < D tj,i d tj,i 1 < ). Here, contrary to the previous homogeneous Markov processes, it is assumed that an individual does not have multiple transitions between observations (e.g. the individual does not leave m and return to m between two consecutive observations). Thus the individual was transition free at time d 0 t j,i and transitioned before d 1 t j,i [6]. Therefore, d 1 P (d 0 t j,i D tj,i < d 1 tj,i t j,i, X tj+1,i = l X tj,i = m) = P (X tj+1,i = l X tj,i = m) f ml (u)du d 0 t j,i (5.10) where the last integral can be expressed as S ml (d 0 t j,i ) S ml (d 1 t j,i ). (5.11) Note that the full likelihood is of the form n m P (d 0 t j,i D tj,i < d 1 t j,i, X tj+1,i l X tj,i = m). (5.12) i=1 j=0 In the Stanford heart transplant data, transplantation might lead to increased mortality directly after the operation. This could be caused from rejection, complications, or infections. Thus looking at the time since transplantation as a covariate may be very useful. Hougaard [7] ran data on this premise. He created a model such that the hazard without transplantation is exp(β 1 a)λ 0 (t) while the hazard with transplantation is exp(β 1 a + β 2 + β 3 (t u))λ 0 (t). Here a is age, and u is the time of transplantation. The results (see Table 5.1) found that adjusting for age, the transplantation did not increase the hazard (β 2 and β 3 are highly non-significant).

23 Chapter 6: Non-Homogeneous Markov Models Now we return to Markov models, but emphasize non-homogeneous Markov models where transition intensities can depend on time. Recall from Chapter 3 that Q(t) is the transition intensity matrix with off-diagonal elements Q ml (t) = lim t 0 P ml (t,t+ t) t λ ml (t) and diagonal elements Q mm = l m λ ml λ mm and the matrix equation dp (t)/dt = P (t)[q(t)]. Generally this model is not solvable for a non-homogenous Markov model, but an illness death model with only three states can be solvable. To consider an example, we must first review product-integration. Recall from chapter 2 that S(t) = P (T > t) = exp( t λ(u)du). Suppose that the (0, t) is split 0 into tm intervals each of length 1 m such that 1 m terms of product-integration P as follows: divides t, then we can express S(t) in S(t) = P (T > t) (6.1) = P (T > t T > 0) (6.2) = P (T > 1m, T > 2m,..., T > tmm ) T > 0 (6.3) = = tm 1 j=0 tm 1 j=0 lim m ( ( ( j P T m, j + 1 ) T > j )) m m [ ( ( j 1 P T m, j + 1 ) T > j )] m m tm 1 j=0 ( ( ( ) ( ))) j 1 1 λ m m (6.4) (6.5) (6.6). = P (0,t) (1 λ(u)du) (6.7) = P (0,t) [1 dλ(u)] (6.8) tm 1 [ ( ( ) ( ))] j + 1 j lim 1 Λ Λ. m m m (6.9) j=0 19

24 20 Now consider a three state, time non-homogeneous Markov, progressive model, with Λ 12, Λ 13, and Λ 23 denoting the cumulative intensity functions of the prospective transitions as shown in Figure 2. In this model, state 2 is the disease-related complication and state 3 is death. Let the data be interval censored when there is movement from state 1 to state 2. Then let E denote the exit time from state 1, T denote the time to death, C denote the time of right censoring, Z be the minimum of T and C, ξ = 1 if a known transition from state 1 to state 2 occurs, else ξ = 0. Similarly, ζ = 1 for known transitions from 1 3 or 2 3 and ζ = 0 otherwise. Also let L and R, with L R, be the consecutive observations that contains a known transition from state 1 2. Therefore, if ξ = 1, then the data is expressed by four random variables, namely L, R, Z, ζ, with 0 < L E R < Z T. Thus the transition from 1 2 is interval censored in the interval [L, R] with the transition from 2 3 is Z if ζ = 1 or right censored at Z if ζ = 0 [4]. Figure 2 Let D (ξ,ζ) be the set of data corresponding to (ξ, ζ). Define the following distribution functions: F E (e) = P (E e) (6.10) F 12 (e) = P (E e, ξ = 1) (6.11) F 13 (e) = P (E e, ξ = 0) (6.12) (6.13)

25 21 with corresponding densities denoted by the respective f. Thus F E (e) = F 12 + F 13. (6.14) When there is a transition from state 2 (ξ = 1), we want to examine the additional survival time. Let the additional survival be of V := T E with its survival function of Ḡ e (v) = P (V > v E = e, ξ = 1). (6.15) This is the additional survival function given that the transition from state 1 2 occurred at time e and the corresponding distribution function is G e (v) = 1 Ḡe(v). Also let F E (e) = 1 F (e) and define the following integrated hazards associated with E, E and V, respectively: Λ 12 (e) = Λ 13 (e) = Λ 23 (v) = (0,e] (0,e] (0,v] f 12 (u) du (6.16) F E (u ) f 13 (u) du (6.17) F E (u ) g e (u) du, (6.18) Ḡ e (u ) where g e is the derivative of G e. We will use the product integral representation to find the likelihood function, hence Ḡ e (v) = P (e,e+v] {1 dλ 23 (u)}. (6.19) Thus the likelihood is where L = L (1,1) L (1,0) L (0,1) L (0,0) (6.20) L (1,1) = = [P (L m < E m < R m )][P (R m < T m < t m)][λ 23 (t m )] (6.21) m D (1,1) P (e,rm](1 dλ 23 )df 12 (e)][p Bm (1 dλ 23 )][λ 23 (t m )](6.22) m D (1,1) [ [A m]

26 22 L (1,0) = = L (0,1) = L (0,0) = [P (L m < E m < R m )][P (T m > Cm)] (6.23) m D (1,0) P (e,rm](1 dλ 23 )df 12 (e)][p Bm (1 dλ 23 )] (6.24) m D (1,0) [ [A m] [f 13 (t m )] (6.25) m D (0,1) [ F (Cm)] (6.26) m D (0,0) with A m = [L m, R m ] and B m = (R m, T m ).

27 Chapter 7: More General Multi-State Models Arbitrary dependence on the history is generally assumed in a general multi-state model[7]. When the model is not a progressive model, there is very little that can be said due to the impossibility of writing general formulas for the transition intensities. Whenever possible, the model will be restructured to make a progressive model. When the model is a progressive model, then with some notation we can write a general formula for the likelihood as shown below. In contrast to earlier likelihoods of homogeneous and heterogeneous Markov models, here we know the times of all state changes. Let the state structure be a recurrent model with an absorbing state. For a generic individual, let E be the time-ordered events that have occurred and T be the corresponding transition times. Also let L be the stopping time (whether it be when censoring occurs or at the end of the study). Then with T j = (T 1,..., T j ), the individual s likelihood is E Tj L [ exp{ λ sj 1 (v T j 1 )dv}λ sj 1,j (T j T j 1 )] exp[ λ se (v T E )dv]. (7.1) T j 1 T E j=1 Consider instead the data as a process from time 0 to time L. By listing each time period as an observation in state s j 1, assuming left truncation at T j 1 until T j, at which time there is a transition into state s j (progressive) with K = E + 1, T K = L, and s K = 0. To include the last interval, we must add an indicator function, let it be D mlj = 1{s j 1 = m, s j = l}. Hence the individual s likelihood can now be written as K [λ ml (T j T j 1 ) D mlj exp{ 1[s j 1 = m] m l j=1 Tj T j 1 λ ml (T j T j 1 )dt}]. (7.2) The likelihood of Equation (7.2) is a simplification of Equation (7.1) since it does 23

28 24 not require knowledge about past history. Thus the individual s likelihood is E Tj L [ exp{ λ sj 1 (u)du}λ sj 1 s j (T j )] exp[ T j 1 λ se (u)du]. T E (7.3) j=1 Then combining the exponential functions and using the notation established in Equation (7.2), we now have K j=1 Tj λ sj 1 s j (T j ) 1{sj 0} exp{ λ sj 1 (u)du}. T j 1 (7.4) The likelihood is made of contributions corresponding to the hazard for each transition and for intervals without transitions there are exponential terms that give the integral of the hazard experienced by the process [7]. Rewriting in this form will allow the researcher to collect terms from different patients and repeated occurrences for a single patient. To do this, we will need to use the notation established in Equation (7.2) and let r m (t) be the cardinality of the individual s risk set at t; note that r m (t) is 1 when the individual is in state m at time t r m (t) = K 1{s j 1 = m, T j 1 < t T j }. (7.5) j=1 Thus the individual s likelihood can be written as K [{ (λ ml (T j )) D mlj } exp( L m l j=1 0 r m (u)λ ml (u)du)]. (7.6) This likelihood is the product over all possible transitions and the hazard for that transition is the only term involved at that point. To obtain the likelihood over all individuals, say i = 1, 2,..., n, then one adds an i subscript to each of the λ, D, and r terms and one takes the products of these likelihoods over all i. When duration times are known, as in semi-markov models, we can also find a general likelihood for progressive models. For a generic individual, let s j for j = 0, 1,..., h be the states arrived at for all previous transitions, i.e. transitions from

29 h. Define ŭ for j = 0, 1,..., h to be the duration times for the jth transition. Then for a general progressive model, the probability that a generic individual is in h, at time t is h [ 0<T 1 <...<T h <t j=1 Tj λ sj 1 s j (T j ŭ j 1 ) exp{ λ sj 1 (v ŭ j 1 )dv}] T j 1 t exp{ λ sh (v ŭ h )dv}. (7.7) T h To obtain the likelihood over all individuals, say i = 1, 2,..., n, then one adds an i subscript to each of the h, T j, and ŭ j terms and one takes the products of these likelihoods over all i. Conditionally simple processes are processes that are conditional on some random variable. In these models, the transition probabilities are calculated conditioned on the random variable and then the variable is integrated out to give the unconditional transition probabilities. Thus the model could have constant hazards conditionally but not unconditionally, or could be Markov conditionally but not unconditionally. Hidden states are models where it is impossible to observe all the states, but it is possible to observe a few superstates. Superstates are combinations of the individual states [7]. Hidden states can help one go from a progressive model to the original model. Many times, a hidden state model does not give interesting information. The transition probabilities will often times include absorbing states. In medical research, it often more interesting to study the non-absorbing states by themselves. In a hidden state model, the initial state must be known precisely [7]. Albuminuria is an example of a non-markov model. Hougaard model checked the non-markov model by estimating the hazards. He found that the macroalbuminuria could be quantitatively compared to a proportional hazards model with left truncation while the regression of the microalbuminuria confirmed that it could not be modeled by a Markov model. The albuminuria data uses a different state structure than the

30 26 Parameter Variable ˆβ(SE) p β 1 Age (years) 0.033(0.015) < 0.05 β 2 Diabetic retinopathy 0.062(0.25) < 0.02 β 3 Albumin (log base 10) 1.47(0.40) < Table 7.1: Estimations of regression coefficients for transitions from normo to micro six structures discussed in chapter 2. The state structure is illustrated in Figure (7.1), and it could be considered an extension of the disability model. Figure 7.1 This set of data can be analyzed in a variety of ways. In this section, we will cover a few of these methods as discussed in Hougaard [7]. First we will look at albuminuria progression where t is the time since baseline measurement. The transition hazards between the three non-death states can be defined in the following ways when there are no covariates present: normoalbuminuria ψ(t) microalbuminuria µ(t) macroalbuminuria. The covariates applied in this application are age, presence of diabetic retinopathy, and the log of albumin. Using these covariates and transition hazards, Hougaard obtained Table 7.1 s results for the normo to micro transition. Note that higher levels of the covariates significantly increase the chance of movement from normo to micro. Similarly, we can study the mortality where the covariates are time-dependent and describe the state which the patient presently occupies. Time is measured from

31 27 Parameter Variable ˆβ(SE) p β 1 Age (years) 0.080(0.022) < β 2 Micro 0.5(0.39) 0.20 β 3 Macro 1.41(0.36) < Table 7.2: Estimations of regression coefficients for death, by time since baseline with time-dependent covariates baseline (or from initial examination). The estimates of the regression coefficients are shown in Table (7.2). Adjusting for age, individuals who are in the macro state have a higher hazard of dying than do individuals in the normo (base) state since β 3 is significantly positive. This application illustrates that different hazards may have different timescales. It is important to consider the time scales as well as the covariates.

32 Chapter 8: Conclusion We found that continuous time multi-state models are very useful in survival analysis, especially in medical research. Right censoring and interval censoring are issues dealt with in the medical world all the time. Hence, it is imperative that multi-state models appropriately deal with censoring. Since most medical studies do not adhere to the assumption of discrete time, continuous time models have to be considered. Similarly, whether or not a transition is observable or non-observable must be looked at. Incorporating covariates may change the results or description of the data and therefore it must be decided how they will be dealt with at the onset of the study. We also concluded that choosing the right state structures can make computations easier. Homogeneous Markov models are the most desirable multi-state models since they are easiest to compute. Other models such as: homogeneous semi-markov models, non-homogeneous Markov models, and general models are often used, though not as desirable. In each of these models, different tools were used to describe the data, namely, likelihoods, hazards, and probabilities. It was evident that recurrent (progressive) models with an absorbing state are the easiest to determine the likelihoods for. 28

33 Appendix A: Data 29

34 30 Table 1 from [3] The data set of Table 1 has 103 patients that entered into the Stanford Heart transplant program and who were followed until death. Of the 103 patients, 69 of them received a heart transplant and 75 patients died.

35 Bibliography [1] Bhat, U.N., and Miller, G.K. Elements of Applied Stochastic Processes. Wiley, New Jersey, [2] Cox, D.R., and Oakes, D.A. Analysis of Survival Data. Chapman and Hall, New York, [3] Crowley, J., and Hu, M. Covariane Analysis of Heart Transplant Survivla Data. Journal of the American Statistical Association 1997;72: [4] Frydman, H. Nonparametric Estimation of a Markov Illness-death Process form Interval-cnesored Observations, with Applications to Diabetes Survival Data. Biometrika 1995; 82: [5] Foucher, Y., Magali,G., Soulillou, J.P., Daurès, J.P. A Semi-Markov Model for Multistate and Interval-censored Data with Mulitple Terminal Events. Application in Renal Transplantation. Statistics in Medicine 2007;26: [6] Foucher, Y., Mathieu, E., Saint-Pierre, P., Durand, J.F., Daurès, J.P. A Semi- Markov Model Based on Generalized Weibull Distribution with an Illustration for HIV Disease. Biometrical Journal 2005; 47: [7] Hougaard, P. Multi-State Models: A Review. Kluwer Academic Publishers, Boston, [8] Johnson, R.A., and Wichern, D.W. Applied Multivariate Statistical Analysis. Prentice-Hall, New Jersey, [9] Kalbfleisch, J.D., and Lawless, J.F. The Analysis of Panel Data Unde a Markov Assumption. Journal of the American Statistical Association 1985; 80:

36 32 [10] Oakes, D.A. A Biometrika Centenary:Survival Analysis. Biometrika 2001; 88: [11] Saint-Pierre, P., Combescure, C., Daurès, J.P. and Godard, P. The Analysis of Astma Control Under a Markov Assumption with the Use of Covariates. Statistics in Medicine 2003; 22: [12] Titman, A.C., and Sharples, L.D. A General Goodness-of-fit Test for Markov and Hidden Markov Models. Statistics in Medicine 2008; 27:

37 Vita Erika Lynn Gibson Born: April 12, 1981, Cleveland, Ohio Undergraduate Study: Piedmont Baptist College and Graduate School Winston-Salem, North Carolina B.S. Secondary Mathematics Education, 2002 Graduate Study: Wake Forest University Winston-Salem, North Carolina M.A. Secondary Mathematics Education, 2004 M.A. Mathematics, 2008 Awards and Honors: Graduate Fellowship, Wake Forest University, Masters Teacher Fellowship, Wake Forest University, Professional Societies: Mathematical Association of America, 2006-present American Mathematical Society, 2006-present 33

38 34 Association for Women in Mathematics, 2006-present North Carolina Council for Teachers of Mathematics, 2003-present National Council of Teachers of Mathematics, 2003-present Publication and Meeting Participation: Does the Integration of Concept Maps Activities Affect Student Achievement? Erika Gibson, Leah McCoy. Studies in Teaching, 70(5): , December Annual NCCTM, September Annual NCTM, Southeastern Section, October 2003.