arxiv: v1 [math.st] 19 Jan 2009

Transcription

1 A MARTINGALE APPROACH TO CONTINUOUS TIME MARGINAL STRUCTURAL MODELS arxiv: v1 [math.st 19 Jan 29 1 KJETIL RØYSLAND Abstract. Marginal structural models were introduced in order to provide estimates of causal effects from interventions based on observational studies in epidemiological research. We present a variant of the marginal structural strategy in continuous time using martingale theory and marked point processes. This offers a mathematical interpretation of marginal structural models that has not been available before. Our approach starts with a characterization of reasonable models of randomized trials in terms of local independence. Such a model gives a martingale measure that is equivalent to the observational measure. The continuous time likelihood ratio process with respect to these two probability measures corresponds to the weights in a discrete time marginal structural model. In order to do inference for the new measure, we can simulate sampling using the observed data weighted by this likelihood ratio. 1. Introduction We will consider the following scenario: A patient has a disease. In order to avoid an event as for instance death, a specific treatment can be given. The given treatment will typically depend on the patient s previous health condition. Typically, one could decide not to initiate treatment until the patient becomes sufficiently ill. We would like to give an estimate of the treatment effect. A natural way to do so is to do some sort of randomized trial. This means that we would have to set up an experiment on a group of patients where the treatment was initiated by randomization independently of each patient s previous health condition. However, such a study may not be available since it could require a lot of resources and the randomization could systematically impose treatments that were harmful to the patients. In order to take advantage of another type of data, we could try to base our estimates of the treatment effect on an observational study. Suppose we have observations of a group of patients where the given treatments were chosen by doctors. If we as a first attempt assumed that the standard Cox proportional hazards regression model for the event death was likely to described our scenario, then we could easily obtain an estimate of the hazard ratio between the group given treatment and the group not given treatment. However, such a naive analysis would most likely introduce a bias compared to the estimate based on the randomized trial. The reason is that the health condition of the patient will be a predictor of both treatment and death, i.e. it is likely to be a confounder [SHL + 5. Key words and phrases. Causal inference, marginal structural models, stochastic differential equations, equivalent martingale measures, graphical models. 1 Supported by the Research Council of Norway. Project: 1762/V3 1

2 2 KJETIL RØYSLAND We can easily imagine two opposite scenarios where this confounder would complicate estimates: Due to considerable costs, reduced life quality or possibly drug resistance, one could decide that the treatment should not be initiated until the patients are sufficiently ill. A naive marginal analysis based on data from an observational study would then quickly lead us to believe that the treatment effect was less than the true treatment effect. Conversely, if we for some reason decided only to initiate treatment for patients with a good health condition and not for the ones with a poor condition, then a naive marginal analysis would quickly lead us to believe that the treatment effect was better than the true treatment effect. In order to solve this problem, one might suggest to compute an estimate of the treatment effect conditionally on the health condition of the patient. However, in several situations, it is likely that the previous treatment will improve the patient s general health condition. This improvement will in itself typically postpone the time of death. The conditional effect estimate we described would only incorporate the direct treatment effect, not the effect that is due to an improvement of the patient s health condition. There is also another source of bias that we have to consider in order to lay hands on the causal effect of treatment, that is censoring. We assume that a patient may drop out of the study at a time and not return, i.e. we have right censoring. The given treatment, calender time and the patient s health condition might lead to such a drop out. If we do a naive analysis based on the patients that are still in the study, then we actually condition on a common effect of these variables, i.e we introduce a selection bias [HHDR4. Any reasonable quantification of the treatment effect should have an interpretation within a mathematical model. The multiplicative intensity model introduced by Aalen [Aal78 provides a framework for methods that are often used in survival analysis. It gives a neat interpretation of estimates from a Cox regression using martingale theory, [ABGK93. However, we are forced to move outside the standard Cox regression framework since we have to deal with the mentioned time dependent confounder effects due to a patient s underlying health condition. In order to provide a meaningful estimate of the treatment effect, with a simple interpretation, we could try to construct a rich model that also describes the dynamics of the underlying biological processes. Such mechanisms are likely to be very complicated and there might not be sufficient knowledge or data available. For this reason we could try to fit a marginal model of a suitable randomized trial for our scenario. This will be our strategy in what follows. One attempt to provide a marginal estimate of the causal treatment effect this way is due to Robins and is presented in [RHB. This method uses marginal structural models and relies on the additional assumption that there are no unmeasured confounders, i.e. there does not exist an unobserved process that is a predictor of a pair of observed processes in the marginal model. The idea is to apply some clever weights to the observations. This weighting results in a pseudo population that is different from the observed population. The key property of this pseudo population is that the selection bias and the treatment confounding due to the patient s health condition become negligible. Now, one can for instance proceed with a weighted Cox regression to obtain a marginal estimate of the effect of treatment. The method has been used several times on epidemiological studies. In [HBR it was used to estimate the effect of Zidovudine on the survival of

3 MARTINGALES AND MARGINAL STRUCTURAL MODELS 3 HIV-Positive men in the Multicenter AIDS Cohort Study. Moreover, the method was also used in [SHL + 5 to give an estimate of the hazard ratio for the effect of highly active antiviral treatment (HAART on progression to AIDS or death for HIV patients in Switzerland. The method suggested by Robins involves a discretization of time into successive intervals. Let {A k } k be random variables that reflect the treatment for times k = 1, 2,... and let {L k } k be random variables that denote various measurements reflecting the underlying health condition at times k = 1, 2,.... Moreover, let {a k } k denote the treatment given to a particular patient and let {l k } k denote the measurements for the particular patient. Let (1 AW t = k t P(A k = a k A j = a j, j < k P(A k = a k A j = a j, L j = l j, L k = l k, j < k Now, let {C k } k be a sequence of random variables that tell if a randomly chosen patient is censored at the k th time interval or not. Moreover, let (2 CW t = k t P(C k = C j =, A j = a j, j < k P(C k = C j =, A j = a j, L j = l j, L k = l k, j < k At time t this particular patient is given the weight AW t CW t. We will consider continuous time versions of the marginal structural models based on marked point processes. The idea is to characterize reasonable models of a randomized trial, the randomized trial measures, using martingale theory. This offers a mathematical interpretation of marginal structural models that has not been available before. We characterize a class of of reasonable models of randomized trials in terms of local independence. Such a model corresponds to a particular equivalent martingale measure. The continuous time likelihood ratio process between this measure and the observational probability measure corresponds to the weights in a discrete time marginal structural model. In order to do inference about this new measure, we can simulate samples using the observed data weighted by this likelihood ratio. Note that the fundamental mathematical concept of equivalent martingale measures is very central in mathematical finance where they are called risk-neutral measures. The concept is used in the pricing of derivatives. Moreover, if such a measure exists with respect to a model for a market, then the model will not allow arbitrages, see [PS6, [Øks3 and [KS98. Another approach to causal inference within our scenario is to use the so called structural nested models. These models were also introduced by Robins, see [Rob92 and [Rob98. J. Lok has developed continuous time versions of such models, using counting processes and martingale theory, see [Lok8. 2. Marked point processes and local independence 2.1. A class of marked point processes. We will formulate our continuous time model using marked point processes. In order to do this, we need some notation and definitions. More information about such processes and martingales can be found in [Bré81, [ABGK93 and [Pro5. We let (Ω, F, Q be a probability space where F F contains the possible information available at time. We let V = {1,..., n} and let N 1,..., N n form a multivariate counting process. Whenever X V, let {F X t } t denote the filtration in F that is generated by F and the processes N i

4 4 KJETIL RØYSLAND for i X. We let F t := Ft V assume that and let τ 1,...,τ n be F t -adapted stopping times and J i (t := N i (t I(s τ i ds is a local Q-martingale with respect to F t for every i V. Let λ 1,...,λ n be non negative and F t -predictable processes such that λ i(sds < for every t, P a.s. Moreover, assume that they are also compatible with the stopped Poisson processes in the sense that λ i (s; ω > if and only if s τ i for every i V. Finally, we assume that the process: R (t := ( τi exp 1 λ i (sds λ i (s Ni(s i V is a uniformly integrable Q-martingale with respect to F t. Note that by Proposition 1 in the Appendix, this is the case if: ( τi ( (3 E Q [exp λi (s 1 2 ds <. i V By the same proposition, P = lim R (tq t defines a unique probability measure on (Ω, F such that P Q and such that M i (t := N i (t s t λ i (sds is a local P-martingale with respect to F t for every i V. Now, λ i is the intensity of the counting process N i with respect to P and the filtration F t. We will think of the probability measure P as an observational measure, i.e. P gives the frequencies of observations of an observable system. Moreover, we will occasionally write E[ instead of E P [. If X V, and i X, there exists, by [Bré81, Theorem II T14, an Ft X- predictable non negative process λ i X such that E P [λ i (t F t = λ i X (t for every t and M i X (t := N i (t λ i X (sds is a P-martingale with respect to Ft X, P a.s. If X, Y V, then we will occasionally use the following abbreviations: F X,Y t := Ft X Y and λ i X,Y := λ i X Y. Let S be a finite set such that S S V. We will occasionally identify the set S S with an S-valued càdlàg process X : R + Ω S such that whenever (x 1, x 2 S S, then (4 N x1,x 2 (t; ω = #{s t X(s ; ω = x 1, X(s; ω = x 2 } Local independence. We will need a relation that tells if a process is irrelevant or not for the development of another process in the future. One could think of several such irrelevance relations, however we will consider one particular, namely local independence. The local independence relation was first defined for Markov chains in [Sch7 by Schweder. Aalen introduced the bi variate version for semi martingales in [Aal87. However, we will take advantage of the work of Didelez who has studied local independence for marked point processes thoroughly in several papers, see for instance [Did8, [Did7 and [Did.

5 MARTINGALES AND MARGINAL STRUCTURAL MODELS 5 Definition 1. Let X, Y, Z V be mutually disjoint and let T be an F X,Y,Z t -adapted stopping time. We say that the multivariate process N Y is locally independent of N X up to time T and given N Z if there exists an F Y,Z t -predictable process µ i, such that Ni T (t I(s Tµ i (sds is a local martingale with respect to F X,Y,Z t for every i Y. This is denoted by N X T N Y N Z. Otherwise, we say that N Y is locally dependent of N X before time T, given N Z. This is denoted by N X T N Y N Z. To achieve some intuition around this definition, let T be an F Y,Z t -adapted stopping time and assume that there exists a K > such that µ K. Now, we can check that N X T N Y N Z implies that 1 lim h h E[NT j (t NT j (t h FY,Z t h = µ j(t I(t T 1 = lim h h E[NT j (t NT j (s h FX,Y,Z t h for every t and j Y. This means that the infinitesimal expected change of the process N T X at time t is independent of the paths of the processes NT j for j X before t. Didelez also considered graphical models based on local independence, see [Did8. The proper objects to consider for doing this are the so called local independence graphs. Definition 2. Let V V. A directed graph (V, E such that (j, k / E = {j} T {k} V \ {j, k} is called a local independence graph for the counting process {N j } j V. These graphs will later prove to be very useful in order to represent complex models. 3. Models of generalized randomized trials 3.1. A model of the observable history of a patient. We will now formulate a model of a patient using jump processes as in 4. We will think of these processes as observations of a patient in an observational study. We consider the finite marked space V that consists of the the possible events. A simplified treatment strategy can be to give no treatment until the time T A when the treatment is initiated and continued constantly until the patient s death. However, we will assume that the treatment can change between several states. We let A = {a 1,...,a j } V be the set of possible state transitions of the time dependent treatment process and let A(t denote the given treatment at time t. We assume that N a1,..., N aj count the various transitions and let λ A = (λ a1,...,λ aj be predictable and non-negative. Let L = {l 1,..., l i } V such that A L = denote the possible changes in the process that reflect the patient s health condition. This could be successive measurements of various blood values etc. We assume that N l1,...,n li count their respective transitions and let λ L = (λ l1,..., λ li be F t -predictable and nonnegative.

6 6 KJETIL RØYSLAND Moreover, we assume that the patient can be right censored, i.e. we will not be able to observe the patient after time T C. This can happen because the study ends, but it can also be a drop out due to poor health or recovery. We let c V \(A L denote this event and let C(t = I(t T C and letλ c be F t -predictable and nonnegative processes. Finally, we let d c such that V = {d, c} A L denote the event death. We let T D denote the time when death occurs, let D(t := I(t T D and let λ d be F t -predictable and non-negative processes. If the condition (3 is satisfied, then there exists a probability measure P such that P Q and the processes λ 1,..., λ n form the intensities for their respective counting processes with respect to P and F t. We think of P as the observational probability measure, i.e. P is interpreted as the frequency of observations of our system. We assume that all the processes are stopped at time T D. This obviously means that the process D influences the other processes. However, we consider this dependency as trivial. As we will see in Lemma 2, we should consider local independence before T D, because then we will automatically have that D TD A C, L, D TD C A, L and D TD L A, C. We also assume that C TD A D, L, C TD D A, L and C TD L A, D. This means that censoring does not carry any information about the other process that we could not obtain if we left C out of the analysis. We can represent these relations using the following local independence graph: C L D A 3.2. Randomized trial measures. An important property of a randomized trial is that interventions in form of treatments are likely to be done in a different pattern than in an observational study. This means that frequencies of various observations in the randomized trial are determined by a different probability measure P on Ω. Since we consider time dependent treatments, we have to generalize the notion of a randomized trial slightly. In our experiments, the patient s previous health condition should be irrelevant for the how the treatment process vary in time. Therefore we will consider models where the treatment process A is locally independent of the remaining processes. Such a randomization strategy gives a probability measure P that carries the frequencies of potential observations of our randomized trial. Our goal is to provide a model of such a trial where a marginal estimate of the treatment effect can be interpreted as causal under the assumption of no unmeasured confounders. We will only consider models such that P P. This gives us a likelihood ratio d P d P dp such that dp P = P and makes us able to use Girsanov s theorem to do various computations. We will now define the randomized trial models and then provide some intuition around the key conditions. Definition 3. We will say that a probability measure P P is a randomized trial measure if

7 MARTINGALES AND MARGINAL STRUCTURAL MODELS 7 (1 The process M D (t = N D (t is a an F t -adapted local P-martingale, (2 The process M L (t = N L (t λ D (sds λ L (sds is a vector valued F t -adapted local P-martingale, (3 There exists a vector valued, F A,D t -predictable and non negative process µ A such that M A A (t := N A (t µ A (sds is a vector valued F t -adapted local P-martingale, (4 There exists a vector valued, F A,C,D t -predictable and non negative process κ such that M C C,A (t := C(t κ(sds is an F t -adapted local P-martingale. (5 P and P coincides on F, i.e. E P [h = E P[h for every bounded and F - measurable random variable h. The conditions (1 and (2 mean that the F t -predictable intensities for the state transitions of the processes L and D coincide for the the probability measures P and P. In other words, the best possible predictions of L t and D t, given the total history up to the moment just before t with respect to the observational measure P, coincide with the the best possible predictions with respect to P, given the same history. This is important since we should maintain the original dynamics of the underlying biological processes when we carry out the randomized trial. The condition (3 means that, with respect to P, the treatment process is locally independent of every other process, when conditioned on the remaining ones. This corresponds to the randomization of the treatment strategy, i.e. the given treatment is chosen independently of the patient s previous health condition. In general, there exist many treatment strategies that yield local independent treatment processes. The intuitive meaning of (4 is that censoring is locally independent of the confounder process L when conditioned on the remaining processes. Such a dependence would most likely lead to selection bias, since conditioning on C typically produces a statistical association between the treatment process A and the confounder process L. In Theorem 1 we will see that there sometimes exists a unique equivalent randomized trial measure, given µ and κ. Moreover, we will se that if so, then we also obtain independent censoring in the marginal model where the process L is left out. Theorem 1. If there exists a randomized trial measure P on F, then it is unique, given µ A and κ. We also obtain the following marginal local independence graph

8 8 KJETIL RØYSLAND with respect to P: C D A Conversely, suppose that the following conditions are satisfied: (1 For every i A, there exists an F A,D t -predictable process µ i such that λ i (s = if and only if µ i (s = for every s and i A, P a.s. (2 There exists an F A,C,D t -predictable process κ such that κ(s = if and only if λ C (s = for every s, P a.s. (3 We have: [ ( ( κ E P exp 1 2 λc + (µ i 1 2 λi ds <. λ C λ i i A Now, the equation: ( ( C(s κ(s (5 R(t := exp λ C (s κ(sds λ C (s (6 i A s t ( ( Ni(s µi (s exp λ i (s µ i (sds λ i (s defines a positive and uniformly bounded F t -adapted P-martingale and (7 P = limt R(tP defines a randomized trial measure P on F, such that (3 and (4 are satisfied. The condition (1 essentially means that if a particular change of treatment during an infinitesimal interval do not occur almost surely with respect to the observational probability measure P, given the previous full history, then this is also the case with respect to the randomized trial measure P and vice versa. Analogously, the condition (2 means that if censoring during an infinitesimal interval do not occur almost surely with respect to the observational probability measure P, given the previous full history, then this is also the case with respect to the randomized trial measure P and vice versa. The condition (3 is a regularity condition related to Novikov s condition that ensures the existence of P. In the simplified setting with A = {a}, there are two natural candidates for µ and κ respectively. The first is to choose µ and κ constantly equal 1 on the support of λ A and λ C. However, since 1/λ A could approach if λ A was small, we would typically obtain an unbounded likelihood ratio R. In order to deal with this problem, we could consider the stabilized version of this ratio, letting µ A (s = λ A A,D (s and κ = λ C A,C,D. However, note that the conditions (1, (2 and (3 are not automatically satisfied with these choices and have to be checked for each scenario Independent censoring. Following Andersen et al. [ABGK93, the right censoring at time T C is said to be independent for the event T D if the F A,D,C t - intensity for D coincides with the corresponding F A,D t -intensity. This is a convenient assumption when we do Cox regression analysis on right censored data. s t

9 MARTINGALES AND MARGINAL STRUCTURAL MODELS 9 However, in the marginal model where L is left out, this is just another way of saying that D is locally independent of C when conditioned on D. The previous theorem says that this is true with respect to the probability measure P. 4. Estimation of the likelihood ratio process For simplicity, we assume that an individual stays on the same treatment after initiation at time T A until death or censoring. We let Y A denote the at risk function for starting treatment, i.e. Y A (t = I(t T A. Moreover, we assume that the conditions provided by Theorem 1 are satisfied with µ A = λ A A,D and κ = λ C A,C,D. We consider a partition of the interval [, t, say = s < s 1 < < s m 1 < s m = t. Suppose that the probability of starting treatment in every interval (s k, s k+1 is relatively small, i.e. the intensity λ A is uniformly bounded. We will see that this leads to a natural method to estimate the likelihood ratio process that coincides with the discrete time weights in the marginal structural models proposed by Robins, [RHB. The argument in Proposition 3 from the Appendix shows that the following identity holds P a.s.: lim sup l s l s l 1 ( P ω (A(s k = A(s k ; ω Fs A,D k 1 P ω (A(s k = A(s k ; ω F sk 1 s k t ( ( λa A,D (s; ω A(s;ω =exp λ A (s; ω λ A A,D (s; ωds. λ A (s; ω s t Suppose that the probability for right censoring in any interval (s k, s k+1 is relatively small, i.e. the intensity λ C is uniformly bounded. A very similar argument as in Proposition 3 shows that: lim sup l s l s l 1 ( P ω (C(s k = F A,C,D s k 1 P ω (C(s k = F sk 1 s k t ( =exp λ C (s; ω λ C A,C,D (s; ωds for P almost every ω such that t < T C (ω. Putting these identities together, we obtain the likelihood ratio: R(t; ω = ( lim sup l s l s l 1 s k t s k t P ω (A(s k = A(s k ; ω F A,D s k 1 P ω (A(s k = A(s k ; ω F sk 1. ( P ω (C(s k = Fs A,C,D k 1 P ω (C(s k = F sk 1 Now suppose that the observed history of an individual, say a k, c k, d k and l k s k t is described by an ω Ω, i.e. a k = A(s k ; ω, c k = C(s k ; ω, d k = D(s k ; ω

10 1 KJETIL RØYSLAND and l k (s = L(s k ; ω for s k t. Note that we have: and P ω (A(s k = A(s k ; ω F A,D s k 1 P(A(s k = a k A(s j = a j, C(s j =, D(s j =, j < k P ω (A(s k = A(s k ; ω F sk 1 P(A(s k = a k A(s j = a j, C(s j =, D(s j =, L(s j = l j, L k = l k, j < k P ω (C(s k = F A,D s k 1 P(C(s k = A(s j = a j, C(s j =, D(s j =, j < k P ω (C(s k = F sk 1 P(C(s k = C(s j =, A(s j = a j, D(s j =, L(s j = l j, L k = l k, j < k. This means that we obtain an approximation: (8 R(t; ω AW t CW t, for P almost every ω such that t < T C (ω. This shows that the weights from the discrete time marginal models form an approximation of the likelihood ratio process R before T C. The conditional probabilities in the product formulas for (1 and (2 that are used in (8 can be estimated using logistic regression. This is done in Hernan et al. [HBR, Sterne et al. [SHL + 5 and Choi et al. [CHS Inference for equivalent randomized trial measures 5.1. Weighted Cox regression. We would like to make inference about the randomized trial measure based on observations of n independent individuals until death or censoring. The aim is to provide a reasonable marginal estimate of the treatment effect where we do not pay attention to the confounder process L. Let X(t denote the baseline covariates and the treatment given at time t. Moreover, let Y (t denote the at risk function, i.e. Y (t = I(t inf{t D, T C }. Suppose that we had P-distributed observations of covariates, events and censoring from n independent individuals. If H is an integrable and F A,C,D t -measurable random variable and h 1,..., h n are P- distributed samples of H from n independent individuals, then we define: P n H = 1 h k. n We could do a Cox regression for the death hazard ratio with respect to treatment and no treatment. The corresponding partial score function based on these samples would be given by: τ Ũ n (β, τ = n P n (X(t P n Y (tx(t e βx(t dd(t. P n Y (te βx(t We suppose that {h k } and {r k } are P-distributed samples of H and R for n independent individuals and define: P n HR(t = 1 h k r k. n k=1 k=1

11 MARTINGALES AND MARGINAL STRUCTURAL MODELS 11 Note that if HR(t is integrable, then by the law of large numbers: (9 lim n P n HR(t = E P [HR(t = E P [HR = Ẽ P[H = lim n Pn H. This identity suggests that the weighted partial score function (1 based on P- distributed samples of n independent individuals yields the following estimating equation for the hazard ratio with respect to the randomized trial measure P: τ Un R (β, τ = n P n R(τ (X(t P nr(t Y (tx(t e βx(t (1 dd(t. P n R(t Y (te βx(t Note that this gives the estimator for the causal treatment effect that is used in [CHS + 2, [Few4, [SHL + 5 and [HBR Applications to analyses of HIV-patients. We will now look at two applications of marginal structural models to epidemiological studies that can be interpreted within our framework. The first analysis was done by Sterne et al. in [SHL + 5. The intention was to examine the effectiveness of highly active antiretroviral therapy (HAART on HIV-infected patients using observational data from the Swiss HIV Cohort Study. The data was collected from 2161 HIV-infected patients with follow-up time between January 1996 and September 23. Of the 2161 patients, 717 received the HAART treatment and 22 progressed to AIDS or death at some time during their follow-up. Prognostic blood values, such as few CD4 cells or many copies of HIV-1 RNA were found to be strong predictors of both initiation of HAART and progression to AIDS or death. This means that the analysis should be done using a method as for instance marginal structural modeling that can deal with such confounding. In order to fit the weights on the right hand side of (8, they applied logistic regression. To estimate the hazard rate ratio between the treated and non treated groups in the marginal structural model, they used the weighted Cox proportional hazards model that gives the weighted score function (1. This resulted in the estimated hazard rate ratio adjusted for baseline covariates:.14, 95% CI (.7,.29. For comparison, they also fitted an unweighted and baseline adjusted Cox model and an unweighted Cox regression with time dependent covariates. We have included these estimates in Table 1. As we immediately see, HAART seems much more effective when estimated using marginal structural models than estimated using straightforward Cox models. Table 1. Estimated effects of HAART from various models in [SHL + 5. HR 95% CI Marginal structural model with baseline covariates.14 (.7,.29 Unweighted model with baseline covariates.36 (.23,.56 Unweighted model with time dependent covariates.64 (.42,.98 Another application of marginal structural models for analyzing HIV-patients where the contrast to straight forward Cox regression is even greater, was done by Hernan et al. in [HBR. They used these methods in order to estimate the effect of an early treatment, Zidovudine, on the survival of HIV-Patients. The data came from the Multicenter AIDS Cohort Study (MACS and consisted of observations of 5622 HIV-positive men with follow-up periods between March 1986

12 12 KJETIL RØYSLAND and October As in [SHL + 5, they fitted the weights on the right hand side of (8 using logistic regression. In order to approximately fit a weighted Cox proportional hazards model, they used pooled logistic regression. This resulted in the following estimated hazard rate ratio adjusted for baseline covariates:.7, 95% CI(.6, 1.. For comparison, they also fitted an unweighted and baseline adjusted Cox model that gave the following estimated hazard rate ratio adjusted for baseline covariates: 2.4, 95% CI( 1.9, 2.8. We see that Zidovudine did not seem to have a protecting effect when analyzed by a straight forward baseline adjusted Cox model, while it seemed to have a protecting effect using a baseline adjusted marginal structural model. Appendix A. Proofs A.1. Local independence. Note that we have stated the definition of local independence slightly different than the definition given by Didelez in [Did8. However, as will be shown in Lemma 1, these definitions are equivalent. In the present definition, the relation to martingale theory is more apparent. Lemma 1. If X, Y, Z V are disjoint and V = X Y Z then we have that N X T N Y N Z if and only if there exists a non negative F Y,Z t -predictable µ i such that λ i X,Y,Z (ti(t T = µ(ti(t T, Pa.s. for every i Y. If T is an F Y,Z t -stopping time and N X T N Y N Z, then for every i Y. λ i X,Y,Z (ti(t T = λ i Y,Z I(t T, Pa.s. Proof. If there exists such an F Y,Z -predictable µ, then N i (t t µ(si(s Tds = N i (t λ i X,Y,Z (si(s Tds is a local martingale with respect to the filtration F X,Y,Z t. Conversely, if N X T N Y N Z then the uniqueness part of the Doob-Meyer decomposition theorem gives us that for every t. λ i X,Y,Z (si(s Tds = µ(si(s Tds Sometimes we may not be interested in dependencies that are considered trivial. This could for instance be dependencies due to an absorbing state of a process X as for instance death. We will now see how one can handle such situations. Lemma 2. If X, Y, Z V are disjoint, V = X Y Z and T is an F t -adapted stopping time such that such that then we also have that: T inf{s > N i (s > for every i X}, N X T N Y N Z.

13 MARTINGALES AND MARGINAL STRUCTURAL MODELS 13 Proof. First we define Tn X := inf{t j X N j(t n}. By [Bré81, Theorem A.2 T34 a process Y is Ft X -predictable if and only it admits the representation: (11 Y (t = h (n (ti(tn X < t T n+1 X n 1 for a sequence of B + F X T n -measurable functions {h (n } n. If we choose a stopping time T as above then the representation (11 immediately gives us that N X T N Y N Z. A.2. Proof of the main theorem. We will now prove the main theorem of this paper. The problem of existence and uniqueness of an equivalent randomized trial measure is an example of what is known as a martingale problem. The problem is essentially to find an equivalent probability measure such that a given family of stochastic processes form martingales with respect to a given filtration. Such a measure is often called a martingale measure. The reason why we chose to emphasize this aspect is that martingale problems for marked point processes have been studied in great detail by several authors. This gives us a rich toolbox of results that can be applied in our setting. A key observation for us is that the likelihood ratio process in Theorem 1 is an example of a Doléans-Dade exponential. This means that the likelihood ratio process is a solution of stochastic a differential equation of the form: dψ t = Ψ t dφ t, see for instance [JS3 and [Jac75. Product integrals as in [GJ9 are solutions of such equations. Proposition 1. Suppose that P is a probability measure on (Ω, F and u 1,..., u n are F t -predictable and non negative processes such that K i := N i (t u i (sds is a local P -martingale with respect to F t for every 1 i n. Moreover, suppose that v 1,...,v n are F t -predictable and non negative processes such that u i (s; ω = if and only if v i (s; ω =, P a.s. for every 1 i n. If (12 R 1 (t := i ( exp u i (s v i (sds is a uniformly integrable martingale, then (13 P 1 = lim t R 1 (tp s t ( Ni(s vi (s u i (s defines a unique P -equivalent probability measure on (Ω, F such that N j (t v j (sds is a local P 1 -martingale with respect to F t for every 1 j n. If [ ( ( v i (s (14 E P exp u i (s 1 2 ui (sds <, i V then R 1 is a uniformly integrable P -martingale with respect to F t.

14 14 KJETIL RØYSLAND Proof. First note that P 1 defines an equivalent probability measure by Doob s convergence theorem. Theorem [Pro5, 37 states that if we let Φ(t := ( vj (s j u j (s 1 dk j (s, then R 1 (t ( = exp Φ(t 1 2 [Φ, Φ(t = i <s t ( exp u i (s v i (sds ( (1 + Φ(sexp Φ(s + 12 ( Φ(s2 s t ( Ni(s vi (s u i (s is the unique F t -adapted solution of the stochastic differential equation: (15 R 1 (t = 1 + ( vj (s R 1 (s j u j (s 1 dk j (s. Now, K i, R 1 t = j = = ( vj (s R 1 (s u j (s 1 K i, K j s ( vi (s R 1 (s u i (s 1 K i, K i s ( R 1 (s v i (s u i (s ds and by Girsanov s theorem, [Pro5, II 4, the process K i (t 1 R 1 (s d K i, R 1 s = K i (t = N i (t ( v i (s u i (s ds v i (sds is a local P 1 -martingale with respect to F t for every 1 i n. The uniqueness part follows directly from the uniqueness of martingale measures for marked point processes, see [JS3, Theorem III A version of Novikov s condition for jump processes, [PS6, Theorem 6 gives that if Φ(s > 1, Φ is a square integrable martingale and E P [exp Φ, Φ <, then R 1 is a uniformly integrable P -martingale with respect to F t. That Φ(s > 1, follows immediately from our assumptions. Moreover, to see that Φ is square integrable, we let {s n } n be a localizing sequence for [Φ, Φ and note that by the monotone convergence theorem we have that E P [ [Φ, Φ =limn E P [ [Φ, Φsn = limn E P [ Φ, Φ sn = EP [ Φ, Φ E P [ exp Φ, Φ.

15 MARTINGALES AND MARGINAL STRUCTURAL MODELS 15 By [Pro5, Corollary 4, p.74, Φ is square integrable if this expression is bounded. Moreover, we have that Φ, Φ = ( 2 vi (s i u i (s 1 d K i, K i s = ( 2 vi (s u i (s 1 u i (sds, i.e. the last claim follows from (14. The next result and it s proof are essentially the same as in [Did8, A1. Proposition 3. We consider a slightly different setting, and our proof is a bit more detailed. The original proof in [Did used stochastic filtering techniques from [AHN92. Proposition 2. Suppose X, W, Z, Y V such that Y W and (W X\(Z Y =. Moreover, suppose that T is a finite F Z,Y t -adapted stopping time and that τ i is {i} Y Z F -adapted for every i V. If we have that: t (1 X T W Z, (2 W T X \ (Z Y Z Y, (3 X T {k} Z W or W T {k} Z Y X for every k Z \ Y, then X T Y Z. Proof. We suppose that V = X W Z, X = X \ (Z Y, W = W \ Y and Z = Z \Y. Now X W = and some simple manipulations with these relations tell us that: (1 X T W Z Y, (2 W T X Z Y, (3 X T {k} W Z Y or W T {k} X Z Y for every k Z Y, (4 X T Y W Z. Let Pt T = P F and Q T t = Q F. Recall that N 1,..., N n have the intensities I( τ 1,..., I( τ n with respect to the probability measure Q and the filtration F t. By Doob s optional sampling Theorem, the likelihood ratio between Pt T and Q T t is given by: dpt T dq T = R (t T = ( T τi exp 1 λ i (sds λ i (s Ni(s. t i V s The local independence assumptions imply the following: If k X, then λ k (ti(t T is F X,Z,Y ( -measurable, If k W, then λ k (ti(t T is F W,Z,Y ( -measurable, If k Z Y, then λ k (ti(t T is either F X,Z,Y ( F W,Z,Y ( -measurable. i -measurable or This means that there exist both an F X,Z,Y -adapted process L X and an F W,Z,Y - adapted process L W such that dp T t = L dq T X (tl W (t for every t. t Using Bayes formula [Bré81, Lemma L5, we see that if H is F X,Z,Y -measurable then: E P [H F W,Z,Y T t dq T t = E Q[H dp F W,Z,Y E Q [ dp t T F W,Z,Y dq T t, P.a.e.

16 16 KJETIL RØYSLAND Using the above factorization of dp T t, we get dq T t E P [H F W,Z,Y = E Q[HL X (tl W (t F W,Z,Y E Q [L X (tl W (t F W,Z,Y,Z,Y = E Q[HL X (t F W E Q [L X (t F W,Z,Y, P.a.e. Since N k and N j are conditionally independent whenever k j and X W =, we have that Ft X F W t F Z,Y t with respect to Q. Now E P [H F W,Z = E Q[HL X (t F Z,Y E Q [L X (t F Z,Y, P.a.e. and E P [H F W,Z,Y is F Z,Y -measurable. This implies that and F X,Z,Y that E P [H F W,Z,Y = E P [H F Z,Y F W,Z,Y F Z,Y F X,Z,Y ( with respect to P. Note that this also implies F W,Z,Y ( FZ,Y ( with respect to P. The innovation theorem, [Bré81, II T14, states that whenever F V, we have λ k F (t = E[λ k F F t P.a.e. Moreover, by the uniqueness of the Doob-Meyer decomposition and the relation X T Y W Z, we see that λ k (ti(t T = λ k W,Z,Y (ti(t T. These observations justify the following computation: λ k Z,Y (ti(t T = λ k Z,Y (ti(t T = E[λ k (ti(t T F Z,Y t =E[E[λ k (ti(t T F W,Z,Y t F X,Z,Y t = E[λ k (ti(t T F X,Z,Y t =λ k X,Z,Y (ti(t T = λ k X,Z,Y (ti(t T. But this says that X T {k} Z Y for every k Y, i.e. we end up with the relation: X T Y Z. Proof of Theorem 1. The existence and uniqueness part of the claim follows directly from Proposition 1. Our situation can now be described with the following local independence graph: C L D. A Note that we have the following relations with respect to the measure P: (1 C TD L, D A, (2 L TD C A, D, (3 C TD A L, D. By Proposition 2, we obtain the relation C TD D A. Moreover, we also see that: (1 C TD L, A D, (2 L TD C A, D, (3 C TD D L, A.

17 MARTINGALES AND MARGINAL STRUCTURAL MODELS 17 Using Proposition 2 again, we obtain the relation C TD following local independence graph: C D A D. This gives the A A.3. Discrete time approximations of the likelihood ratio process. We will now look at some discrete time approximations of the likelihood ratio process in the case where N A is a single counting process and the randomization strategy µ equals λ A A,D. The first reason for doing this is that it provides understanding about the likelihood ratio process itself. The second reason is that it also shows that the weights in the marginal structural models by Robins [RHB form natural approximations of the likelihood ratio process (5. Proposition 3. Let τ (,, let {S n } n be an increasing family of finite subsets in [, τ. Let S n(ω := {s S n N A (s; ω = N A (s ; ω} and let S 1 n(ω := S n \S n(ω. Moreover, if s S n, let s := max{t S n t < s}. We define (16 (17 W (n (τ; ω s = E ω [N s N s FA,D E ω [N s N s Sn 1(ω s F s and suppose that the following conditions are satisfied: (1 lim n sup s Sn s s = (2 There exists a J > for every r Y s.t. Now, P a.s. lim W (n (τ; ω n =exp( τ Proof. First we define: 1 E ω [N t N t F 1 E ω [N t N t Sn (ω t F t λ A (s; ω J, P a.s. λ A (s λ A A,D (sds s τ λ (n A (t; ω = λ (n A A,D (t; ω = ( λa A,D (s; ω λ A (s; ω s S n (ω I(s < t se ω [λ A (t F s A,D t NA(s;ω s S n (ω I(s < t se ω [λ A A,D (t F A,D s. Note that λ (n A (t; ω, λ(n A A,D (t; ω J. By the martingale convergence theorem and the assumption lim n sup s Sn s s =, we see that lim n λ (n A (t; ω = λ A(t; ω and lim n λ (n A A,D (t; ω = λ A(t; ω P a.s.

18 18 KJETIL RØYSLAND Now E ω [N A (s N A (s FA,D s 1 E ω [N A (t N A (t F (18 E ω [N A (s N A (s F s Sn 1(ω s 1 E ω [N A (t N A (t F t Sn (ω t (19 = s S 1 n (ω u Sn (ω k 1 E ω [ s s λ A A,D (tdt F A,D s E ω [ s s λ A (tdt F s u S n (ω A,D t 1 E ω [ u λ A A,D (tdt F A,D 1 E ω [ u λ A (tdt F u If E ω [ u λ A A,D (tdt Fu A,D < 1 and E ω [ u λ A (tdt F u < 1, then the logarithm of the second product in (19 equals: u u log(1 E ω [ λ A A,D (tdt Fu A,D log(1 E ω [ λ A (tdt F u u Sn (ω = 1 u k E ω[ λ A (tdt F u k 1 u k E ω[ λ A A,D (tdt Fu A,D k k 1 The previous expression now equals: τ λ (n A (t λ(n A A,D (tdt + u Sn (ω k 2 u Sn (ω k 2 1 u k E ω[ λ A A,D (tdt Fu A,D k. By Fubini s theorem and Jensen s inequality, we see that: 1 u (2 k E ω[ λ A (tdt F u k (21 (22 (23 u Sn (ω k 2 = u Sn (ω k 2 u Sn (ω k 2 = k 2 ( 1 u k 1 k (u k 1 E ω [λ A (tdt F u dr 1 k (u k 1 τ u 1 u k E ω[ λ A (tdt F u k k E ω [λ A (t F u k dt λ (nk A (tdt. By choosing n large enough, we can assume that there exists a θ < 1 such that the summands in (2 are bounded by (θ k τ 1 (k sup s S n (ω s s. By the dominated convergence theorem, we see that: lim n lim n u Sn (ω k 2 k 2 1 k E ω[ 1 k (u k 1 An identical argument using λ (n A A,D 1 lim n k E ω[ u Sn (ω k 2 u λ A (tdt F u k τ instead of λ(n u λ (nk A (tdt =. A gives that: λ A A,D (tdt F A,D k =.

19 MARTINGALES AND MARGINAL STRUCTURAL MODELS 19 Finally, by the dominated convergence theorem: lim n u S n (ω ( lim n 1 E ω [ u λ A A,D (tdt Fu A,D 1 E ω [ u λ A (tdt F u u 1 = exp k E ω[ λ A (tdt F u k u Sn (ω k 1 1 u k E ω[ λ A A,D (tdt Fu A,D k u ( τ = exp λ A (t λ A A,D (tdt. Suppose N A ( ; ω jumps l times before τ. Let T m (ω denote the m th jump of N A (ω before τ. When n is large enough, Sn 1 (ω consists of exactly l elements, say s n 1 < < s n l. Moreover, sn m < T m (ω s n m for every m l and we see that: lim n = s S 1 n T m (ω τ E ω [ s s λ A A,D (tdt F A,D s E ω [ s s λ A (tdt F s lim n E ω [ s n m s λ n A A,D (tdt F A,D s n m m lim n E ω [ s = ( λa A,D (s; ω NA(s;ω. n m s λ n A (tdt F s n m m λ A (s; ω s τ Acknowledgement I would like to thank Odd O. Aalen, Vanessa Didelez and Jon Michael Gran for helpful discussions related to this project. References [Aal78 Odd Aalen. Nonparametric inference for a family of counting processes. Ann. Statist., 6(4:71 726, [Aal87 Odd O. Aalen. Dynamic modelling and causality. Scand. Actuar. J., (3-4:177 19, [ABGK93 Per Kragh Andersen, Ørnulf Borgan, Richard D. Gill, and Niels Keiding. Statistical models based on counting processes. Springer Series in Statistics. Springer-Verlag, New York, [AHN92 Elja Arjas, Pentti Haara, and Ikka Norros. Filtering the histories of a partially observed marked point process. Stochastic Process. Appl., 4(2:225 25, [Bré81 Pierre Brémaud. Point processes and queues. Springer-Verlag, New York, Martingale dynamics, Springer Series in Statistics. [CHS + 2 H.K. Choi, M.A. Hernán, J.D. Seeger, J.M. Robins, and F. Wolfe. Methotrexate and mortality in patients with rheumatoid arthritis: a prospective study. The Lancet, 359(9313: , 22. [Did Vanessa Didelez. Graphical models for event history analysis based on local independence. Logos Verlag Berlin, Berlin, 2. Dissertation, Universität Dortmund, Dortmund, 2.

20 2 KJETIL RØYSLAND [Did7 Vanessa Didelez. Graphical models for composable finite Markov processes. Scand. J. Statist., 34(1: , 27. [Did8 Vanessa Didelez. Graphical models for marked point processes based on local independence. J.R Statist. Soc. B, 7(1: , 28. [Few4 Z. Fewell. Controlling for time-dependent confounding using marginal structural models. Stata Journal, 4(4:42 42(19, 24. [GJ9 Richard D. Gill and Søren Johansen. A survey of product-integration with a view toward application in survival analysis. Ann. Statist., 18(4: , 199. [HBR M.A. Hernán, B. Brumback, and J.M. Robins. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology, 11:561 57, Sep 2. [HHDR4 M.A. Hernán, S. Hernández-Díaz, and J.M. Robins. A Structural Approach to Selection Bias. Epidemiology, 15(5:615, 24. [Jac75 Jean Jacod. Multivariate point processes: predictable projection, Radon-Nikodým derivatives, representation of martingales. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 31: , 1974/75. [JS3 Jean Jacod and Albert N. Shiryaev. Limit theorems for stochastic processes, volume 288 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences. Springer-Verlag, Berlin, second edition, 23. [KS98 Ioannis Karatzas and Steven E. Shreve. Methods of mathematical finance, volume 39 of Applications of Mathematics (New York. Springer- Verlag, New York, [Lok8 Judith J. Lok. Statistical modeling of causal effects in continuous time. Ann. Statist., 36(3: , 28. [Øks3 Bernt Øksendal. Stochastic differential equations. Universitext. Springer-Verlag, Berlin, sixth edition, 23. An introduction with applications. [Pro5 Philip E. Protter. Stochastic integration and differential equations, volume 21 of Stochastic Modelling and Applied Probability. Springer- Verlag, Berlin, 25. Second edition. Version 2.1, Corrected third printing. [PS6 Phillip E. Protter and Kazuhuro Shimbo. No arbitrage and general semimartingales. Preprint: protter/webpapers/na-girsanov8.pdf, 26. [RHB J.M. Robins, M.A. Hernán, and B. Brumback. Marginal structural models and causal inference in epidemiology. Epidemiology, 11:55 56, Sep 2. [Rob92 J. Robins. Estimation of the time-dependent accelerated failure time model in the presence of confounding factors. Biometrika, 79(2: , [Rob98 JM Robins. Structural nested failure time models. The Encyclopedia of Biostatistics, pages , [Sch7 Tore Schweder. Composable Markov processes. J. Appl. Probability, 7:4 41, 197.

21 MARTINGALES AND MARGINAL STRUCTURAL MODELS 21 [SHL + 5 J.A. Sterne, M.A. Hernán, B. Ledergerber, K. Tilling, R. Weber, P. Sendi, M. Rickenbach, J.M. Robins, and M. Egger. Long-term effectiveness of potent antiretroviral therapy in preventing AIDS and death: a prospective cohort study. Lancet, 366: , 25. University of Oslo, Institute of Basic Medical Sciences, Department of Biostatistics, Boks 1122 Blindern, 317 Oslo, Norway address: kjetil.roysland@medisin.uio.no