Discussion of Identifiability and Estimation of Causal Effects in Randomized. Trials with Noncompliance and Completely Non-ignorable Missing Data

Transcription

1 Biometrics 000, DOI: Discussion of Identifiability and Estimation of Causal Effects in Randomized Trials with Noncompliance and Completely Non-ignorable Missing Data Dylan S. Small Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104, U.S.A. and Jing Cheng Division of Biostatistics, University of Florida College of Medicine, Gainesville, FL 32610, U.S.A. 1

2 Discussion of CGZ Paper on Randomized Trials with Noncompliance and CN Missing Data 1 1. Introduction We congratulate Chen, Geng and Zhou (CGZ) on an important contribution to the analysis of randomized trials with noncompliance and missing data, two common complications in randomized trials. Most previous methods for analyzing trials in which noncompliance and missing data occur together have either assumed (i) the outcomes are missing at random or (ii) latent ignorability. Missing at random or latent ignorability assume that missingness is not related to the outcome within certain strata of random assignment, compliance and treatment received. For situations in which missingness is related to outcomes within these strata, CGZ develop novel methods of identifying causal effects. To emphasize the importance of CGZ s methods and illustrate several points, we consider a trial in which missingness might be related to outcome. We use the same notation as CGZ throughout. Ferro et al. (1996) conducted a trial of methods for teaching breast selfexamination (BSE) for breast cancer screening in which women were randomly assigned to either a standard treatment of receiving mailed information about BSE (Z = 0) or an enhanced treatment of attendance in a self-exam course in addition to receiving the mailed information (Z = 1). A woman was considered to have received the treatment (D = 1) if she attended the self-exam course. Of the women assigned to attend the self-exam course, only 55% of the women actually attended the course. The primary outcome for the study was whether or not a woman practiced BSE one year after the beginning of the study, where the outcome was obtained from a mailed self-administered questionnaire. Only 65% of the women returned the questionnaire (R = 1). The data, taken from Mealli et al. (2004) s Table 1, are shown in our Table 1. There are structural zeroes for Z = 0, D = 1 because women assigned to the standard treatment could not attend the self-exam course. Missingness might depend on the outcome because women who are practicing BSE might feel more commitment to returning a questionnaire for a study on BSE. Missingness might also

3 2 Biometrics, depend on compliance class and random assignment. For example, if lack of cooperativeness in returning the questionnaire is associated with lack of cooperativeness in following assigned protocol, we would expect never takers to have more missingness than compliers. [Table 1 about here.] CGZ s model of Section 3.1 handles the situation in which missingness depends on the outcome, but, conditional on the outcome, missingness does not depend on compliance class and random assignment. For situations in which missingness might depend on compliance class and/or random assignment conditional on the outcome, CGZ introduce additional identification strategies in Section 3.2. These strategies require covariates that predict outcomes but not missingness in a certain way. Such covariates are not always available. In the remainder of this contribution, we discuss additional analytical methods for situations in which missingness might depend not only on outcomes but also on compliance class and/or random assignment. We focus on the following topics: (1) we formulate a class of models for missingness that contains CGZ s model of Section 3.1; (2) we discuss how the goodness of fit of CGZ s models can be tested; (3) we discuss methods of sensitivity analysis for when a researcher is not completely confident in a model s assumptions. 2. Models for Missingness In this section, we discuss data without covariates and formulate a class of models for the observable data (Z i, D i, Y i, R i), where Y i = Y i if R i = 1 and Y i =? if R i = 0; CGZ s model in their Section 3.1 is a member of this class of models. We shall write the distribution of (Z i, D i, Y i, R i ) in terms of the partially unobservable (Z i, U i, Y i, R i ). To do this, we define the partially unobserved compliance class U i to be consistent with the observed (D i, Z i ) if P(D(Z i ) = D i U = U i ) > 0. Similarly, we define the partially unobserved Y i to be consistent with the observed (Y i, R i ) if P(Y = Y i R = R i, Y = Y i ) > 0. We can express

4 Discussion of CGZ Paper on Randomized Trials with Noncompliance and CN Missing Data 3 the distribution of (Z i, D i, Y i, R i) in terms of the distribution of (Z i, U i, Y i, R i ) as follows: P(Z i, D i, Y i, R i) = P(Z i, U i, Y i, R i ). (1) U i : U i consistent with (D i, Z i ) Y i : Y i consistent with (Yi, i) We now consider a class of models for the summands (Z i, U i, Y i, R i ) in (1). We decompose the probability distribution of (Z i, U i, Y i, R i ) as P(Z i, U i, Y i, R i ) = P(Z i, U i, Y i )P(R i Z i, U i, Y i ). For P(Z i, U i, Y i ), under CGZ s Assumptions 1-4, we have the following, where I( ) is the indicator function: P(Z i, U i, Y i ) = ξ Z i (1 ξ) 1 Z i ω I(U i=n) n θ Y ii(u i =a) 10a ω I(U i=a) a [1 ω a ω n ] I(U i=c) (1 θ 10a ) (1 Y i)i(u i =a) θ Y ii(u i =n) 11n (1 θ 11n ) (1 Y i)i(u i =n) θ Y ii(u i =c)(1 Z i ) 10c (1 θ 10c ) (1 Y i)i(u i =c)(1 Z i ) θ Y ii(u i =c)z i 11c (1 θ 11c ) (1 Y i)i(u i =c)z i. (2) For P(R i Z i, U i, Y i ), under the monotonicity assumption (Assumption 3 in CGZ), the saturated model can be written as logit[p(r i = 1 Z i, U i, Y i )] = α 0 + α 1 I(Y i = 1) + α 2 I(U i = a) + α 3 I(U i = n) + α 4 I(Z i = 1, U i = a) + α 5 I(Z i = 1, U i = n) + α 6 I(Z i = 1, U i = c). (3) Under model (2)-(3) for the partially unobservable data (Z i, U i, Y i, R i ), the model (1) for the observable data (Z i, D i, Y i, R i) contains 14 free parameters. However, the contingency table of the the observable data (Z i, D i, Y i, R i) (Table 3 in CGZ) contains 12 cells, meaning at most 11 free parameters can be identified. Thus, in order to identify all the parameters in model (2)-(3), we must impose at least three constraints. Table 2 presents nine models that impose constraints on the model for missingness (3) that enable identification. Note that as in CGZ s Theorem 1, there are regularity conditions needed beyond one of the models in

5 4 Biometrics, Table 2 holding to ensure identification. Also, note that if our goal is only to identify certain parameters, e.g., the intention to treat effect, and not the entire distribution of (Z, U, Y, R), less stringent conditions may be needed; see Imai (2007) for discussion of this. [Table 2 about here.] The first five models we consider do not allow the outcome to affect missingness within strata of (Z, U). 1. Missing completely at random (MCAR): α 1 = α 2 = α 3 = α 4 = α 5 = α 6 = 0. In this model, missingness does not depend on outcome, random assignment or compliance class. 2. Missing at random (MAR): α 1 = 0, α 2 + α 4 = α 6, α 3 = 0. In this model, missingness is independent of the value of Y given treatment assignment Z and treatment received D. 3. Latent ignorability and response exclusion restriction for never takers and always takers (LI-RER-NT&AT): α 1 = α 4 = α 5 = 0. This is the model formulated by Frangakis and Rubin (1999) and discussed by CGZ. Note that a subject s response behavior is said to satisfy the response exclusion restriction if the missingness of the subject s outcome would be the same irrespective of whether the subject was assigned to the treatment or control. 4. Latent ignorability and response exclusion restriction for compliers and always takers (LI-RER-C&AT): α 1 = α 4 = α 6 = 0. This alternative to model 3 (LI-RER-NT&AT) was formulated by Mealli et al. (2004) in a setting in which there were no always takers. 5. Latent ignorability and response exclusion restriction for all compliance classes (LI-RER- All): α 1 = α 4 = α 5 = α 6 = 0. This model combines the assumptions of models 3 (LI-RER-NT&AT) and 4 (LI-RER-C&AT). The remaining models we consider allow for outcome to affect missingness within strata of (Z, U).

6 Discussion of CGZ Paper on Randomized Trials with Noncompliance and CN Missing Data 5 6. Missingness affected only by outcome the completely non-ignorable without covariates (CN-WOCOV) model: α 2 = α 3 = α 4 = α 5 = α 6 = 0. This is the model developed by CGZ in their Section 3.1. CGZ s Assumption 6 implies P(R = 1 Z, Y, U) = P(R = 1 Y ) which imposes the constraints α 2 = α 3 = α 4 = α 5 = α 6 = 0 on the model (3) for missingness. 7. Missingness affected only by outcome and treatment received (M-YD): α 2 = α 6, α 3 = α 4 = α 5 = 0. This model implies the following for d = 0, 1, y = 0, 1: P(R i = 1 Y i = y, D i = d, Z i = 1) = P(R i = 1 Y i = y, D i = d, Z i = 0). (4) Imai (2007) formulates model (4) and shows how it enables identification of the complier average causal effect (CACE). Imai s model contains the CN-WOCOV model, but also allows for an effect of receiving the treatment on missingness. In the BSE study, Imai s model would be a useful extension of CN-WOCOV if attending the course on BSE creates a further commitment to returning the study questionnaire beyond that created just by practicing BSE. 8. Missingness affected only by outcome and random assignment (M-YZ): α 2 = α 3 = 0, α 4 = α 5 = α 6. This model also contains the CN-WOCOV model. It differs from Imai s model in that it allows for an effect of random assignment but not of treatment received once random assignment is controlled for. 9. Response exclusion restriction for all compliance classes (RER-All): α 4 = α 5 = α 6 = 0. This is another model that contains the CN-WOCOV model. It allows for the outcome and the compliance class to affect missingness, but not the random assignment.

7 6 Biometrics, Testing CGZ s Models 3.1 Testing the CN-WOCOV Model CGZ s CN-WOCOV model uses nine parameters to model the contingency table of observable data (Table 3 in CGZ) whereas a saturated model has 11 parameters. Consequently the goodness of fit of the CN-WOCOV model can be tested. For the flu shot data in CGZ s Table 3, a likelihood ratio test of the CN-WOCOV model versus its saturated counterpart was conducted (p < 0.01 on 2 d.f.) with a bootstrap-based test giving similar results. Thus, there is strong evidence that the CN-WOCOV model does not hold for the flu shot data. For the BSE study, a goodness of fit test does not reject the CN-WOCOV model: a likelihood ratio test of the CN-WOCOV model versus its saturated counterpart gives a p-value of 0.28 on 1 d.f. A goodness of fit test is useful for detecting certain settings in which the CN-WOCOV model does not hold. However, it is important to realize that the goodness of fit test is not consistent for some alternatives, i.e., there are members of the model (2)-(3) that are not part of the CN-WOCOV model, a submodel of (2)-(3), for which the power of the goodness of fit test does not converge to one asymptotically. The reason is as follows. Let θ denote the 14 parameters in model (2)-(3). We can divide the parameter space of θ into equivalence classes of the θ that give the same distribution for the observable data (Z i, D i, Y i, R i). Each of these equivalence classes is a three-dimensional manifold. An equivalence class contains at most one θ that is in the CN-WOCOV model. If the true θ is not in the CN-WOCOV model but is in an equivalence class that contains a point in the CN-WOCOV model, then the goodness of fit test is not consistent (because the true θ has the same distribution for the observed data as a θ in the CN-WOCOV model). We illustrate this in the BSE study. There are no always takers for the BSE study so ω a, θ 10a, α 2 and α 4 are not considered in (2)-(3) and the CN-WOCOV model is the submodel of (2)-(3) that constrains α 3 = α 5 = α 6 = 0. The MLE

8 Discussion of CGZ Paper on Randomized Trials with Noncompliance and CN Missing Data 7 of the CN-WOCOV model is ˆθ = (ˆξ = 0.501, ˆω n = 0.447, ˆθ 11n = 0.191, ˆθ 11c = 0.717, ˆθ 10c = 0.839, ˆα 0 = 0.913, ˆα 1 = 6.555, ˆα 3 = 0, ˆα 5 = 0, ˆα 6 = 0). There is a two-dimensional manifold of θ that give the same distribution for the observable data (Z i, D i, Y i, R i) as ˆθ. If we constrain α 6 = 0, there is a one-dimensional manifold of θ that give the same distribution for the observable data as ˆθ. If the true θ is in this manifold but not equal to ˆθ, the goodness of fit test is not consistent. The coordinates of (α 3, α 5 ) for the θ in this manifold that satisfy 2 < α 3 < 2 are plotted in Figure 1. [Figure 1 about here.] 3.2 Testing CGZ s Models That Use Covariates in Identification In CGZ s Section 3.2, they formulate two models for identifying the CACE that use covariates that are assumed to affect the outcome within certain strata of (Z, U) but are assumed not to affect missingness within these strata. The approach for testing the CN-WOCOV model described above can be adapted to CGZ s models that use covariates. In particular, if X is binary, then CGZ s model based on Assumption 7 has 18 parameters and CGZ s model based on Assumption 8 has 22 parameters, while the saturated model for the observable data has 23 parameters. Thus, the goodness of fit of both of CGZ s models can be tested. 4. Sensitivity Analysis For many studies in which an investigator thinks that one of CGZ s models is reasonable, the investigator will not be completely confident that the model holds. For such studies, a sensitivity analysis should be done. A sensitivity analysis examines the impact of substantively plausible violations on the inferences that can be made (Rosenbaum, 2002, Ch. 4). The output of a sensitivity analysis that we consider is a sensitivity interval, an analogue of a confidence interval (CI) for when there is uncertainty about a model s assumptions. We now illustrate how a sensitivity analysis can be done for the CN-WOCOV model,

9 8 Biometrics, using the BSE study as an example. For the BSE study, the CN-WOCOV model s estimate of the CACE is with a 95% CI of ( 0.265, 0.021). This CI does not fully reflect our uncertainty about the CACE if we are not completely confident that the CN-WOCOV model holds. One reason to be concerned about the CN-WOCOV model holding that was discussed in the introduction is that never takers might be less cooperative than compliers in responding to the questionnaire. In other words, α 3 might be less than zero in model (3). Another reason to be concerned about the CN-WOCOV model holding that was raised by Mealli et al. (2004) is that once never takers show that they are unwilling to follow their assigned protocol, as they do under assignment to the enhanced treatment (Z = 1) but not under assignment to the standard treatment (Z = 0), they might be less inclined to respond to the questionnaire. In other words, α 5 might be less than zero. For illustrative purposes, we will consider a range of concern for (α 3, α 5 ) of 2 α 3, α 5 0 in a clinical study we were involved in, we would consult with the clinical investigators to determine the range of concern for (α 3, α 5 ). Our procedure for forming a 100(1 γ)% sensitivity interval (SI) for the CACE is to first form a joint confidence region for (α 3, α 5, CACE) under the assumption that α 3 and α 5 belong to the range of concern (and α 6 = 0) and then to project this confidence region to form a CI for the CACE. To form the joint confidence region, our approach is to invert a test of H 0 : CACE = CACE, α 3 = α3, α 5 = α5 for (α 3, α 5 ) in the range of concern for (α 3, α 5 ), where we carry out the test of H 0 by testing H 0A : α 3 = α 3, α 5 = α 5 at level γ 1 < γ and H 0B : CACE = CACE α 3 = α 3, α 5 = α 5 at level γ γ 1, and then reject H 0 if H 0A or H 0B is rejected. By Bonferroni s inequality, the size of the test that rejects H 0 if H 0A or H 0B is rejected is at most γ. To put more emphasis on the test of H 0B, we chose γ 1 = for γ = 0.05 in our example. The SI is the set of all CACE s for which there exists an (α 3, α 5 ) such that H 0 : CACE = CACE, α 3 = α 3, α 5 = α 5 is not rejected. The SI is analogous to a

10 Discussion of CGZ Paper on Randomized Trials with Noncompliance and CN Missing Data 9 CI in that as long as the true (α 3, α 5 ) belongs to the range of concern, the SI will contain the true CACE at least 1 γ proportion of the time. See Small (2007) for further discussion of SIs. The SI is difficult to compute exactly but can easily be approximated by considering the (α 3, α 5 ) on a grid over the range of concern. Figure 2 shows for such a grid for the BSE study, the (α 3, α 5 ) that are accepted and rejected at level A summary of our procedure for finding the 95% SI is as follows: for each (α 3, α 5 ) that is accepted in Figure 2 (the solid circles), we form a 95.5% CI for the CACE assuming the given value of (α 3, α 5 ) holds (and α 6 = 0); the 95% SI is (minimum of lower end points of CIs, maximum of upper end points of CIs). Using this procedure, we obtain a 95% SI of ( 0.475, 0.245), which is more than twice as wide as the 95% CI of ( 0.265, 0.021) that is obtained if we are sure that the CN- WOCOV model holds. In summary, if we are concerned that (α 3, α 5 ) might be in the range of 2 α 3, α 5 0, then we are substantially more uncertain about the CACE than if we are confident that the CN-WOCOV model holds (which would mean α 3 = α 5 = 0). [Figure 2 about here.] 5. Conclusions CGZ make a valuable contribution to the analysis of randomized trials with noncompliance and missing outcomes by formulating identifiable models that allow for the outcome to affect missingness. In our discussion, we have shown how CGZ s models relate to a larger class of models. We have also discussed tests of CGZ s models and an approach to sensitivity analysis for CGZ s models. We recommend the following analysis strategy when one of CGZ s models (or one of the other models described in Section 2) is thought to be plausible. If the model has less parameters than the saturated model for the observed data, test the goodness of fit of the model as we did in Section 3. If the model is not rejected by the test (or cannot be

11 10 Biometrics, tested), then carry out a sensitivity analysis for the most plausible violations of the model s assumptions as we illustrated in Section 4. The resulting sensitivity interval for the CACE (or any other parameter of interest) expresses our uncertainty about the CACE, accounting for both sampling fluctuations and uncertainty about the assumptions of the model holding. References Ferro, S., Caroli, A., Nanni, O., Biggeri, A. and Gambi, A. (1996). A randomized trial on breast self-examination in Faenze (Northern Italy). Tumori 82, Frangakis, C.E. and Rubin, D.B. (1999). Addressing complications of intention-to-treat analysis in the combined presence of all-or-none treatment-noncompliance and subsequent missing outcomes. Biometrika 86, Imai, K. (2007). Statistical analysis of randomized experiments with nonignorable missing binary outcomes: application to a voting experiment. Technical report: Mealli, F., Imbens, G. W., Ferro, S. and Biggeri, A. (2004). Analyzing a randomized trial on breast self-examination with noncompliance and missing outcomes. Biostatistics, 5, Rosenbaum, P. R. (2002). Observational studies, 2nd edition. Springer, New York. Small, D. (2007). Sensitivity analysis for instrumental variables regression with overidentifying restrictions. Journal of the American Statistical Association, 102,

12 Discussion of CGZ Paper on Randomized Trials with Noncompliance and CN Missing Data 11 Observationally Equivalent Parameter Values α α 3 Figure 1. The (α 3, α 5 ) for the θ that lie on the manifold of parameters that give the same distribution for the observable data as ˆθ, the MLE from the BSE study.

13 12 Biometrics, Range of concern for α 3,α 5 α α 3,α 5 not rejected α 3,α 5 rejected α 3 Figure 2. Results of tests of (α 3, α 5 ) in the range of concern for the BSE study.

14 Discussion of CGZ Paper on Randomized Trials with Noncompliance and CN Missing Data 13 Table 1 Breast Self Examination Study (Mealli et al., 2004) Z = 0, D = 0 Z = 0, D = 1 Z = 1, D = 0 Z = 1, D = 1 R = 1, Y = R = 1, Y = R = 0, Y =?

15 14 Biometrics, Table 2 Models for Missingness. Sets of constraints on the model for missingness (3) that enable identification of the parameters in (2)-(3). The full names for the abbreviated model descriptions in the table are provided in Section 2. # Model Constraints Additional Description α 1 α 2 α 3 α 4 α 5 α 6 Constraints 1 MCAR MAR 0 0 α 2 + α 4 = α 6 3 LI-RER-NT&AT LI-RER-C&AT LI-RER-All CN-WOCOV M-YD α 2 = α 6 8 M-YZ 0 0 α 4 = α 5 = α 6 9 RER-All 0 0 0