Division of Pharmacoepidemiology And Pharmacoeconomics Technical Report Series

Transcription

1 Division of Pharmacoepidemiology And Pharmacoeconomics Technical Report Series Year: 2013 #006 The Expected Value of Information in Prospective Drug Safety Monitoring Jessica M. Franklin a, Amanda R. Patrick a, Milton C. Weinstein b, Robert J. Glynn a, Sebastian Schneeweiss a a.) Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women s Hospital and Harvard Medical School, Boston, MA b.) Department of Health Policy and Management, Harvard School of Public Health, Boston, MA

2 Series Editors: Sebastian Schneeweiss, MD, ScD Jerry Avorn, MD Robert J. Glynn, ScD, PhD Niteesh K. Choudhry, MD, PhD Jeremy A. Rassen, ScD Josh Gagne, PharmD, ScD Contact Information: Division of Pharmacoepidemiology and Pharmacoeconomics Department of Medicine Brigham and Women s Hospital and Harvard Medical School 1620 Tremont St., Suite 3030 Boston, MA Tel: Fax:

3 Technical Report: The Expected Value of Information in Prospective Drug Safety Monitoring Jessica M Franklin 1 Amanda R Patrick 1 Milton C Weinstein 2 Robert J Glynn 1 Sebastian Schneeweiss 1 April 12, : Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women s Hospital and Harvard Medical School, Boston, MA 2: Department of Health Policy and Management, Harvard School of Public Health, Boston, MA 1 Introduction The Food and Drug Administration (FDA) Amendment Act of 2007 called for the creation of an active safety monitoring system for newly marketed medications with access to data on 100 million Americans by 2012 (FDA, 2007). In response to this legislation, the FDA created the Sentinel Initiative, an attempt to use existing electronic healthcare data for prospective safety monitoring of regulated medical products, augmenting the FDA s existing post-market safety surveillance systems. The Sentinel System is intended to provide the FDA with the ability to quickly answer questions about potential safety concerns and to facilitate assessment of the impact of FDA regulatory actions on public health. A key issue, therefore, is determining when evidence about a potential safety issue is sufficient to warrant regulatory action. One such action is the release of a medication safety warning, for example, the recent warnings of elevated cardiovascular risks to patients taking Avandia (rosiglitazone) for control of diabetes. If an alternative treatment with similar indications is available, for example, pioglitazone, then 1

4 patients on the alternative treatment provide a natural referent group for comparison of risks within the Sentinel System. In addition, a safety warning on one drug will likely result in increased use of the alternative treatment among future patients that have these indications. Therefore, the comparative safety of these two drugs has implications for both the decision to issue a safety warning, as well as the public health impacts of that decision. This scenario lends itself naturally to a Bayesian decision analysis. In a Bayesian decision analysis, investigators can incorporate prior information about the risks of adverse events on each treatment to evaluate the optimal strategy among the available alternatives: 1) issue a warning on the treatment that is currently believed to be less safe, thereby effectively choosing the alternative treatment for all future patients, or 2) do not issue a warning, and continue observational monitoring of the safety of both treatments. This decision reduces to determining the expected value of the additional information that will be gained from further monitoring; if this parameter is greater than the losses incurred from monitoring, then further monitoring is warranted; if less, then continuing to expose patients to the inferior treatment may not be prudent. Because the observations collected in monitoring (observed safety events in a treated population) also contribute to the utility of the monitoring decision, this problem may be viewed as a variation of the two-armed bandit problem, initially considered by Thompson (1933) in the context of clinical trials, where the specific proportion of patients on each treatment may be modified sequentially by the investigators. Similar problems have also been addressed in the literature on group sequential trials or adaptive trial design from both a frequentist perspective (for example, Pocock (1977, 1982); O Brien and Fleming (1979); Wang and Tsiatis (1987); Müller and Schäfer (2001), among others) and from a Bayesian perspective (for example, Berry and Ho (1988); Lewis and Berry (1994); Stallard et al. (1999); Cheng and Shen (2005), among others). However, all of this prior work was done in the context of clinical trials, where the goal is to minimize a loss function within the constraints of a pre-specified Type I error rate. By default, any decision rule that constrains the false positive rate discounts the utility of patient outcomes observed in 2

5 the study in favor of the outcomes of future patients. While this strategy may be reasonable in randomized clinical trials with a small number of consenting study patients, the same argument may not be made in the context of population-scale observational studies. Therefore, we aim to minimize the health losses across all patients on the horizon, both those currently receiving the drug and all future patients that might receive it. In this paper, we develop the statistical framework for calculating the expected utility of additional monitoring in the case of multiple adverse outcomes, where only one outcome has an uncertain probability of occurring on each treatment. In addition, we derive the decision rule for choosing to issue a warning or to continue monitoring after observing a given number of events on each treatment. We provide software for the calculations presented, and we apply our framework to the prospective monitoring of prasugrel, a newly-introduced medication for prevention of thrombotic cardiovascular events among patients with acute coronary syndrome, and clopidogrel, a drug with similar indications that has been on the market for several years. In Section 2, we derive the calculations needed for the Bayesian decision analysis. In Section 3, we present the application, and in section 4, we discuss the potential and limitations of this framework for evaluating the decision to issue warnings on potential safety signals. 2 Calculating the Expected Value of Monitoring We take an M-step induction approach, similar to the approach presented in Lewis and Berry (1994). We consider monitoring patients receiving one of two treatments until a pre-specified amount of person-time t has been observed on each treatment out of the total horizon of exposed person-time T on either drug. We assume that there is a pre-specified maximum number of monitoring periods allowable, M. We characterize the decisions to be made as a vector of potential actions, A = (A 0, A 1,..., A M ), where A m is the action following the m th monitoring period and M M is the total number of monitoring periods observed. We let A m = 0 indicate the decision to monitor; A m = 1 indicates the decision to choose treatment 1 and reject 3

6 treatment 2; and A m = 2 indicates the decision to choose treatment 2 and reject treatment 1. In order to observe M monitoring periods, we must choose monitoring in each period leading up to period M, so we require that A 0 = A 1 =... = A M 1 = 0. The final decision, A M, is restricted to values in {1, 2}, since at that point monitoring is terminated. We denote with A the set of vectors A that satisfy these constraints. We observe Y m = (Y m1, Y m2 ), the number of events on treatments 1 and 2, respectively, during monitoring period m. After the final monitoring period, we observe Y f, the number of events in the remaining person-time after the end of monitoring, where all patients are either on one drug or the other. We assume that each observed event count has a Poisson distribution: Y ma P oisson(tλ a ) (1) (Y f A M = a) P oisson((t 2Mt)λ a ) (2) where λ a is the event rate on treatment a, T is the total amount of person-time that will be accumulated on either drug over the life-time of the decision, and t is the amount of person-time observed on each drug in each monitoring period (which we assume, for now, to be constant). We also assume that Y m1 and Y m2 are independent for all m so that Y m1 does not contribute information about λ 2 and vice versa. Finally, we posit Gamma prior distributions for each rate parameter: λ a Gamma(α a, β a ) (3) Let D m be the sum of all events following monitoring period m: M D m = (Y k1 + Y k2 ) + Y f. (4) k=m+1 Our goal is to sequentially choose actions A A that minimize E(D 0 ), the total expected number of outcome events. This problem reduces to choosing, following each stage of monitoring, the 4

7 action a that will minimize E(D m A m = a, y 1,..., y m ). 2.1 The Base Case: One Outcome We begin by considering the initial decision, A 0, before any data has been collected via monitoring. The total number of expected events under each alternative decision is given by: E(D 0 A 0 = 1) = E(Y f A 0 = 1) (5) E(D 0 A 0 = 2) = E(Y f A 0 = 2) (6) E(D 0 A 0 = 0) = E(Y 1,1 + Y 1,2 + D 1 A 0 = 0) (7) = E(Y 1,1 ) + E(Y 1,2 ) + [ ] min 1 A 1 = a, y 1 ) P (y 1 ) (8) a {0,1,2} y 1 The expansion of D 0 into its components, given A 0, follows directly from its definition (when A 0 {1, 2}, then M = 0 and D 0 = Y f ). Therefore, given our distributional assumptions above, we can calculate equations (5) and (6) in terms of the input parameters T, t, α a, and β a, as shown in the Appendix. We can also directly calculate most of equation (8) with the exception of E(D 1 A 1 = 0, y 1 ). If A 1 = 0, then we monitor further and observe Y 2. Expanding this term further, we have E(D 1 A 1 = 0, y 1 ) = E(Y 2,1 + Y 2,2 + D 2 A 1 = 0, y 1 ). Note that this expansion is identical to equation (7) except that m = 1 is replaced by m = 2 and we are now calculating conditional on y 1. Thus, we can write equation (8) more generally for any number of prior monitoring periods 0 m < M 1, d m = E(D m A m = 0, y 1,..., y m ) = E(Y m+1,1 + Y m+1,2 + D m+1 A m = 0, y 1,..., y m ) = E(Y m+1,1 y 1,..., y m ) + E(Y m+1,2 y 1,..., y m ) + [ ] min E(D m+1 A m+1 = a, y 1,..., y m ) P (y m+1 ). (9) a {0,1,2} y m+1 Again, each of these terms may be computed directly based on our distributional assumptions 5

8 (details are in Appendix) except for d m+1 = E(D m+1 A m+1 = 0, y 1,..., y m ), which, using equation (9), may be written as a function of known quantities and d m+2. In this way, we iterate through recursive calculations until we reach m = M 1. At this point, the next monitoring period is required to be the final monitoring period, so A M = 0 is not an option. Thus, we must modify the equation for d m slightly: d M 1 = E(Y M,1 + Y M,2 + D M A M 1 = 0, y 1,..., y M 1) = E(Y M,1 y 1,..., y M 1) + E(Y M,2 y 1,..., y M 1) + [ ] min E(Y f A M = a, y 1,..., y M 1) P (y M ) (10) a {1,2} y M This equation is completely calculable in terms of the input parameters. Therefore, we may calculate E(D 0 A 0 = a) for all potential values of a, and we may choose A 0 in order to minimize this expectation. 2.2 Extending to Multiple Outcomes When considering multiple outcomes, we require that the events rates on each treatment are known for all but one outcome (we do not collect information on these outcomes during the monitoring periods). We refer to these outcomes with known event rates as secondary outcomes. One outcome, which we refer to as the primary outcome, may have unknown rates of occurrence on each treatment, as in the previous section. We also require that known preference weights are used to quantify the relative importance of each outcome. Suppose there are n secondary outcomes. Let X im = (X im1, X im2 ), and X if be defined as the event counts for outcome i {1,..., n}, similar to the event counts Y m and Y f for the primary outcome. Also define C im, the sum of all events for outcome i following monitoring period m, equivalent to the definition of D m for the primary outcome. Let w = (w 0, w 1,..., w n ) denote a vector of preference weights, where n i=0 w i = 1, w 0 refers to the weight on the primary outcome, and w i is the weight on secondary outcome i. 6

9 Our goal is now to sequentially choose actions a that minimize the expected future weighted losses, given the information collected on the primary outcome, E(L m A m = a, y 1,..., y m ), where L m = w 0 D m + n i=1 w ic im. Since we only collect information on the primary outcome event, the calculations from the previous section need only a small modification to apply to the more general case of expected losses. In equations (9) and (10), we simply replace D m with L m, and replace Y m1 and Y m2 with Y m1 + n i=1 X im1 and Y m2 + n i=1 X im2, respectively. 2.3 Varying the monitoring period length In the previous sections, we always assumed that an equal amount of person-time was observed on each treatment during each monitoring period. In practice, this assumption may often be unrealistic, especially in the context of a newly-marketed medication. When a new medication is introduced, only a small proportion of patients initiating treatment will use the new therapy. During this time, monitoring periods may be lengthened in order to observe a minimum amount of person-time on the new treatment. As time passes, more patients and prescribers will choose the new treatment, and eventually an equilibrium is reached where the proportion of patients on each treatment remains constant. Therefore, we let t m1 be the amount of person-time observed on treatment 1 during monitoring period m, and we assume that this quantity is fixed across monitoring periods. We define t m as the total amount of person-time that is observed on either of the two treatments during monitoring period m, and we assume that this quantity varies over time. In particular, we specify a function f(t) that defines the proportion of person-time on treatment 1 as a function of total observed person-time, so that for a given t 1 = t m1 we may find t m such that t 1 = t m t m 1 f(u)du (11) where t 0 = 0. The amount of person-time on treatment 2 in monitoring period m is then calculated t 2m = t m t 1. 7

10 We consider two potential functions for f. First, f(t) = c where 0 < c < 1 specifies a constant proportion of patients on each treatment (so that t m and t m2 do not vary across monitoring periods). Second, we consider an S-shaped function for uptake of the new drug: f(t) = 2ct 2 /b 2, 0 t < b/2 c 2c(b t) 2 /b 2, b/2 t < b c, t b (12) where 0 < c < 1 represents the proportion of patients on treatment 1 in the final equilibrium stage and b > 0 represents the amount of accumulated person-time required to reach that stage, as shown in Figure 1. In this figure, the total person-time per monitoring period, t m, is shown for the first 3 monitoring periods for a specific value of t 1. In order to implement these time varying treatment effects, we simply modify the distributional assumptions from equations (1) and (2): Y ma P oisson(t ma λ a ) (13) M (Y f A M = a) P oisson((t t m)λ a ). (14) m=1 References Berry, D. and Ho, C. (1988). One-sided sequential stopping boundaries for clinical trials: A decision-theoretic approach. Biometrics pages Cheng, Y. and Shen, Y. (2005). Bayesian adaptive designs for clinical trials. Biometrika 92, 633. FDA (2007). Agency Emergency Processing Under the Office of Management and Budget Review; Certification to Accompany Drug, Biological Product, and Device Applications or Submissions, volume 72 (238), pages U.S. Congress. 8

11 Figure 1: S-curve for the proportion of patients on treatment 1 as a function of the total observed person-time, where b is the amount of person-time observed prior to reaching equilibrium and c is the equilibrium level. The endpoints of the first three monitoring periods are given on the x-axis. c f(t) 0 t 1 * t t 2 * b t 3 * Lewis, R. and Berry, D. (1994). Group sequential clinical trials: A classical evaluation of bayesian decision-theoretic designs. Journal of the American Statistical Association pages Müller, H. and Schäfer, H. (2001). Adaptive group sequential designs for clinical trials: Combining the advantages of adaptive and of classical group sequential approaches. Biometrics 57, O Brien, P. and Fleming, T. (1979). A multiple testing procedure for clinical trials. Biometrics pages Pocock, S. (1977). Group sequential methods in the design and analysis of clinical trials. Biometrika 64, 191. Pocock, S. (1982). Interim analyses for randomized clinical trials: the group sequential approach. Biometrics pages Stallard, N., Thall, P., and Whitehead, J. (1999). Decision theoretic designs for phase ii clinical trials with multiple outcomes. Biometrics 55,

12 Thompson, W. (1933). On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25, Wang, S. and Tsiatis, A. (1987). Approximately optimal one-parameter boundaries for group sequential trials. Biometrics pages A Calculation details In this section, we present the detailed calculations for each equation in Section 2.1. First, for equations (5) and (6), we have E(Y f A 0 = a) = E(T λ a ) = T α a /β a. In order to calculate equations (9) and (10), we note that E(Y m+1,a y 1,..., y m ) = E(tλ a y 1,..., y m ) and E(Y f A M = a, y 1,..., y M ) = E((T 2Mt)λ a y 1,..., y M ), each of which can be calculated from the posterior distribution: m λ a y 1,..., y m Gamma(α a + y k,a, β a + mt). (15) In addition, by integrating out the rate parameters, we have the joint marginal probability distribution of Y m as the product of the independent negative Binomial distributions for each Y ma. Thus, we may rewrite equation (9) in terms of input parameters: k=1 d m = t α 1 + m k=1 y k1 β 1 + mt + [ min P (y m+1 ) = NB y m+1 ( y m+1,1 ; α t α 2 + m k=1 y k2 β 2 + mt { α 1 + m+1 k=1 y k1 β 1 + (m + 1)t, α 2 + m β 1 + mt y k1, β 1 + (m + 1)t k=1 m+1 k=1 y k2 β 2 + (m + 1)t, d m+1 ) NB ( }] y m+1,2 ; α 2 + P (y m+1 ) (16) ) m β 2 + mt y k2, β 2 + (m + 1)t k=1 (17) 10