Conditional Inference in Two-Stage Adaptive Experiments via the Bootstrap

Size: px

Start display at page:

Download "Conditional Inference in Two-Stage Adaptive Experiments via the Bootstrap"

Lindsay Terry
5 years ago
Views:

1 Conditional Inference in Two-Stage Adaptive Experiments via the Bootstrap Adam Lane, HaiYing Wang, and Nancy Flournoy Abstract We study two-stage adaptive designs in which data accumulated in the first stage are used to select the design for a second stage. Inference following such experiments is often conducted by ignoring the adaptive procedure and treating the design as fixed. Alternative inference procedures approximate the variance of the parameter estimates by approximating the inverse of expected Fisher information. Both of these inferential methods often rely on normal distribution assumptions in order to create confidence intervals. In an effort to improve inference, we develop bootstrap methods that condition on a non-ancillary statistic that defines the second stage design. Introduction In many experiments, the evaluation of designs with respect to a specific objective requires knowledge of the true model parameters. Examples include optimal designs for the estimation of nonlinear functions of the parameters in linear models; optimal designs for the estimation of linear functions of parameters in nonlinear models; and dose-finding designs where it is desired to treat patients at a pre-specified quantile of the response function. In the absence of perfect knowledge of the model parameters, it is appealing to update initial parameter estimates using data accumulated from all previous stages and to allocate observations in the current stage by an assessment of designs evaluated at these estimates. Such procedures result in designs that are functions of random variables whose distributions depend on the model parameters, i.e., the designs are not ancillary statistics. A. Lane ( ) Cincinnati Children s Hospital Medical Center, 3333 Burnet Avenue, Cincinnati, H 459, USA Adam.Lane@cchmc.org H. Wang University of New Hampshire, 33 Academic Way, Durham, NH 0384, USA HaiYing.Wang@unh.edu N. Flournoy University of Missouri, 46 Middlebush Hall, Columbia, M 65, USA flournoyn@missouri.edu Springer International Publishing Switzerland 06 J. Kunert et al. (eds.), mda - Advances in Model-riented Design and Analysis, Contributions to Statistics, DI 0.007/ _0 73

2 74 A. Lane et al. In practice it is common to conduct inference ignoring the adaptive nature of the experiment and treating the design as if it were an ancillary statistic. The potential issues resulting from such procedures are well known; see []. Briefly, when a design is ancillary, it is non-informative with respect to model parameters. Analysing adaptive experiments with non-ancillary designs as if the design were ancillary does not account for all the information contained in the design. An alternative often suggested is to approximate the variance of the maximum likelihood estimate (MLE) with an approximation of the inverse of the expected Fisher information (defined below). Regardless of the method, the distribution of the MLE resulting from adaptive experiments are often assumed to follow a normal distribution when used to create confidence intervals. We develop three bootstrap procedures that approximate the distribution of the MLE conditional on a non-ancillary statistic that defines the second stage design. The bootstrap reduces the reliance on the assumption of normality. A bootstrap procedure in the context of adaptive experiments with binary outcomes, primarily in the context of urn models, has been developed previously [7]. The Model and an Illustration of the Problem Consider an experiment conducted to measure a constant. Suppose it is possible to obtain unbiased observations y of from two sources; each source k D r; s produces errors from a N.0; k /,where r s are known constants. Throughout this paper Y and y denote the random variable and its observed realization, respectively. A function./ is defined to determine which of the two sources should be used based on maximizing efficiency, ethics, cost or other considerations. For illustration, we use./ D r if <cand s otherwise, given some constant c. The adaptive procedure is as follows: obtain an initial cohort of n independent observations, y D.y ;:::;y n /, from source r. Evaluate at the MLE of based on first stage observations; D y D P n jd y j=n. The function.y / determines the source from which a second cohort of n independent observations, y D.y ;:::;y n /, will be obtained and induces a correlation between Y and Y. This model has been used to illustrate conditional inference in experiments with ancillary designs; see [] and[4]. Here we use the model to illustrate conditional inference with non-ancillary designs in adaptive experiments. Let N D n C n represent the total sample size; and let S r D.; c/ and S s D.c; / be the regions of the parameter space that selects the second stage source. The second stage design is completely determined by the non-ancillary variable I.Y S k /, k D r; s, wherei./ is the indicator function. Therefore, Y jy S k is independent of Y, k D r; s. The log likelihood is l Dn.y / =r n.y / =.y / C constant. Defining w.y / D.n =r Cn =.y / /,themle based on both stages is D w.y /.n y =r C n y =.y / /.

3 Conditional Inference in Two-Stage Adaptive Experiments via the Bootstrap 75 To approximate the distribution of.y ; Y /jy S k, k D r; s, three bootstrap methods are developed. Resulting parameter estimates and confidence intervals are compared to analyses where the variance of the MLE is approximated using the inverse of the observed and expected Fisher information: I. / D Œ@ l =@ D D w.y / and F D EŒI. /, respectively. Confidence intervals for comparative methods are then constructed using normal quantiles. If the design were ancillary, then the variance of the MLE would be w.y /.UsingF to approximate the variance, does not treat the design as ancillary. But it is a function of and must be estimated; F is used here. In contrast, since Y jy S k is independent of Y,.Y ; Y /jy S k has variance h i Var.Y ; Y /jy S k D w k n 4 r Var n Y jy S k C k ; where w k D.n =r C n =k /. The design of the second stage is completely determined by I.Y S k /. Therefore f.y /jy S k gdk. The function w.y / depends on Y only through.y /. Hence fw.y /jy S k gdw k,fork D r; s. Let p k D P.Y S k /, E k D EŒ.Y ; Y /jy S k, V k D VarŒ.Y ; Y /jy S k and I k D I. /, k D r; s for short. From a series of 0,000 simulations, Table presents, for k D r; s, p k, E k, Nbse and tail probabilities, PŒ < C l and PŒ > C u,wherebse is either VarŒ.Y ; Y /, V k, Ik or F ; C l D Z = bse, C u D C Z = bse and Z is the quantile of the standard normal distribution. Values of D, r D, s D 3, n D 50, n D 50, c D C = p n and D 0:05 are used throughout. Both Ik and F are significantly greater than V k, k D r; s. BothV k, k D r; s, are significantly less than VarŒ.Y ; Y /. IfY S r,thene r is nearly unbiased, and despite slightly unbalanced tail probabilities, overall coverage would be adequate if V r were known and could be used. When Y S s, the bias is considerable and the coverage is unacceptable. We focus throughout on improvements when Y S s. Table Conditional probability, conditional expectation, variance approximation method along with its estimate and tail probabilities by the source of the second stage observations Source p k E k Variance approximation Nbse PŒ < C l PŒ > C u r V r VarŒ.Y ; Y / Ir F s V s VarŒ.Y ; Y / Is F

4 76 A. Lane et al. 3 Conditional Bootstrap Inference In an effort to improve inference following adaptive experiments, three bootstrap methods are developed. We begin with a straightforward and intuitive conditional bootstrap procedure. Unfortunately, as will be discussed, the conditions required for this procedure to give accurate inference are extremely restrictive. Conditional Bootstrap Method (BM). Construct a probability distribution, F i, by putting mass =n i at each point y i ;:::;y ini, i D ;. With fixed F and F, draw random samples of size n and n from F and F, respectively. Denote the vector of bootstrap samples as y i ; i D ;.. Repeat step B times. For the bootstrap samples satisfying y S.y /,useny and Ny to find.use.c l ; C u / D.Q = ; Q = / as the. / confidence interval, where Q is the quantile of the bootstrapped sample distribution.y ; Y /jy S.y /. Let P be the empirical probability measure given Y and let Œ b represent the bth sampled bootstrap. Then P.Y S.y // P B bd I.Œy b S.y //=B is the probability the mean of a bootstrap sample is in S.y /. Table presents the simulation results p s,e s,v s, C l, C u,pœ < C l and PŒ > C u. The first row results are for the distribution of.y ; Y /jy S s.the second and third row results use Is and F, and tail probabilities found using Table Results are conditional on the region Y S s. Method describes the procedure by which the approximate distribution was obtained; p s is approximated by P B bd I.Œy b S s /=B for the bootstrap and by P.Y S s j D / for the expected information. E s,v s, the confidence limits and tail probabilities are also provided. All values are averages from 0,000 simulations except the column p s which uses the median. True refers to either the empirical distribution of.y ; Y /jy S s or Q.Y ; Y /jy S s obtained from simulation Estimate Method p s E s N*V s C l C u PŒ < C l PŒ > C u True F Is BM C BM Q True Q Q Q C BM

5 Conditional Inference in Two-Stage Adaptive Experiments via the Bootstrap 77 quantiles of the standard normal distribution. The fourth row results are from BM using B D 000 bootstrap samples. All values are averages except p s whose values are skewed and for which the median provides a better summary of the data. Both P.Y S s/ and P.Y S s j D / are significantly greater than p s. The methods BM and F produce similar variances, both of which are less than I s.all three approximations are significantly greater than V s. However, the mean of the bootstrap distribution has greater bias than. Thus BM does not improve inference in comparison to using F. 3. Technical Details for the Conditional Bootstrap In this section we consider the MLE conditional on an indicator function of the first stage sample mean. This illustrates the implications of conditioning in two-stage adaptive experiments with non-ancillary designs and why the inference produced by BM was poor. Let T k D.a k ; d k /,wherea k D C a 0 k =p n, d k D C dk 0 =p n, a 0 k and dk 0 are constants. This is a slightly more general division of the parameter space than S k which has only one of a 0 k and d0 k finite. Note, we consider the parameter space defined in a local neighborhood of. This is done so that P.Y S k /, k D r; s has positive probability not equal to for large n ; otherwise the design would be deterministic for large first stage sample sizes and not adaptive. Consider h P p n Y Y < xj p n Y Y p i n.a k ;d k / h P f p n Y Y < xg\f p n.y Y / p i n.a k ;d k /g D h pn P.Y Y / p i n.a k ;d k / D P f p n Y < xg\f p n.y / p n.a k ;d k /g C o./ P p n.y / p n.a k ;d k / C o./ D P p n Y < xj p n Y p n.a k ;d k / C o./; () where x R, and the equalities hold almost surely under P. Note, from the left hand side of equation (), that a conditional bootstrap should include bootstrap samples satisfying p n Y Y p n.a k ;d k /,i.e.,y.a k " ; d k " /,where " D Y. BM considers bootstrap samples satisfying Y.a k; d k / and hence

6 78 A. Lane et al. its poor performance. Unfortunately, is unknown and must be estimated. In (), if one can replace with an estimator (say Q ) that converges to at a rate faster than p n,then h P p n Y Y < xj p n Y Y p n.a k ;d Q k / Q i D P p n Y < xj p n Y p n.a k ;d k / C o P./: () In a single stage experiment this would not be possible. However, in a twostage adaptive experiment in which n D o.n /,suchanestimatormayexist. Theoretically this is somewhat unsatisfactory, but practically this is interesting especially since it has been shown that two-stage experiments are optimized when n D. p n /;see[5]and[6]. It is also common, for practical or logistical reasons, that a small pilot study precedes a much larger follow-up. Note Y jy T k TN.; k =n I T k /,wheretn denotes a truncated normal distribution. Suppose an estimator,, Q exists as described and let QT k D.a k Q" ; d k Q" /,whereq" D Y.Then() Q implies fy jy QT k g can be approximated by a TN.y ;k =n I QT k / and therefore h i E Y Y jy QT k Y C b k. /I Q (3) h i Var Y Y jy QT k k n C Q.a k/f Q.a k/g Q.d k/f Q.d k/g U n : b k. /o Q where b k./ D k Œf.a k /gf.d k /g=. p n U/,.a k / D p n.a k /= k, U D f.b/g f.a/g;./ and./ represent the probability density function (PDF) and cumulative distribution function (CDF) of the standard normal distribution, respectively. Note a conditional bootstrap leads to an additional bias term in the expectation in equation (3). This must be accounted for in any conditional bootstrap procedure. 4 Adjusted Conditional Bootstrap Methods The bootstrap methods developed in this section are predicated on the existence of an estimator that converges to at a rate faster than p n. Provided standard regularity conditions hold and n D o.n /, such an estimator will always exists in the form of the second stage MLE conditional on the second stage design. However, in cases where the bias has an explicit form, it is possible to obtain Q by adapting

7 Conditional Inference in Two-Stage Adaptive Experiments via the Bootstrap 79 the bias reduction method suggested in [8]. Note h i E.Y ; Y /jy S k D w k E n Y =k C n Y =k jy S k D w k n = r C n = k C wk n b k./= r D C w k n b k./= r : Let b k./ D w k n b k./=r. Then a bias corrected estimate of is Q D b k. /. Q The Newton-Raphson method was used to solve this equation. ne iteration was sufficient, that is, Q D k. /,where k. / D b k. /=ŒC.@b k./=@/ D was used for analytic expressions and numeric calculations. Now we develop a bootstrap method that adjusts the conditioning region per the discussion in Sect. 3.. Let QS r Df; c Q" g and QS s DfcQ" ; g, where Q" D Y. Q Adjusted Conditional Bootstrap Method (BM): Repeat BM keeping only bootstrap samples satisfying y QS.y /. Let C D b.y /. / Q and.cl C ; Cu C/ D.QC =; QC = / as the. / confidence interval, where QC is the quantile of the bootstrap sample distribution of C.Y ; Y /jy QS.y /. Note C is used in place of to correct for the additional bias term previously discussed. A concern in inference when the variance of the MLE depends on the parameter estimates is how sensitive the approximate distribution is to this estimate. Figure (top left) plots the histogram of the simulated distribution of f.y ; Y /jy S s g E s (solid line). In the same figure, for simulations that correspond to the 0.05, 0.50 and quantiles of, histograms of the bootstrap sample distributions of f C.Y ; Y /jy QS s g (dotted, dashed and dot-dashed line) are plotted. This figure illustrates how well BM works across the domain of at approximating the distribution.y ; Y /jy S s. Compare this to Fig. (top right) which plots the PDF of N.0; F / for F evaluated at the same quantiles of. The bootstrap / and it provides a better approximation to the shape of the target distribution. The fifth row of Table shows that with BM P.Y QS s / is nearly equal to p s ; E Œ C.Y ; Y /jy QS s is approximately equal to E s ;Var Œ C.Y ; Y /jy QS s is only slightly greater than V s ; and the confidence interval endpoints correspond to the quantiles of. It is clear that BM provides a more accurate representation of the conditional distribution of than the alternatives. However, BM still fails to provide adequate coverage. This is not a problem with the bootstrap but rather a problem with the distribution of being tightly centered around the wrong value. distribution is less sensitive to the value of than N.0; F

8 80 A. Lane et al. Considering the poor coverage of BM, we develop a bootstrap procedure for the bias corrected estimator Q. Recall Q D k. /, k D r; s; is simply a function of. Thus BM can be adapted to estimate its distribution. Because the distribution of the bias is skewed, bias corrected bootstrap confidence intervals were used; see [3]. Bias Adjusted Conditional Bootstrap Method (BM3): Replace C in BM with Q C D C.y /. C /. Use.C l ; C u / D Œ b CDF f.v 0 C Z = /g; b CDF f.v 0 Z = /g,where b CDF.t/ D P Œ Q C.Y ; Y /jy QS.y / < t and v 0 D b CDF. Q /. Since Q is a function of, for comparison we approximate the variance of Q with Q D Œf@..y /.//=@g F D. Figure (bottom left) plots the histogram of the simulated distribution of f.y Q ; Y /jy S s geœ.y Q ; Y /jy S s (solid line). In the same figure, for simulations that correspond to the 0.05, 0.50 and quantiles of, Q histograms of the bootstrap sample distributions of f QC.Y ; Y /jy QS s g Q (dotted, dashed and dot-dashed line) are plotted. For comparison, Fig. (bottom right) plots the probability density functions of a N.0; Q / with Q evaluated at the same quantiles of. Q nce again we see that Fig. Histograms of bootstrap distributions f C.Y ; Y /jy QS s g (top left) and f Q C.Y ; Y /jy QS s g Q (bottom left). Probability density functions of N.0; F / (top right) and N.0; Q / (bottom right). The dotted, dot dashed and dashed lines correspond to the bootstrap distribution or expected information for the 0.05, 0.50, and of quantiles of or Q. In each case the solid line is the histogram of the distribution of f.y ; Y /jy S s ge s or f Q.Y ; Y /jy S s geœq.y ; Y /jy S s

9 Conditional Inference in Two-Stage Adaptive Experiments via the Bootstrap 8 Table 3 Mean square error (MSE) for the true distribution of and Q along with the bootstrap MSE of C or Q C for the case when Y s Estimate Method MSE True 4.9 C BM 4.98 Q True 5.5 Q C BM3 5.7 the BM3 well approximates the distribution.y Q ; Y /jy S s across most of the domain of, Q perhaps with the exception of the 0.05 quantile. The sixth row of Table shows the simulation results the distribution of Q.Y ; Y /jy S s. The seventh and eighth rows are results for Q and BM3, respectively. BM3 provides an unbiased estimate of the mean; slightly overestimates the VarŒ.Y Q ; Y /jy S s ; and provides confidence limits which coincide with the correct quantiles of. Q Coverage is slightly less than the nominal level, but is a significant improvement over inference methods for. Note using Q significantly overestimates VarŒ.Y Q ; Y /jy S s and slightly skews the confidence limits. The performance of C and QC reflects the bias versus variance tradeoff. Table 3 compares the mean square error (MSE) for the simulated distribution of and Q along with the bootstrap MSE of C and QC for the case when.y / D s. Despite the shortcomings of the procedure for C, it still provides a lower MSE than QC in this example. References. Baldi Antognini, A., Giovagnoli, A.: n the asympotic inference for response-adaptive experiments. METRN Int. J. Stat. 49, 9 46 (006). Dette, H., Bornkamp, B., Bretz, F.: n the efficiency of two-stage response-adaptive designs. Stat. Med. 3, (03) 3. Efron, B.: Nonparametric standard errors and confidence intervals. Can. J. Stat. 9, (98) 4. Efron, B., Hinkley, D.V.: Assessing the accuracy of the maximum likelihood estimate: observed versus expected Fisher information. Biometrika 65, (978) 5. Lane, A., Yao, P., Flournoy, N.: Information in a two-stage adaptive optimal design. J. Stat. Plan. Inference 44, (04) 6. Pronzato, L., Pazman, A.: Design of Experiments in Nonlinear Models. Springer, New York (03) 7. Rosenberger, W.F., Hu, F.: Bootstrap methods for adaptive designs. Stat. Med. 8, (999) 8. Whitehead, J.: n the bias of maximum likelihood estimation following a sequential test. Biometrika 73, (986)

Information in a Two-Stage Adaptive Optimal Design

Information in a Two-Stage Adaptive Optimal Design Department of Statistics, University of Missouri Designed Experiments: Recent Advances in Methods and Applications DEMA 2011 Isaac Newton Institute for