The bootstrap and Markov chain Monte Carlo

Transcription

1 The bootstrap and Markov chain Monte Carlo Bradley Efron Stanford University Abstract This note concerns the use of parametric bootstrap sampling to carry out Bayesian inference calculations. This is only possible in a subset of those problems amenable to MCMC analysis, but when feasible the bootstrap approach offers both computational and theoretical advantages. The discussion here is in terms of a simple example, with no attempt at a general analysis. Keywords: Bayes credible intervals, Gibbs sampling, MCMC, importance sampling 1 Introduction Markov chain Monte Carlo (MCMC) is the great success story of modern-day Bayesian statistics. MCMC, and its sister method Gibbs sampling, permit the numerical calculation of posterior distributions in situations far too complicated for analytic expression. Together they have produced a forceful surge of Bayesian applications in our journals, as seen for instance in Li et al. (2010), Chakraborty et al. (2010), and Ramirez-Cobo et al. (2010). The point of this brief note is to say that in some situations the bootstrap, in particular the parametric bootstrap, offers an easier path toward the calculation of Bayes posterior distributions. An important but somewhat under-appreciated article by Newton and Raftery (1994) made the same point using nonparametric bootstrapping. By going parametric, we can illustrate more explicitly the bootstrap/mcmc connection. The arguments here will be made mainly in terms of a simple example, with no attempt at the mathematical justifications seen in Newton and Raftery. It is not really surprising that the bootstrap and MCMC share some overlapping territory. Both are general-purpose computer-based algorithmic methods for assessing statistical accuracy, and both enable the statistician to deal effectively with nuisance parameters, obtaining inferences for the interesting part of the problem. On the less salubrious side, both share the tendency of general-purpose algorithms toward overuse. Of course the two methodologies operate in competing inferential realms: frequentist for the bootstrap, Bayesian for MCMC. Here, as in Newton and Raftery, we leap over that divide, bringing bootstrap calculations to bear on the Bayesian world. The working assumption is that we have a Bayesian prior in mind and the only question is how to compute its posterior distribution. Arguments about the merits of Bayesian versus frequentist analysis will not be taken up here, except for our main point that the two camps share some common ground. Research supported in part by NIH grant 8R01 EB and by NSF grant DMS

2 Section 2 develops the connecting link between bootstrap and Bayesian calculations, mainly in terms of a simple component of variance model. The bootstrap approach to Bayesian inference for this model is carried out in Section 3, and then compared with Gibbs sampling. We end with some brief comments on the comparison. 2 Bootstrap and Bayes computations We consider the following components of variance (or random effects) model: a N n (α 0 1, σ 2 0I) and y a N n (a, I). (2.1) That is, a 1, a 2,..., a n the unobserved random effects are an independent sample of size n from a normal distribution having mean α 0 and variance σ0 2 ; we get to see the n independent normal variates y i ind y i N (a i, 1) for i = 1, 2,..., n. (2.2) Marginally, integrating out the unobserved random effects a i, where y i ind N (α 0, σ 2 ) i = 1, 2,..., n, (2.3) σ 2 = σ (2.4) is the total variance. In this section we will concentrate on assessing the variability of the bivariate parameter the maximum likelihood estimate (mle) of β is ( ˆβ = (ˆα 0, ˆσ 2) = β = (α 0, σ 2 ). (2.5) ȳ, ) n (y i ȳ) /n 2, (2.6) i=1 ȳ = n 1 y i/n; ˆβ is sufficient for β, with distributions ˆα 0 N (α 0, σ 2 /n) independent of ˆσ 2 σ 2 χ 2 n 1/n, (2.7) where χ 2 n 1 indicates a chi-squared variate having n 1 degrees of freedom. A parametric bootstrap replication ˆβ is obtained by substituting ˆβ = (ˆα 0, ˆσ 2 ) for the parameter value β = (α 0, σ 2 ) in (2.7), ˆα 0 N (ˆα 0, ˆσ 2 /n ) independent of ˆσ 2 ˆσ 2 χ 2 n 1/n. (2.8) Here ˆβ = (ˆα 0, ˆσ 0 2) is fixed at its observed value, while ˆβ = (ˆα 0, ˆσ2 ) varies according to (2.8). To harmonize notation with that for Bayesian inference we will denote ˆβ simply β, that now indicates any point in the sample space B of possible β s, not necessarily a true value. Suppose that θ = t(β) is any function of β, for example θ = σ 2 /α 0, or { 1 if σ 2 2 θ = 0 if σ 2 (2.9) < 2. 2

3 Starting with a prior density π(β) on β, Bayes theorem says that θ has posterior expectation, given ˆβ, / E{θ ˆβ} = t(β)π(β)g β ( ˆβ) dβ π(β)g β ( ˆβ) dβ, (2.10) B B where g β ( ˆβ) is the likelihood function: the density of ˆβ = (ˆα 0, ˆσ 2 ) in (2.7) (now fixed at its observed value) as β = (α 0, σ 2 ) varies over B. From (2.7) we calculate g β ( ˆβ) { = c exp n [ 2σ 2 (α 0 ˆα 0 ) 2 + ˆσ 2]} / (σ 2 ) n/2 (2.11) for the components of variance problem, where c is a constant that does not depend upon β. Define the conversion ratio R(β) = g β ( ˆβ)/g ˆβ(β), (2.12) the denominator being the bootstrap density (2.8), with β = (α 0, σ 2 ) denoting ˆβ = (ˆα 0, ˆσ2 ). We can rewrite (2.10) as / E{θ ˆβ} = t(β)π(β)r(β)g ˆβ(β) dβ π(β)r(β)g ˆβ(β) dβ. (2.13) B B Now the integrals are over the bootstrap density g ˆβ(β) rather than the likelihood function in (2.10). This allows E{θ ˆβ} to be directly estimated via bootstrap sampling. Suppose we draw a parametric bootstrap sample by independently sampling B times according to (2.8), g ˆβ β 1, β 2,..., β B. Let R i = R(β i ), π i = π(β i ), and t i = t(β i ). Both the numerator and denominator of (2.13) are expectations with respect to the density g ˆβ(β). These can be estimated by bootstrap sample averages, leading to the estimate of E{θ ˆβ}, Ê{θ ˆβ} = B / B t i π i R i π i R i. (2.14) i=1 In this way, any Bayesian posterior expectation can be evaluated from parametric bootstrap replications. None of this is very novel, except for the focus on the parametric bootstrap: (2.14) is a standard importance sampling procedure, as described in Chapter 23 of Lange (2010). A connection between the nonparametric bootstrap and Bayesian inference was suggested under the name Bayesian bootstrap in Rubin (1981), and also in Section 10.6 of Efron (1982). Newton and Raftery (1994) make the connection more tangible, applying (2.14) with nonparametric bootstrap samples. Parametric bootstrapping makes the connection more tangible still, since in favorable situations we can write down and examine explicit representations for the conversion factor R(β) (2.12). For our components of variance problem, (2.11) and the related expression for g ˆβ(β) yield ( ) σ 2 3/2 R(β) = ˆσ 2 exp n ( {(α 0 ˆα 0 ) 2 1σ 2 2 1ˆσ ) ( ˆσ σ 2 σ2 ˆσ 2 i=1 ) + 2 log σ2 ˆσ 2 }. (2.15) The factor R is interesting in its own right. It relates sample space integration to parameter space integration, to quote Fraser et al. (2010), investigating a related problem. 3

4 A considerable advantage of bootstrap estimate (2.14), compared to its Gibbs sampling or MCMC analogues, is that an accuracy estimate for Ê{θ ˆβ} is available from the same data used in its calculation. In (2.14) define r i = π i R i and s i = t i π i R i, (2.16) so Ê{θ ˆβ} = s/ r. (2.17) Let Ĉov be the usual 2-by-2 sample covariance matrix calculated from the B vectors (s i, r i ), say ) (ĉss ĉ Ĉov = sr. (2.18) (For example, ĉ sr = (s i s)(r i r)/b.) Then standard delta-method calculations yield ĉ rs ĉ rr }.= Ê{θ β} var {Ê{θ β} 2 ) (ĉss B s 2 2ĉsr s r + ĉrr r 2. (2.19) Formula (2.19) is possible because the bootstrap vectors (s i, r i ) are drawn independently of each other, which is not the case in MCMC sampling. 3 An example As an example of the bootstrap/bayes computations of Section 2, a simulated data vector y was generated according to (2.1), with n = 100, α 0 = 1, and σ 2 0 = (3.1) Marginally, the independent obervations y i had distribution as in (2.3). The maximum likelihood estimates (2.6) were y i ind N (α 0, σ 2 ) [σ 2 = 1.25] (3.2) ˆβ = (ˆα 0, ˆσ 2) = (1.005, 1.295). (3.3) Here we will only concern ourselves with calculating the posterior distribution of σ 2. B = 25, 000 bootstrap replications (many times more than necessary, as we shall see), β 1, β 2,..., β B, were generated from the bootstrap distribution g ˆβ(β) (2.8), and the conversion factors R i = R(β i ) (2.12) calculated. The prior density π(β) was taken to be π(β) = 1/σ 2. (3.4) (Or equivalently, π(β) = c/σ 2 for any positive constant c, since c factors out of Bayesian calculations, as seen in (2.10).) Formula (3.4) is Jeffreys uninformative prior for inferences concerning the total variance σ 2. Finally, the adjusted factors r i = π i R i = π(β i )R(β i ) were calculated. We now have B = 25, 000 points β i, each with associated weight r i, and these can be used to estimate any posterior expectation E{θ ˆβ}, as in (2.16) (2.17). Table 1 describes the posterior distribution of the total variance σ 2. The interval [0.75,2.15], which contains almost all of the bootstrap replications σi 2, has been divided into bins of width 4

5 Boot Bayes Table 1: Top row: bin counts for B = 25, 000 parametric bootstrap replications of σ 2, as in (2.8); for example, 497 values σi 2 fell into second bin,.85 σ2 i <.95. Bottom row: weighted bin counts (3.5), for posterior density of σ 2 given y; using Jeffreys invariant prior density 1/σ each. The top row of the table shows the bootstrap histogram counts, for example, 1707 of the σi 2 values fell into the third bin, 0.95 σ2 i < The bottom row shows the Bayesian bin counts, weighted according to r i = π i R i, Bayes k = B kth bin / B r i r i, (3.5) scaled to sum to B. The weights r i were smaller than 1 in the third bin, resulting in fewer Bayes counts, 1010 compared to The opposite happens at the other end of the scale. 1 count > bootstrap Jeffreys and Bca sig2tot > Figure 1: Distribution of total variance σ 2 given ˆβ (3.3); dashed curve counts from 25,000 parametric bootstrap replications (2.8), Boot row of Table 1; solid curve weighted counts (3.5), weights r i = π(β i )R(β i ), with π(β) the Jeffreys prior 1/σ 2 and R(β) the conversion factor (2.15). Corrected bootstrap distribution Bca agrees with Jeffreys. Figure 1 graphs the Boot and Bayes counts of Table 1. We see that the Jeffreys Bayes distribution is shifted to the right of the Boot distribution. It can be shown that the Jeffreys prior π(β) = 1/σ 2 is almost exactly correct from a frequentist point of view: for example, the upper 90% point for the Jeffreys posterior distribution is almost exactly the upper endpoint of the usual 90% one-sided confidence interval for σ 2. (The Welch Peers theory of 1963 shows this 5

6 kind of Bayes-frequentist agreement holding asymptotically for all priors of the form π(β) = h(α 0 )/σ 2, with h( ) any smooth positive function. The components of variance problem is unusual in allowing a simple expression for the Welch Peers prior; see Section 6 of Efron (1993).) α se cv σ 2 [α] se cv Table 2: Estimated posterior quantiles σ 2 [α] for total variance σ 2, given data (3.3) and Jeffreys prior π(β) = 1/σ 2. Also shown are standard errors (se) and coefficients of variation (cv), obtained using (2.19), as explained in text. Table 2 shows the estimated posterior quantiles σ 2 [α] for σ 2, given data (3.3) and Jeffreys prior (3.4); for example, σ 2 [0.95] = 1.69 is the empirical 95th percentile of the 25,000 bootstrap replications weighted according to r i = π i R i as in (3.5). Table 2 also shows standard errors derived from (2.19), i.e., the variability due to working with B = 25, 000 bootstrap replications (rather than B = ). As an example, let θ = t(β) in (2.9) equal 0 or 1 as σ 2 i is greater or less than 1.69, the value of σ 2 [0.95]. Following through (2.16) (2.17) results in as the standard error of Ê{θ ˆβ} = 0.95, and this is the se value shown in the second column s next-to-bottom row, along with cv = (= /( )). The standard error for ˆσ 2 [α] = 1.69 is then obtained from the usual delta-method approximation /g[0.95] = , where g[0.95] is the height of the Jeffreys prior density curve at All of the standard errors are small. If we had stopped at B = 5000 bootstrap replications, they would be multiplied by 5 = 2.2, and would still indicate good accuracy. At B = 1000 the multiplying factor increases to 5, and we might begin to worry about simulation errors. All of these calculations depend on the adjusted conversion factor r i = π i R i, with π i = 1/σ 2 i and R i = R(β i ) as in (2.15). Figure 2 plots r i versus σ 2 i for the B = 25, 000 bootstrap replications from (2.8). These are less or greater than 1 as σ 2 i is less or greater than ˆσ2 = 1.30, accounting for the rightward shift of the Bayes posterior density in Figure 1. Much of the simulation standard error seen in Table 2 comes from the large values of r i on the right side of Figure 2. Strategems for decreasing simulation error exist. In this case we might modify the bootstrap sampling recipe (2.8), say to ˆσ 2 = 2ˆσ 2 χ 2 n 1/n. (3.6) This would extend the dashed curve rightwards in Figure 1, reducing the large values of R i and r i. We will not worry further about estimation efficency here. The Bca system of confidence intervals ( bias-corrected and adjusted, Efron, 1987) adjust the raw bootstrap distribution represented by the dashed curve in Figure 1 to achieve 6

7 r > sig2tot > Figure 2: Adjusted conversion factor r i = π i R i for the B = 25, 000 bootstrap replications, plotted versus the bootstrap replications σi 2; the factor is less or greater than 1 as σ2 i is less or greater than ˆσ 2 = 1.30, moving the Jeffreys posterior density rightward in Figure 1. second-order accurate frequentist coverage: a level-α Bca interval has true average probability α + O(1/n), n the sample size, rather than α + O(1/ n) for the usual first-order approximate intervals. The corresponding confidence distribution (e.g., assigning probability 0.01 to the region between the 0.90 and 0.91 Bca interval endpoints) requires a different conversion factor R Bca (β), that does not involve any Bayesian input. In the example of Figure 1, the Bca confidence density almost perfectly matches the Jeffreys posterior density. All of the calculations so far have concerned the total variance σ 2 = σ We might be more interested in σ0 2 itself, the variance of the random effects a i in model (2.11). Simply shifting the posterior densities of Figure 1 a unit to the left has an obvious defect: some of the probability (0.051 for the raw bootstrap, for Jeffreys/Bca) winds up below zero, defying the obvious constraint σ We need a prior density π(β) that has only positive support for σ0 2. The Jeffreys analogue π(β) = 1/σ0 2 looks promising, but fails because of its infinite spike at zero. (Unlike σ 2, σ0 2 = σ2 1 has substantial likelihood near 0 given the observed data (3.3).) The toy example in Spiegelhalter et al. (1996, Sect. 2.3) suggests using prior density π(β) : α 0 N (0, V 0 ) independent of σ 2 0 ν 0 /G ν0, (3.7) V 0 = 10, 000 and ν 0 = 0.01, in a situation similar to (2.11), with G ν0 indicating a gamma distribution having shape parameter ν 0. Prior or hyperprior assumptions like (3.7) are familiar in the MCMC literature. We can reweight our B = 25, 000 bootstrap replications β i to compute the posterior distribution of σ0 2 for prior (3.7), as in (3.5); R i is still provided by (2.15) (with σ 2 = σ ); the Bayes prior is π i = c exp{ ν 0 /σ0i} 2 / σ 2(ν 0+1) 0i, (3.8) 7

8 from the inverse gamma distribution in (3.7), the α 0 N (0, V 0 ) portion of the prior having almost no effect. mcmc counts posterior density inverse gamma prior original bootstrap sigma0^2 Figure 3: Heavy pointed curve indicates posterior density of σ 2 0 in model (2.1), data (3.3); inverse gamma prior (3.7) as calculated from weighted bootstrap replications. Stepped curve is same posterior density calculation by Gibbs sampling algorithm (3.9). Dashed curve shows unweighted bootstrap counts of Figure 2 shifted one unit left. The heavy pointed curve in Figure 3 shows the posterior distribution of σ0 2 under model (3.7), as calculated from the weighted bootstrap replications. It looks completely different than a simple leftward translation of the curves in Figure 1. Now the posterior density has its high point at the minimum allowable value σ0 2 = 0, and decreases monotonically to the right. A Gibbs sampling analysis was performed, as a check on the weighted bootstrap results. Given the data y, or equivalently given the sufficient statistics (ˆα 0, ˆσ 2 ) (3.3), a Gibbs analysis cycles through the three unobserved quantities in (2.11), namely a, α 0, and σ0 2. The complete conditional distribution of each one given the other two is calculated from the specifications (2.1) and (3.7): (( ) ) [ a α 0, σ0 2 α0 N ν σ y /h 1, I/h 1 h 1 = 1 ] σ ( ) [ nā α 0 a, σ0 2 N /h σ0 2 2, 1/h 2 h 2 = 1 + n ] V 0 σ0 2 (3.9) [ ] n σ0 α 2 0, a N (ν 0 + S/2)/G ν0 +n/2 S = (a i ā) 2. Here n = 100 is the sample size, ā = a i /n, 1 is a vector of n 1 s, I is the n n identity matrix, and G ν indicates a gamma random variable with shape parameter ν. Starting from α 0 = 0 and σ 2 0 = 1 as initial values, 25,000 cycles through (3.9) produced 25,000 values σ2 0i. 1 8

9 Their step histogram in Figure 3 agrees reasonably well with the bootstrap results. (Formula (2.19) suggests that the bootstrap estimates are nearly exact.) A comparison of Figure 3 with Figure 1 raises some important points: The inverse gamma prior distribution (3.7) (3.8) is far from uninformative. The posterior distribution of σ 2 0 = σ2 1 is strongly pulled toward zero, putting nearly 50% of its posterior probability on σ This compares with 25% for σ2 1.2 in the Jeffreys/Bca analysis of Figure 1. Changing ν 0 = 0.01 to in (3.7) substantially increases the posterior concentration of σ0 2 near 0. Inverse gamma priors are especially convenient for Gibbs sampling implementation, as in (3.9), but should not be assigned casually. The mle ˆσ 2 = (y i ȳ) 2 /n in (2.6) ignores the constraint σ 2 1, the actual mle being max(1, ˆσ 2 ). This suggests that for the dashed curve in Figure 3 we could erase the portion below zero and replace it with a probability atom of size at 0. More accurately, we could shift the Jeffreys/Bca density of Figure 1 one unit left, replacing the part below zero with an atom of size at 0. In a genuine random effects application, we might well allocate some prior probability to σ0 2 = 0 (as is not done in (3.7)). This makes for more difficult Bayesian analysis. The Bca approach of the previous remark is an attractive option here. Priors that are convenient for MCMC or Gibbs sampling applications play no special role in the bootstrap/bayes approach of this paper. Convenience priors can be risky, in my opinion, and should be avoided in favor of frequentist methods in the absence of genuine Bayesian prior information. References Chakraborty, A., Gelfand, A. E., Wilson, A. M., Latimer, A. M. and John A. Silander, J. (2010). Modeling large scale species abundance with latent spatial processes. Ann. Appl. Statist. 4: , doi: /10-aoas335. Efron, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans, CBMS-NSF Regional Conference Series in Applied Mathematics 38. Philadelphia, Pa.: Society for Industrial and Applied Mathematics (SIAM). Efron, B. (1987). Better bootstrap confidence intervals. J. Amer. Statist. Assoc. 82: , with comments and a rejoinder by the author. Efron, B. (1993). Bayes and likelihood calculations from confidence intervals. Biometrika 80: Fraser, D. A. S., Reid, N., Marras, E. and Yi, G. Y. (2010). Default priors for bayesian and frequentist inference. J. Roy. Statist. Soc. Ser. B 72: , doi: /j x. Lange, K. (2010). Numerical Analysis for Statisticians. Statistics and Computing. New York: Springer, 2nd ed. 9

10 Li, B., Nychka, D. W. and Ammann, C. M. (2010). The value of multiproxy reconstruction of past climate. Journal of the American Statistical Association 105: , doi: /jasa ap Newton, M. A. and Raftery, A. E. (1994). Approximate Bayesian inference with the weighted likelihood bootstrap. J. Roy. Statist. Soc. Ser. B 56: 3 48, with discussion and a reply by the authors. Ramirez-Cobo, P., Lillo, R. E., Wilson, S. and Wiper, M. P. (2010). Bayesian inference for double Pareto lognormal queues. Ann. Appl. Statist. 4: , doi: /10-aoas336. Rubin, D. B. (1981). The Bayesian bootstrap. Ann. Statist. 9: Spiegelhalter, D. J., Best, N. G., Gilks, W. R. and Inskip, H. (1996). Hepatitis B: A case study im MCMC methods. In Gilks, W. R., Richardson, S. and Spiegelhalter, D. J. (eds), Markov chain Monte Carlo in practice, Interdisciplinary Statistics. London: Chapman & Hall, Welch, B. L. and Peers, H. W. (1963). On formulae for confidence points based on integrals of weighted likelihoods. J. Roy. Statist. Soc. Ser. B 25: