Geweke, J. (1989). Bayesian inference in econometric models using Monte Carlo integration.

Transcription

1 Geweke, J (1989) Bayesian inferene in eonometri models using Monte Carlo integration Eonometria, 57, 1317{1339 Geyer, C J (1992) On the onvergene of Monte Carlo maximum likelihood alulations Tehnial Report 571, Shool of Statistis, University of Minnesota Geyer, C J and Thompson, E A (1992) Constrained Monte Carlo maximum likelihood for dependent data (with disussion) J Roy Statist So Ser B, 54, 657{700 Gilks, W R, Clayton, D G, Spiegelhalter, D J, Best, N G, MNeil, A J, Sharples, L D and Kirby, A J (1993) Modeling omplexity: Appliations of Gibbs sampling in mediine J Roy Statist So Ser B, 55, to appear Hastings, W K (1970) Monte Carlo sampling methods using Markov hains and their appliations Biometrika, 57, 97{109 Hill, B M (1965) Inferene about variane omponents in the one-way model J Amer Statist Asso, 60, 806{825 Liu, J, Wong, W and Kong, A (1991) Correlation struture and onvergene rate of the Gibbs sampler with various sans Tehnial Report 304, Department of Statistis, University of Chiago Metropolis, N, Rosenbluth, A, Rosenbluth, M, Teller, A and Teller, E (1953) Equations of state alulations by fast omputing mahines J Chemial Physis, 21, 1087{ 1091 Nummelin, E (1978) A splitting tehnique for Harris reurrent markov hains Wahrsheinlihkeitstheorie verw Gebiete, 43, 309{318 Nummelin, E (1984) General Irreduible Markov Chains and Non-Negative Operators Cambridge: Cambridge University Press Potts, R B (1952) Some generalized order-disorder transformations Pro Camb Phil So, 48, 106{109 Ripley, B D (1987) Stohasti Simulation New York, NY: Wiley Smith, A F M and Roberts, G O (1993) Bayesian omputation via the Gibbs sampler and related Markov hain Monte Carlo methods J Roy Statist So Ser B, 55, to appear Swendsen, R H and Wang, J-S (1987) simulations Phys Rev Lett, 58, 86{88 Nonuniversal ritial dynamis in Monte Carlo Tanner, M A and Wong, W H (1987) The alulation of posterior distributions by data augmentation (with disussion) J Amer Statist Asso, 82, 528{550 Tierney, L (1991a) Exploring posterior distributions using Markov hains In Computer Siene and Statistis: 23rd Symposium on the Interfae to appear Tierney, L (1991b) Markov hains for exploring posterior distributions Tehnial Report 560, Shool of Statistis, University of Minnesota Yu, B (1992) Density estimation in the L 1 norm for depdendent data with appliations to the Gibbs sampler Ann Statist,??, to appear 22

2 Proof of Proposition 3 Let A 2 E and x 2 E be arbitrary, and set B = A? fxg Sine has no atoms, s(x)(a) = s(x)(b) A B q(x; y)(x; y)(dy): q(x; y)(x; y)(dy) This yields the required result 2 Proof of Proposition 4 Using the simplied onditional regeneration probability (45), the equilibrium regeneration probability is (x)p(x; y)r(x; y)(dx)(dy) = (x)f(y) min ; 1 min w(x) ; 1 (dx)(dy) Referenes = = = min ff(x)(x)g (dx) min min min f(x); 1 (x) (dx) 1; 1 2 (x) f(x)(dx) f(x) f(y); 1 (y) (dy) Athreya, K B and Ney, P (1978) A new approah to the limit theory of reurrent Markov hains Trans Amer Math So, 245, 493{501 Besag, J and Green, P J (1993) Spatial statistis and Bayesian omputation J Roy Statist So Ser B, 55, to appear Box, G E P and Tiao, G C (1973) Bayesian Inferene in Statistial Analysis Reading, MA: Addison-Wesley Bratley, P, Fox, B L and Shrage, L E (1987) A Guide to Simulation (seond ed) New York, NY: Springer Gaver, D P and O'Muirheartaigh, I G (1987) Robust empirial Bayes analysis of event rates Tehnometris, 29, 1{15 Gelfand, A E, Hills, S E, Raine-Poon, A and Smith, A F M (1990) Illustration of Bayesian inferene in normal data models using Gibbs sampling J Amer Statist Asso, 85, 972{985 Gelfand, A E and Smith, A F M (1990) Sampling based approahes to alulating marginal densities J Amer Statist Asso, 85, 398{409 Gelman, A and Rubin, D B (1992) Inferene from iterative simulation using multiple sequenes Statistial Siene,??, to appear

3 and j1? g(y)j(dy) = j1? (s)g(y)? (1? (s))g(y)j(dy) j1? (s)g(y)j(dy) + 1? (s) = 2(1? (s)) Combining these results produes k (dx)p (x; dy)? (dx)(dy) k 2(1 + (s))(1? (s)) = 2(1? f(s)g 2 ) as laimed 2 Proof of Theorem 2 Let P (x; dy) denote the Metropolis kernel, and let A n+1 be the event that the andidate for X n+1 is aepted The onditional probability of A n+1, given X n = x and X n+1 = y, is given by P (A n+1 jx n = x; X n+1 = y) = q(x; y)(x; y)(dy) : P (x; dy) The onditional probability that S n = 1, given X n = x and X n+1 = y, is therefore P (S n = 1jX n = x; X n+1 = y) = P (fs n = 1g \ A n+1 jx n = x; X n+1 = y) = P fs n = 1jX n = x; X n+1 = y; A n+1 gp (A n+1 jx n = x; X n+1 = y) q(x; y)(x; y)(dy) = r A (x; y) P (x; dy) s 0 (x) 0 (dy) q(x; y)(x; y)(dy) = q(x; y)(x; y)(dy) P (x; dy) = s0 (x) 0 (dy) P (x; dy) ; whih is the onditional regeneration probability (32) used in the onstrution of Theorem 1 2 Proof of Theorem 3 It is suient to show that min w(x) ; 1 min w(x) ; 1 min ; 1 for all x and y and for any > 0 To see this, note that if =w(x) 1, then min w(x) ; 1 min ; 1 = w(x) min ; 1 = min w(x) ; w(x) whereas if =w(x) 1, then min w(x) ; 1 min ; 1 = min min w(x) ; 1 ; ; 1 min w(x) ;

4 imply that there is a problem, but do suggest that the dependene in the sampler is worth examining more losely Of ourse, as with i:i:d: sampling in general, there is no guarantee based on only a nite run of a sampler that a longer run would not produe several very long tours that might orrespond to explorations of additional modes of the posterior distribution Construting a good density for use in independene steps an help to redue this problem, but it annot eliminate it In priniple, the methods outlined in this paper an be used to split samplers whenever the single step transition density is available; this is the ase for most samplers proposed for exploring posterior distributions The resulting splits will not always be satisfatory there are many reasonable samplers for whih the basi mixing rate is too slow to provide reasonable splits based on a single transition A hybrid algorithm that inorporates independene steps may help to aelerate mixing; the regenerative analysis an be used to asses whether this attempt at aeleration has sueeded Suess depends on the quality of the andidate generation density Good methods for hoosing these densities are available for many problems, but more work is needed in developing methods for deriving satisfatory densities for high dimensional problems Adaptive methods for produing these densities seem partiularly worthy of further exploration 7 Proofs Proof of Theorem 1 The onstrution of (X n ; S n?1 ) is given in Nummelin (1984, pages 61-62) Corollary 42 of Nummelin shows that the reurrene of X n implies that the renewal sequene T i is reurrent, ie that all regeneration times are nite Convergene of the observed regeneration rate follows from Theorem 3 of Tierney (1991b), and the expression for the mean time between regenerations is given on page 76 of Nummelin 2 Proof of Proposition 1 For the yle kernel, (P 1 P 2 )(x; dy) = s(x) P 1 (x; du)p 2 (u; dy) (du)p 2 (u; dy) = s(x)(p 2 )(dy); thus (s; P 2 ) is a split of P 1 P 2 For the mixture kernel, P 1 (x; dy) + (1? )P 2 (x; dy) P 1 (x; dy) s(x)(dy): So (s; ) is a split of P 1 + (1? )P 2 as long as > 0 2 Proof of Proposition 2 Sine is invariant for P, integrating the split inequality (31) with respet to (dx) shows that (dy) (s)(dy) As a result, is absolutely ontinuous with respet to with a density g suh that 0 (s)g(x) 1 for all x Therefore k (dx)p (x; dy)? (dx)(dy) k k (dx)p (x; dy)? (dx)s(x)(dy) k + k (dx)s(x)(dy)? (dx)(dy) k = 1? (s) + j1? s(x)g(y)j(dx)(dy): Now j1? s(x)g(y)j(dx)(dy) j1? s(x)j(dx) + = 1? (s) + (s) s(x)j1? g(y)j(dx)(dy) j1? g(y)j(dy) 19

5 bond is plaed between the verties, and b ij = 0 A set of bonds partitions the vertex set V into onneted omponents Conditional on the bonds b ij, the x i 's are the same within omponents, and the omponent olors are seleted independently and uniformly from the available L olors The joint distribution of (x; b) is proportional to e?(jej?p b ij) (1? e? )P bij on the set of (x; b) values suh that b ij = 0 whenever x i 6= x j, and is zero elsewhere; here jej is the number of edges in the graph, and sums are over edges The algorithm is a two-oordinate Gibbs sampler that alternates between seleting bonds and olors from these onditional distributions It is possible to split this Gibbs sampler along the lines of Setion 421 A natural hoie for the distinguished state ~x is the state where all verties have the same olor; the sampler then generates a new set of bonds as i:i:d: Bernoulli random variables, and then selets a new set of olors for the resulting omponents In omputing the inmum needed to nd the splitting probability s() in Equation 48, taking D = E, it is easy to see that any dierene in olor produes an inmum of zero So s() will be just the indiator of whether all verties have the same olor or not, and the split will our eah time the sampler returns to a onguration in whih all states have the same olor An alternating sampler using independene steps an also be onstruted using Gibbs steps started at a uniform olor onguration as the andidate generator Sine the onditional distribution of the olors given bonds is just uniform on the L (b) possible omponent olorings, where (b) is the number of omponents, the weight funtion for this independene step is proportional to L (b) Both the split of the Gibbs sampler itself and the split of the alternating hain should work reasonably well if is not too small and the graph not too large As a simple illustration, we used an m m grid with m = 32 and two olors, L = 2 The parameter was hosen to make the bond plaement probability (1? e? ) equal to 0:8 (This orresponds to a temperature well below the freezing point of the innite Ising lattie; a more elaborate andidate generation density would be needed for lower values of or higher values of m) Both samplers were started with all verties the same olor For the pure Gibbs sampler with a split on returns to all one olor, using a run of 5000 gave 197 regenerations, or 196 omplete tours; this gives an estimated regeneration rate of 0039 and an estimated mean tour length of 256 Using a preliminary Gibbs sampler run of length 100, the optimal hoie of for the independene split was found to be = e 4, with an estimated regeneration rate for the independene steps of 0166 Based on a run of 5000 of the alternating hain, the estimated rate was 0073, orresponding to a mean tour length of 137; there were 358 regenerations, or 357 omplete tours 6 Conlusions Identifying regeneration points in a Markov Chain sampler eliminates initialization issues and allows variane estimates to be omputed based on i:i:d: observations In addition, it allows Markov hain sampling to take advantage of a parallel omputing environment without the problems reated by many short Markov hain runs when regeneration points are not available Regeneration rates and regeneration time distributions are also useful as a diagnosti of sampler performane High regeneration rates and regeneration patterns that are lose to uniform suggest that the sampler is working well Low rates or non-uniform regeneration patterns do not neessarily 18

6 Observation Smoothed Regeneration Rate Observation Smoothed Regeneration Rate (a) = 1 (b) = 3 Observation Smoothed Regeneration Rate Observation Smoothed Regeneration Rate () = 5 (d) = 7 Figure 3: Smoothed regeneration rate plots for the bivariate normal mixture example based on runs of length 5000 from an alternating Gibbs/independene sampler The Plot bakgrounds show jitter plots of the observed regeneration times 17

7 Tours Regen Rate X 1 X 2 SE B (X 1 ) SE R (X 1 ) SE B (X 2 ) SE R (X 2 ) Table 1: Summary statistis for Gibbs/independene samplers for mixtures of bivariate normal densities Standard errors were omputed using bath means (SE B ) and the regenerative method (SE R ) probability was 05 For suiently large the density is bimodal with the modes displaed along the diagonal; the Gibbs sampler should therefore have some diulty in moving from one mode to the other An alternating Gibbs/independene hybrid sampler was onstruted with a single Gibbs step from the origin as the andidate generation density for the independene steps This example is intended to model a situation where preliminary exploration has revealed one mode, the mode at the origin, but a seond, equally important, mode is in fat present at (; ) Runs of length 5000 were performed for equal to 1, 3, 5, and 7 Table 1 shows the number of omplete tours, the estimated equilibrium regeneration rate, and the sample means of the two oordinates Two estimated standard errors are given for eah sample mean, a bath mean estimate based on bathes of size 50, and a regenerative estimate based on (22) The means of the marginal distributions of the two oordinates under are equal to =2 Figure 3 shows the smoothed regeneration rate plots as well as jitter plots of the observed regeneration times for the four values of As expeted, the performane of the sampler deteriorates as inreases At = 5 there are several large gaps in the regeneration times, orresponding to periods when the sampler is in the mode at (; ) For = 7 the sampler starts in the mode at the origin, moves to the seond mode after approximately 800 observations, and returns to the mode at the origin after a total of approximately 3800 observations The estimated standard errors are also onsiderably larger for = 5 and = 7 The regenerative simulation analysis learly reveals that the sampler is not behaving well for = 5 and = 7 In a real example, further exploration should reveal the seond mode Inorporating this mode into a andidate generation density for independene steps in a hybrid sampler should produe a sampler with muh better properties 54 Splitting and the Swendsen{Wang Algorithm Even though our work is primarily motivated by appliations in Bayesian and maximum likelihood omputations, the ideas an also be used in other Markov hain Monte Carlo problems As an illustration, we show how they an be applied to the Swendsen{Wang algorithm Swendsen and Wang (1987) propose a method for sampling the Potts (1952) model (Besag and Green 1993), the multiolor generalization of the Ising model This model assumes the verties V = f1; ; Mg of a graph (V; E) are eah given one of L olors, x i The distribution of the olors is assumed proportional to expf?(x)g, where (x) is the number of edges (i; j) 2 E for whih x i 6= x j, and is a non-negative onstant Given the olors x i, the algorithm adds auxiliary bond variables, b ij No bonds are plaed between verties with dierent olors If x i = x j and (i; j) is an edge in the graph, then with probability 1? exp(?) a bond is plaed between verties i and j, and b ij = 1 Otherwise, no 16

8 Observation Smoothed Regeneration Rate Observation Smoothed Regeneration Rate (a) Gibbs Step Candidate Dist (b) Approx Marginal Candidate Dist Figure 2: Smoothed regeneration rate plots for the variane omponents example with prior II for sampler runs of length 5000 from two alternating Gibbs/independene samplers The plot bakgrounds show jitter plots of the observed regeneration times These results suggest that the Gibbs sampler is working reasonably well But it may be possible to improve the sampler by using a better andidate generation density in the independene steps If we ould sample, 2, and 2 e from their joint marginal posterior distribution, then we would only need to draw i 's from their onditional distribution given the other parameters to obtain an i:i:d: sample from the posterior distribution This suggests that we may be able to improve our andidate generation density by mathing the distribution used to generate, 2, and 2 e more losely to their marginal distribution As a very simple attempt, we tried generating them independently with 2 and 2 e hosen from log normal distributions and from a t distribution with ve degrees of freedom The parameters for these distributions were estimated from the preliminary run of length 100 of the Gibbs sampler Using Equation (46) and the preliminary sample, the optimal was approximately = 1 with an estimated regeneration rate for the independene steps of 0632 Using a sample of 5000 from the alternating sampler, the estimated regeneration rate was 0293, giving an estimated expeted tour length of 341, and the number of observed regenerations was 1481 The smoothed regeneration rate plot and a jitter plot of the regeneration times are show in Figure 2(b) This example illustrates several points The regenerative simulation analysis an give an indiation of problems in a sampler, and it an also be used to indiate when there is room for improvement in the sampler For many hierarhial models a Gibbs sampler an be improved by ombining it with independene steps that sample the hyperparameters from an approximation to their joint marginal posterior distribution In this example a simple math of features of the one dimensional margins of the three parameters was suient to produe a signiant improvement In other ases it might be useful to use kernel density estimates of the marginal distribution, or other approahes that take advantage of features in the joint distribution 53 An Artiial Example As an artiial example that illustrates the diagnosti performane of regenerative simulation, we onsidered a distribution that is a mixture of a bivariate standard normal distribution and a bivariate standard normal distribution shifted to have its enter at the point (; ) The mixing 15

9 52 A Normal Variane Components Model Gelfand et al (1990, Setion 4) onsider a variane omponents model where Y ij = i + e ij ; i = 1; ; K; j = 1; ; J; with e ij, given the i and e, 2 having independent N(0; e) 2 distributions, and the i, given and the varianes, being independent N(; 2) random variables The parameters, 2 and 2 e are independent N( 0 ; 2), IG(a 0 1; b 1 ), and IG(a 2 ; b 2 ) random variables, respetively Here IG(a; b) denotes an inverse gamma distribution Gelfand et al use a Gibbs sampler based on the onditional distributions jy; 2 ; ; e 2 = IG a K; b e 2 jy; ; ; 2 = IG a KJ; b jy; ; 2 ; 2 e = N P i 2 + K2 e i jy; ; 2 ; 2 e = N J2 Y i + 2 e J e ; ; X (i? ) 2 X X (Yij? i ) e 2 + K2 e 2 2 e J e with the i onditionally independent given Y,, 2, and 2 e It would also be possible to ombine the i and, sine their onditional distribution, given the data and the other parameters, is a multivariate normal distribution This should result in a more eient Gibbs sampler, but for omparison we used the deomposition of Gelfand et al (1990) The data set used by Gelfand et al is an artiial data set introdued by Box and Tiao (1973, p 247) This data set was generated from the model with = 5, 2 = 4 and 2 e = 16 The prior distribution parameters used for and e 2 were 0 = 0, 0 = 10 12, and a 2 = b 2 = 0 For 2 Gelfand et al onsider two ases, I: a 1 = b 1 = 0 and II: a 1 = 1=2 with several values of b 1 ; we onsider only the ase ase b 1 = 1 Both these prior distributions are improper beause of the hoie of prior distribution for e 2 It is possible to use the ideas of Setion 421 to split the Gibbs sampler for this problem But the algebra is rather messy, so we will only apply the hybrid approah of Setion 422 We use an alternating Gibbs/independene hain with the Gibbs step generating rst new values for 2 and e, 2 then, and nally new i 's The independene andidate density is a single Gibbs step started at the average of 100 preliminary runs of the pure Gibbs sampler For prior I, the optimal hoie of based on a preliminary sample of 100 and Equation (46) was approximately = e?7, with an estimated regeneration rate of 0020 But in a run of 5000 with the alternating sampler, a split using this value of produed only two regenerations and an estimated regeneration rate of This in itself does not imply that there is a problem, but further examination shows that the posterior distribution for this prior is in fat improper (Hill 1965) Any irreduible Markov hain sampler for this posterior will be either null reurrent or dissipative, so the lak of frequent regeneration is not surprising For prior II, a preliminary Gibbs sample of 100 gave an estimated regeneration rate for independene steps of 0152 at = e?2:7 Using a run of 5000 of the alternating sampler, the estimated regeneration rate was 0066, orresponding to an estimated mean tour length of 1515 A total of 332 regenerations, or 331 omplete tours, were observed Figure 2(a) shows the smoothed regeneration rate plot and a jitter plot of the regeneration times 14!! ;

10 Observation Smoothed Regeneration Rate Observation Smoothed Regeneration Rate (a) Gibbs Sampler (b) Alternating Gibbs/Independene Sampler Figure 1: Smoothed regeneration rate plots for the pump data example for sampler runs of length 5000 from a pure Gibbs sampler and an alternating Gibbs/independene sampler The plot bakgrounds show jitter plots of the observed regeneration times started with ~ = 6:7, the approximate posterior mean of, and generating rst and then the i The weight funtion for this independene kernel is w(x) = e ~ (x) Q (ti + (x)) s i+ : This an be standardized to be near one by dividing by its value at ~ = 2:35, the estimated posterior mean of based on a preliminary sample Using a preliminary Gibbs sample of 100 with Equation (46) and the standardized weights produed an estimated optimal value of = 1:1 for the onstant in Theorem 3; the estimated regeneration rate for the independene steps at this value of was 078 The regeneration rate for a hain that alternates Gibbs steps and independene steps should thus be approximately 0:78=2 = 0:39 A run of 5000 from this alternating hain, onsisting of 2500 Gibbs yles and 2500 independene steps, produed an estimated regeneration rate of ^r = 0:42, or an estimated tour length of 1=0:42 = 2:38 A total of 2070 regenerations, or 2069 omplete tours, were observed Figure 1(b) shows a smoothed regeneration rate plot for this Gibbs/independene sampler along with a jitter plot of the observed regeneration times Both the split Gibbs sampler and the alternating sampler have very high regeneration rates and uniform regeneration patterns that onrm that the Gibbs sampler works very well in this problem Sine the independene steps used here only use a single Gibbs step to generate their andidates, they do not signiantly aelerate onvergene of the algorithm; but additional aeleration does not seem neessary in this ase In omputational ost, one full Gibbs/independene yle is lose to one Gibbs step with a split generation; it ould therefore be argued that a split Gibbs sampler run of length 5000 should be ompared to an alternating run of length On the other hand, neither the split generation nor the independene steps need to be arried out on every yle; using them, say, every 5-th yle would redue the ost of heking for regenerations and inrease the expeted tour lengths from approximately 25 to approximately

11 reorded The data were originally analyzed in Gaver and O'Muirheartaigh (1987) and are also reprodued in Tierney (1991b, Table 1) Conditional on a hyperparameter, the individual pump failure rates are assumed to be independent random variables with a gamma distribution G(; ) with density proportional to x?1 e?x The hyperparameter has a gamma distribution G(; ) with = 0:01 and = 1 For the gamma exponent of the rate distribution Gelfand and Smith use the method of moments estimator, = 1:802 For the resulting posterior distribution, given, the i are independent G(+s i ; t i +) random variables, and, given 1 ; ; 10, the distribution of is G( + P 10; i + ) For onstruting a split of the Gibbs sampler, suppose we rst generate, then 1 ; ; 10 Then new values of and i only depend on the previous ones through = P i, and the ratio of the Gibbs sampler transition densities started with two dierent values of only depends on the next state through its value of : p(x; y) (x) p(~x; y) = expf((~x)? (x))(y)g: (~x) + In this equation x and y represent dierent ombinations of and 's To apply the approah outlined in Setion 421, we need to hoose a distinguished value ~x, or its orresponding value of, ~ = (~x), and a set D, whih only needs to depend on, and ompute s() = P f 2 Dj~g inf expf(~? )g 2D ~ + For an interval D = [d 1 ; d 2 ], the minimization produes s() = P fd 1 d 2 j~g expf(~? )d()g; ~ + where d() = ( d1 if < ~ d 2 if : ~ The orresponding onditional probability of a regeneration, given X n = x and X n+1 = y, is r(x; y) = 8 >< >: expf( ~? (x))(d 1? (y))g if (x) < ~ and d 1 d 2 expf(~? (x))(d 2? (y))g if (x) ~ and d 1 d 2 0 otherwise A reasonable approah to hoosing the three parameters ~, d 1 and d 2 of this split is to set ~ equal 67, the approximate posterior mean of based on a preliminary sample, and to hoose d i of the form ~ k~s, where ~ = 2:35 and ~s = 0:69 are the approximate posterior mean and standard deviation of, again based on a preliminary sample By graphing estimated regeneration rates, based on the preliminary sample, for a range of values of k, the optimal value of k was found to be 11; this orresponds to hoosing d 1 = 1:6 and d 2 = 3:1 These hoies are essentially idential to ones obtained by a global optimization over all three parameters Using this split on a Gibbs sampler run of length 5000 produed an estimated regeneration rate of ^r = 0:39, or an estimated expeted tour length of 1=0:39 = 2:56 The number of observed regenerations was 1968, orresponding to 1967 omplete tours Figure 1(a) shows a smoothed regeneration rate plot as well as a jitter plot of the observed regeneration times for this sampler run As a seond approah, we used an alternating sampler in whih a Gibbs yle was followed by an independene step The independene step andidates were generated by a single Gibbs yle 12

12 of two-omponents, ie for d = 2, a split of 1 or 2 is thus automatially a split of the Gibbs sampler One again, starting the hain with a regeneration is easy, sine sampling from (dy) orresponds to taking one Gibbs sampler yle starting at ~x 422 Hybrid Samplers Tierney (1991b) desribes several strategies for forming hybrid samplers by ombining several more basi samplers In partiular, it is often useful to ombine Gibbs samplers with steps from an independene Metropolis hain If a Gibbs sampler is produing highly orrelated observations, then the use of independene steps with a well hosen andidate generation density an help to redue these orrelations Whether the use of independene steps helps or not depends on the quality of the andidate generation density Examining the regeneration pattern produed by a split of the independene steps an provide a useful indiation of the suess of these steps On the other hand, if a Gibbs sampler does seem to be performing well, using periodi independene steps an help to onrm the performane of the sampler and to provide regeneration points for a regenerative analysis The problem of hoosing a good density for the independene steps is similar to the problem of hoosing a good importane sampling density or a good initial density for a Gibbs sampler Split t distributions (Geweke 1989) and the over-dispersed distributions of Gelman and Rubin (1992) may be useful We may also use the normal approximation to the posterior distribution, if one is available, or use the density orresponding to one Gibbs yle from some reasonable initial point, ~x If it is possible to sample from the prior distribution and the likelihood is bounded, then hoosing an independene kernel that samples from the prior distribution with positive probability produes an independene kernel with bounded weight funtion w The omputations required for performing alternate Gibbs yles and independene steps are roughly equivalent to the omputations required for identifying a split using the approah outlined above But there is no need to arry out the areful analysis needed to determine the split of a Gibbs sampler; we only have to do a limited amount of experimenting with preliminary samples or approximate posterior distributions to identify a good hoie of the onstant in Theorem 3 It is of ourse not neessary to alternate independene and Gibbs steps Instead, a sampler ould use an independene step after every ve or ten Gibbs steps This inreases the lengths of the average tours, but should still result in a reasonable number of tours in most simulations, provided the Gibbs sampler is in fat working reasonably well The independene steps an also be inorporated as a mixture rather than as a yle When the independene step andidate distribution f is less spread out than the target distribution, the independene steps do not ontribute signiantly to the mixing of the hain Instead, they an be viewed as simply heking whether the hain has returned to the region ontaining the bulk of the mass of f 5 Examples 51 A Hierarhial Poisson Model One of the examples presented by Gelfand and Smith (1990) is a hierarhial Poisson model Failures in ten pumps at a nulear power plant are assumed to our aording to independent Poisson proesses with eah pump having its own failure rate 1 ; ; 10 The pumps were observed for periods t i of varying lengths, and the numbers of observed failures s i for eah pump were 11

13 and let D = fy : jyj dg for some d > 0 Then for ~x = 0 q(x; y) s q (x) = inf y2d q(~x; y) = exp? 1 2 xt x? djxj A similar approah an be used for any random walk hain based on a spherially symmetri inrement distribution 42 Regeneration and the Gibbs Sampler Suppose the state spae E is a produt of d omponents, E = E 1 E d, an element of E is written as x = (x 1 ; ; x d ) with x i 2 E i, and (x) is a density with respet to a produt measure (dx) = (dx 1 ) (dx d ) Let i (x i jx 1 ; ; x i?1 ; x i+1 ; ; x d ) denote the onditional density of the i-th omponent given all the others The Gibbs sampler (Gelfand and Smith 1990) starting with X n = x generates X n+1 by generating X n+1;1 = y 1 from 1 (y 1 jx 2 ; ; x d ) X n+1;2 = y 2 from 2 (y 2 jy 1 ; x 3 ; ; x d ) X n+1;d = y d from d (y d jy 1 ; ; y d?1 ) Even though examples of very strong dependene are available, experiene suggests that for many problems the dependene in the Gibbs sampler sequene drops o very quikly, often within 10 to 20 yles As a result, it seems reasonable that a regeneration sheme with mean tour lengths on the order of 10 to 20 should be available in these ases For some problems it is possible to identify a split of the Gibbs sampler itself For others, it is easier to form a hybrid algorithm of the Gibbs sampler and independene Metropolis steps, and to use the approah of the preeding setion to identify regenerations that our on the independene hain steps 421 Splitting a Gibbs Sampler The transition kernel of the Gibbs sampler has transition density p(x; y) = 1 (y 1 jx 2 ; ; x d ) 2 (y 2 jy 1 ; x 3 ; ; x d ) d (y d jy 1 ; ; y d?1 ): (4:7) In some ases it may be possible to nd a split of this density by diret examination For others, it may be possible to follow the strategy used for splitting the standard Metropolis transition kernel by hoosing a distinguished point ~x and a set D 2 E, taking (dy) to have density p(~x; y), and setting p(x; y) s(x) = inf y2d p(~x; y) : (4:8) In many problems the minimization required to ompute s(x) an take advantage of the exponential family struture often present in problems where a Gibbs sampler is used; this is the ase for one of the examples disussed below in Setion 5 The omputation of s(x) may also be simplied by a suitable hoie of the ordering of the omponents For example, the nal fator in (47) does not depend on x and therefore anels from the ratio in the denition of s(x) For the purposes of omputing a split, it is therefore useful to plae the most ompliated onditional distribution last in the update sequene In the ase 10 :

14 These expression an be used to estimate the regeneration rate or to selet a good hoie of based on either an approximation to or a preliminary sample To use this expression to estimate the regeneration rate, the normalizing onstant for must be available or must be estimated For an un-normalized density (x), the rst identity in Equation (46) beomes (s) = R (x)(dx) min f(x) (x) ; 1 (x)(dx) 2 : From this expression it is lear that it is not neessary to estimate the normalizing onstant to selet a minimizing value of If and f are N(0; A) and N(0; B) distributions, respetively, then the region of integration for (x) in the third identity in Equation (46) is (!) fx : w(x) < g = X : Y < log jaj1=2 jbj 1=2 with Y = 1 2 X T B?1? A?1 X: The distribution of the quadrati form Y an be represented as a linear ombination of squares of independent standard normals This an be used for the approximate omputation of if and f are approximately normal 412 Standard Metropolis Chains The original version of the Metropolis algorithm of Metropolis et al (1953) assumes a symmetri andidate generation kernel, q(x; y) = q(y; x) In this ase, equation (44) holds with h a onstant To apply Theorem 3, a split (s q ; q ) of q has to be found One useful approah to nding suh a split is to selet as the distribution q the andidate generation density for some distinguished point ~x, so q (dy) = q(~x; y)(dy), and then determine s q (x) as q(x; y) s q (x) = inf y2e q(~x; y) : Unfortunately this sometimes produes a funtion that is identially zero and thus not aeptable A minor modiation is to hoose a onvenient set D 2 E, usually a ompat set, and to take and q (dy) = q(~x; R y)1 D(y)(dy) q(~x; u)(du) D q(x; y) s q (x) = inf y2d q(~x; y) : It is possible to start the hain with a regeneration by rejetion sampling the initial state X 0 from a density proportional to q(~x; y)1 D (y) As an example, onsider a random walk hain with normal inrements, so q(x; y) / exp? 1 2 (y? x)t (y? x) ; 9

15 411 Independene Chains In an independene Metropolis hain (Tierney 1991b, Setion 23) andidates are generated from a xed density f, regardless of the urrent state of the hain; thus q(x; y) = f(y) Equation (44) holds for h = f To apply Theorem 3, hoose s q = 1 and q (dy) = f(y)(dy) This independene hain will be used to form hybrid hains in Setion 422 For an independene hain with the split of Theorems 2 and 3, the distribution 0 has density proportional to (y) f(y) min ; 1 = f(y) min f(y) ; 1 : This an be sampled by rejetion sampling to obtain an initial value X 0 for the hain orresponding to a regeneration The onditional probability (42) of a regeneration at step n, given X n = x, X n+1 = y, and no rejetion, simplies to ( min ; 1 r A (x; y) = = 8 >< >: min ) min ( w(x) ; 1 ) ( w(x) ; 1 ) (45) minfw(x); g if w(x) > and > maxfw(x); g if w(x) < and < 1 otherwise Thus a regeneration is ertain to have ourred if w(x) and are on opposite sides of This suggests that a good hoie for will typially be in the enter of the distribution of the weights w(x) under If the andidates for an independene hain are produed by a rejetion algorithm with andidate generation density proportional to g (Tierney 1991b, Setion 23) and if = 1, then this regeneration probability beomes r A (x; y) = 8 >< >: ( 1 ) if x 2 C or y 2 C g(x) min (x) ; g(y) otherwise, (y) where C = fx : (x) g(x)g is the set where g dominates Thus a regeneration ours whenever X n is at a point where g is an envelope for Simple expressions for the equilibrium regeneration rate of an independene hain are also available: Proposition 4 For an independene hain with andidate generation density f and the split given above, the equilibrium regeneration rate is f(x) (s) = min (x) ; 1 2 (x)(dx) = min 1; 1 (x) 2 f(x)(dx) (46) f(x) = fx:w(x)<g 1 fx:w(x)g (x)(dx) + f(x)(dx)! 2 8

16 positive probability when the andidate step is aepted When estimating the regeneration rate of a Metropolis hain, it is natural to use ^r = 1 t Xt?1 i=0 r A (X i ; X i+1 ) i+1 (4:3) where i+1 is the indiator funtion for the andidate step for X i+1 having been aepted This leaves the question of nding a split that satises (41) This an be done easily by nding a split (s q ; q ) for the kernel Q, provided there exists a positive funtion h suh that h(x)q(x; y) = h(y)q(y; x) (4:4) for all x and y This ondition is formally similar to a reversibility ondition, but h is not required to be a probability density This ondition is satised by independene hains, where q(x; y) = f(y) for some probability density f, by taking h = f It is also satised by the original Metropolis algorithm, where it is assumed that q(x; y) = q(y; x), by taking h to be a onstant If q satises (44) then the aeptane probability (x; y) simplies to where w(x) = (x)=h(x) (x; y) = min w(x) ; 1 Theorem 3 Suppose a Metropolis hain satises (44), and let (s q ; q ) be a split for Q For any > 0, set s 0 (x) = s q (x) min w(x) ; 1 and 0 (dy) = q (dy) min ; 1 : Then (41) holds The 0 given in Theorem 3 is not a probability measure, but this does not matter in (41) as 0 (E) an be absorbed into s 0 The regeneration rate depends on the hoie of the onstant As mentioned in Setion 32, a good hoie for an be determined by maximizing an estimate of the regeneration rate obtained either using an approximation to, for example a normal approximation, or using a preliminary sample It is not neessary to know the normalizing onstant for to determine a good hoie of ; multiplying by a onstant is equivalent to dividing by the same onstant The reason for only permitting regenerations when the andidate step is aepted is twofold On the one hand, this provides (through Theorems 2 and 3) a onvenient formula for simulating the regeneration On the other hand, very little is lost by having no regenerations at the times of rejetion This is beause the dependene between X n and X n+1 is so great in the ase of rejetion that only a very small probability of regeneration ould be assigned anyway This is partiularly so if the Metropolis kernel is split with a that has no atoms, whih must be the ase if has no atoms: Proposition 3 Let (s; ) be a split for the Metropolis hain and assume that has no atoms Then has a density with respet to, denoted by (y), suh that i e (s; ) is a split for q(x; y)(x; y)(dy) q(x; y)(x; y) s(x)(y); 7

17 4 Splitting Some Markov Chain Samplers Given a Markov hain sampler, three general approahes to inorporating regeneration are available The rst approah is to attempt to nd a split for the sampler itself This is possible for ertain speial samplers If it is not possible to nd a split for the original sampler, the seond approah is to form a hybrid sampler that inorporates steps from a sampler for whih a split is available If the resulting hain does not regenerate very rapidly, then the third approah is to again hoose a hybrid strategy, but one speially designed to introdue more frequent regenerations As disussed in Tierney (1991b), the Metropolis algorithm is partiularly useful for onstruting hybrid algorithms with partiular properties 41 Splitting Metropolis Chains Suppose the distribution we wish to sample has a density, also denoted by, with respet to a measure, (dx) = (x)(dx) Hastings (1970) version of the Metropolis algorithm originally introdued by Metropolis et al (1953) generates the next step X n+1 in a Markov hain from the urrent state X n by rst generating a andidate step Y from a transition kernel Q(X n ; dy) = q(x n ; y)(dy) This andidate is aepted with probability (X n ; Y ), where (x; y) = min (y)q(y; x) (x)q(x; y) ; 1 and X n+1 is set equal to Y Otherwise, the andidate is rejeted and X n+1 is set equal to X n For reasons to be disussed in onnetion with Proposition 3 below, it is natural to split a Metropolis kernel by splitting the sub-probability transition density q(x; y)(x; y), ie by nding a pair (s 0 ; 0 ) suh that q(x; y)(x; y)(dy) s 0 (x) 0 (dy): (4:1) Sine the Metropolis kernel P satises P (x; dy) q(x; y)(x; y)(dy); this provides a split of the kernel P The splitting variables S n of Theorem 1 an be generated by allowing a split to our only when a andidate step is aepted: Theorem 2 Suppose the Metropolis hain satises (41), and suppose (X n+1 ; S n ) is generated as follows: (1) draw X n+1 onditionally on X n by taking andidate steps from q(x n ; y)(dy) and aepting or rejeting aording to (X n ; y); (2) if the andidate in step (1) is rejeted, then set S n = 0; otherwise, generate S n as a Bernoulli random variable with suess probability r A (X n ; X n+1 ), where r A (x; y) = s0 (x) 0 (dy) q(x; y)(x; y) : (4:2) Then the proess (X n+1 ; S n ) has the distribution given in Theorem 1 The suess probability r A (x; y) is the onditional probability of a regeneration, given X n = x and X n+1 = y, and given that the andidate is aepted In priniple, it is possible to generate the splitting variables S n diretly using the suess probability (32) But when 0 has atoms the expression for r(x; x) an be ompliated to derive due to the fat that X n = X n+1 an our with 6

18 A low regeneration rate by itself does not imply a highly dependent sequene If a sampler is seleting i:i:d: observations from, then (s; ) is a split for this i:i:d: Markov hain for any s with 0 s(x) 1 for all x and (s) > 0 Its tour lengths N i are geometri random variables with expeted value f(s)g?1 If (s; ) is a split for a dependent sampler, then the mean tour length is also E[N i ] = f(s)g?1, but the distribution is not geometri; typially it will have a heavier upper tail For an i:i:d: sequene the probability of a regeneration ourring at any partiular observation is a onstant As a result, the pattern of regenerations is uniform in the sense that the number of regenerations expeted in a partiular interval is proportional to the size of the interval By the renewal theorem, a similar result is true asymptotially for any renewal proess: If the proess is observed for t periods, then for any a and b with 0 a b 1 the proportion of the observed regenerations that fall in the interval [at; bt] onverges to (b? a)=e[n i ] = (b? a)(s) On the other hand, a Markov hain sampler with a heavy-tailed regeneration distribution that is not observed for a suiently long time period may have oasional long periods with no regenerations that make the overall pattern of the renewals appear non-uniform These observations suggest that a plot of the regeneration pattern and an estimate of the density of regenerations per observation, a smoothed loal regeneration rate, may be useful to assess the performane of a sampler If a sampler is working well, then the regeneration pattern and the smoothed regeneration rate plot should be lose to uniform Departures from uniformity indiate that regenerations are less likely to our when the sampler is in ertain parts of the state spae, and that it is taking onsiderable time, relative to the total observation time, to move from these regions into regions where regeneration is more likely to our The regeneration rate an be estimated by the proportion of observations that result in regenerations A smoothed regeneration rate plot an be produed as a histogram or a kernel density estimate of the observed regeneration times Estimates of the regeneration rate and smoothed regeneration rate plots an also be onstruted without diretly using the split indiators by making use of the onditional regeneration rates; this an be viewed as a slight variane redution by onditioning The regeneration rate an be estimated using a sequene of length t by the average onditional regeneration rate, ^r = 1 t Xt?1 i=0 or, if the normalizing onstant for is available, by ~r = 1 t + 1 r(x i ; X i+1 ) tx i=0 s(x i ): In the ase of Metropolis hains, one would typially use the estimate given by (43) below The smoothed regeneration rate plot an be onstruted by smoothing the r(x i ; X i+1 ) or the s(x i ) If a low regeneration rate or a non-uniform regeneration pattern are observed, then it is worth exploring whether a better split an be obtained Often this an be done as in Setion 5 below by identifying a reasonable lass of splits and then nding the best split in this family by maximizing the estimated regeneration rate or minimizing the mean tour length using preliminary samples or a normal approximation if one is available If no improved split is found, the dependene struture in the sampler should be examined more losely to see if there are any suggestions for ways of reduing dependene, perhaps by onstruting a hybrid sampler of the type desribed in Setion 422 5

19 and the mean time between regenerations is for i = 1; 2; E[N i ] = E[T i? T i?1 ] = 1 (s) The onstrution outlined above only depends on a split (s; ) through the produt s(x)(dy) Thus it is not neessary to determine the normalizing onstant needed to make into a probability measure It is suient to nd a nite, nonzero measure 0 and a funtion s 0 suh that s 0 (x) 0 (dy) P (x; dy) and (s 0 ) > 0; then the split (s; ) is given by (dy) = 0 (dy)= 0 (E) and s(x) = s 0 (x) 0 (E) To apply the result of this theorem eetively, we need to be able to nd a good split of a transition kernel P Some useful approahes for spei types of samplers are disussed in Setion 4; the remainder of this setion examines general issues Splits of a given transition kernel P need not exist If E is ountably generated, Nummelin (1984) shows that it is always possible to nd a split for the m-step transition kernel P m for some m 1 However, in Markov hain simulations P m is rarely available in losed form for any m > 1, so we onsider only the ase m = 1 If a split (s; ) of P does exist, it is not unique: For any s 0 with 0 s 0 (x) s(x) for all x and (s 0 ) > 0, the pair (s 0 ; ) is also a split of P But (s; ) is a better hoie, sine it produes a smaller expeted tour length In general, given several hoies for a split we usually prefer the ones that produe lower mean tour lengths, or higher regeneration rates We will elaborate on this point in the following subsetion If a split of a kernel is not available, then it may be possible to form a hybrid (Tierney 1991b, Setion 24) with another kernel that is easier to split If P 1 and P 2 are transition kernels with invariant distribution, then the yle hybrid kernel P 1 P 2 and the mixture hybrid kernel P 1 + (1? )P 2 for 0 1 are also transition kernels with invariant distribution The yle hybrid orresponds to alternately using P 1 and P 2 to generate a new state; for a mixture, at eah step kernel P 1 is used with probability and kernel P 2 with probability 1? If a split of one of the kernels in a hybrid is available, then a split of the hybrid hain is available: Proposition 1 Suppose P 1 and P 2 are transition kernels with invariant distribution and (s; ) is a split for P 1 Then (s; P 2 ) is a split for the yle kernel P 1 P 2, and (s; ) is a split for the mixture kernel P 1 + (1? )P 2 if 0 < 1 32 The Diagnosti Role of Regenerative Simulation The regeneration rate (s) or its reiproal, the mean tour length E[N i ], and the pattern of the regeneration times an provide useful diagnostis for the performane of a Monte Carlo sampler If E[N i ] is small or the regeneration rate is high, then this suggests that the dependene between the observations produed by the sampler is rather mild A more preise statement is given by the following proposition Proposition 2 Let k k= max A2E (A)?min A2E (A) denote the total variation norm of a signed measure, and suppose (s; ) is a split for a transition kernel P Then k (dx)p (x; dy)? (dx)(dy) k 2(1? f(s)g 2 ): Thus the regeneration rate of a split provides a bound on the departure of the sampler from i:i:d: sampling 4

20 3 Splitting Markov Chains Let fx n : n = 0; 1; g be an irreduible Markov hain on a state spae (E; E) with transition kernel P = P (x; dy) and invariant distribution The sigma algebra E is assumed to be ountably generated These assumptions imply that X n is positive reurrent (see, for example, Tierney 1991b, Theorem 1) Assume in addition that X n is Harris reurrent; this is satised by most Markov hain samplers (Tierney 1991b, Corollaries 1 and 2) 31 Nummelin's Splitting Tehnique Suppose it is possible to nd a funtion s(x) and a probability measure (dy) suh that and (s) = for all x 2 E and all A 2 E Then the density s(x)(dx) > 0 P (x; A) s(x)(a) (3:1) r(x; y) = s(x)(dy) P (x; dy) (3:2) exists and an be taken to satisfy 0 r(x; y) 1 A pair (s; ) satisfying these onditions is alled a split for the transition kernel P Suppose the Markov hain X n is generated as usual by rst generating X 0, and then for eah n = 1; 2;, generating X n+1 from P (X n ; dy) To split this hain, a Bernoulli random variable S n with suess probability P fs n = 1jX 0 ; ; X n+1 ; S 0 ; ; S n?1 g = r(x n ; X n+1 ) is generated for eah n = 0; 1; Thus S n an be generated one X n+1 is available Dene and for eah n = 1; 2; ; let T 0 = inffn 0 : S n = 1g T n = inffn > T n?1 : S n = 1g; with the onvention that the inmum of the empty set is innity Then the T n are regeneration times, and at eah regeneration the Markov hain restarts with initial distribution : Theorem 1 Let X n and S n be onstruted as desribed above Then (X n ; S n?1 ) is a Markov hain with transition kernel given by P fx n+1 2 A; S n = 1jX 0 ; ; X n ; S 0 ; ; S n?1 g = s(x n )(A) P fx n+1 2 A; S n = 0jX 0 ; ; X n ; S 0 ; ; S n?1 g = P (X n ; A)? s(x n )(A) for any A 2 E Hene P fs n = 1jX n ; S n?1 g = s(x n ) and P fx n+1 2 AjX n ; S n?1 ; S n = 1g = (A) The regeneration times T i are almost surely all nite, the equilibrium regeneration rate is S lim S n n!1 n + 1 = (s) = s(x)(dx) = 3 r(x; y)(dx)p (x; dy);

21 and Y i = XT i j=t i?1+1 f(x j ) for i = 1; ; n Then the pairs (N i ; Y i ) are i:i:d:, and if E[jY i j] < 1 and E[N i ] < 1, then ^ n = P Yi P Ni = Y N! (2:1) by the law of large numbers and the renewal theorem If the Y i and N i have nite varianes, then the distribution of p n(^ n? ) onverges to a N(0; 2 ) distribution, and an be estimated using the variane estimation formula for a ratio estimator, ^ = q P(Yi? ^ n N i ) 2 p n N : (2:2) If the number of tours is not large, then a jakknife estimate of variane may be more reliable (Ripley 1987, Setion 64) The motivation for regenerative simulation extends beyond asymptotis Sine E hx Yi i = E hx Ni i (2:3) and hx i X E (Yi? N i ) 2 = Var Yi? N i ; (2:4) the bias in ^ n is only due to ratio estimation, and the same is approximately true for ^ 2 Provided regenerations atually our, there is no need to worry about the bias due to a slow mixing rate This is true both for the estimate itself and for the estimate of unertainty Of ourse, means of i:i:d: samples an be problemati due to a heavy tailed distribution With regenerative simulation, the proess is usually started with a regeneration, so T 0 = 0 If this is not possible, the simulation may have to be run from an arbitrary starting point until a regeneration ours at some random time T 0 0 In this ase the initial observations X 0 ; ; X T0?1 are usually disarded A possible drawbak of regenerative simulation is that using a xed number of tours n leads to a random total run length T n This run length ould be quite long if the time between regenerations is large and very variable A regenerative simulation analysis an be used to estimate the variane of a simulation estimator even if the total run length is xed at, say, t observations If n t is the number of omplete tours within the rst t observations, then an average over all t observations is lose to the average over only the n t omplete tours As a result, the standard error formula (22) remains asymptotially valid Beause of the waiting time paradox, the inomplete tour inluding the nal observation is rather unusual This leads to some biases that may be signiant if the number of tours is small Ripley (1987, Setion 64) and Bratley, Fox and Shrage (1987, Setions 332 and 37) disuss these issues and give further referenes An intermediate strategy is to x a target number t of observations, and to run the simulation until the next regeneration after t, ie to run n t + 1 omplete tours This removes the waiting time paradox, and (23) and (24) remain valid 2