General Construction of Irreversible Kernel in Markov Chain Monte Carlo Metropolis heat bath Suwa Todo Department of Applied Physics, The University of Tokyo Department of Physics, Boston University (from April) Collaborator : Synge Todo 1
Markov Chain Monte Carlo Metropolis et al. (1953) Monte Carlo method Numerical integration method using a (quasi-)random number Uniformly random sampling suffers from curse of dimensionality Importance sampling P Markov chain Monte Carlo (MCMC) = Numerical method generating states from a target distribution Γ Markov chain Sample correlation Autocorrelation time Needed rough number of steps for forgetting the past C(t) = A i+ta i A 2 A 2 A 2 2
For Efficient Sampling Three key points for reducing the autocorrelation in MCMC 1. Determination of ensemble : Multicanonical, Exchange Monte Carlo 2. Selection of candidate states: Cluster algorithm, Hybrid Monte Carlo 3. Optimization of transition kernel: Metropolis, Heat bath (Gibbs sampler) Let us consider a discrete variable Heat bath algorithm (Gibbs sampler) Barker (1965), Geman and Geman (1984) Metropolis (Metropolis-Hastings) algorithm Metropolis et al. (1953), Hastings (197) p(c i c j )= w(c j) k w(c k) p(c i c j )= 1 n 1 min 1, w(c j) w(c i ) The above algorithms are the standard methods in MCMC. However, they are not optimal! 3
What is Optimal? P MG = P IMG = π 2 1 π 1 1... π 3 1 π 1 π 3.. π n 1 π 1 π 1 1 π 1 π 1 1 π 1 π 2 1 π 2 π 1 1 π 1 π 2 1 π 2 π 3 1 π 3 1 π 2 1............ π n 1 π 2 π n 1 π 3 1... y 1 y 1 y 1 w 2 w 1 y 1 y 2 y 2 w 3 w 1 y 1 w 3. w n w 1 y 1 w 2 y 2 y 3......... w n w 2 y 2 w n w 3 y 3 1 y 1 y 2... Peskun (1973) Metropolized Gibbs Sampler Liu (1996) = Gibbs sampler excluding the current state + Metropolis, Pij MG = min(π i /(1 π j ),π i /(1 π i )) Iterative Metropolized Gibbs Sampler, π i = w i j w j π 1 π 2 π n Frigessi et al. (1992) 4
Reversibility of Markov Chain 5
Stochastic Flow 6
Geometric Allocation : n=2 7
Inevitable Rejection? : n=4 σ i 8
H.S. and Todo, (21) Phys. Rev. Lett., 15, 1263 New Algorithm w 1 9
Comparison with Conventional Methods Metropolis heat bath Algorithm 1 1
Acceleration in Potts Model Square lattice Autocorrelation time of the structure factor 1 3 q = 4 q = 8 Binning analysis Ferromagnetic q-state Potts model σ i Wu (1982) int 1 2 1 1 Metropolis Heat Bath Metropolized Gibbs Iterative MG Optimal Average Optimal Rejection Hwang (25) 1.86.9.94.98 Metropolis Heat bath (Gibbs sampler) T.74.76.78 Phase transition temperature Continuous 1st order Significantly speed up! 11
Dynamical Exponent 3.5 3 Autocorrelation time of the structure factor log 1 int 2.5 2 1.5 1.5 -.5.6.8 1 1.2 1.4 1.6 1.8 2 log 1 L Metropolis Heat Bath Metropolized Gibbs Iterative MG Optimal Average Optimal Rejection 2.3 * x - 1.8 τ int L z z 2.3 Same dynamical exponent. But we always gain a factor (6 than Metropolis). 12
Convergence Speed Up Two criteria of Markov chain 1. Short autocorrelation time = small asymptotic variance = high-sampling efficiency 2. Rapid distribution convergence = short burn-in (thermalization) process Square lattice L = 32 Relaxation of the order parameter Start with all up state log 1 m -.5 -.1 -.15 -.2 -.25 -.3 Metropolis Heat Bath Metropolized Gibbs Iterative MG Optimal Average Optimal Rejection -.35.5 1 1.5 2 2.5 3 log 1 t Our algorithm is the best! Not only the sampling efficiency but also the distribution convergence is improved. 13
Extension to Continuous Variable 1.5 1 5-2 2-2 2-5 -1-1 -5 5 1-1 -5 5 1 14
Comparison with Conventional Methods 1 1e+6 1 Heat Bath Overrelaxation Ordered Overrelaxation present c=.4, w=.5 int 1 1 1 1.1 1 1 1 1 2 / 1 15
Beyond Metropolis Frenkel et al. (21), Liu et al. (2), Qin et al. (21) Neal (1994) 16
Decreasing Rejection Rate P (x 1, x 2 ) exp (x1 x 2 ) 2 2σ1 2 + (x 1 + x 2 ) 2 2σ 2 2 (x1 x 2 ) 2 2σ 2 1 + (x 1 + x 2 ) 2 2σ 2 2 h + h2 4. 5 5.6.5.4.3.2.1 N(, 1) Metropolis present n=3 present n=4 2 4 6 8 1 12 14 16 18 2 σ 2 /σ 1 35 3 25 2 15 1 5 Metropolis present n=3 present n=4 5 5 τ int of (x 1 +x 2 ) 2 2 4 6 8 1 12 14 16 18 2 1 5 σ 2 /σ 1 17
cf. H.S. and Todo, Phys. Rev. Lett., 15, 1263 (21) arxiv:116.3562 Summary Metropolis heat bath Suwa Todo 18