A new class of interacting Marov Chain Monte Carlo methods P Del Moral, A Doucet INRIA Bordeaux & UBC Vancouver Worshop on Numerics and Stochastics, Helsini, August 2008
Outline 1 Introduction Stochastic sampling problems Some stochastic engineering models Interacting sampling methods 2 Some classical distribution flows Feynman-Kac models Static Boltzmann-Gibbs models 3 Interacting stochastic sampling technology Mean field particle methods Interacting Marov chain Monte Carlo models (i-mcmc) 4 Functional fluctuation & comparisons 5 Some references
Stochastic sampling problems Stochastic sampling problems Nonlinear distribution flow with level of complexity η n (dx n ) = γ n(dx n ) γ n (1) Two objectives : Time index n N State var x n E n 1 Sampling independent random variables wrt η n 2 Computation of the normalizing constants γ n (1) (= Z n Partition functions)
Some stochastic engineering models Stochastic engineering Conditional & Boltzmann-Gibbs measures Filtering: Signal-Observation (X n, Y n ) [Radar, Sonar, GPS, ] η n = Law(X n (Y 0,, Y n )) Rare events: [Overflows, ruin processes, epidemic propagations,] η n = Law(X n n intermediate events) & Z n = P (Rare event) Molecular simulation: [ground state energies, directed polymers] η n := Feynman-Kac/Boltzmann-Gibbs Free Marov motion in an absorbing medium Combinatorial counting, Global optimization, HMM η n = 1 Z n e βnv (x) λ(dx) or η n = 1 Z n 1 An (x) λ(dx)
Interacting sampling methods Two simple ingredients Find or Understand the probability mass transformation η n = Φ n (η n 1 ) Cooling schemes, temp variations, constraints sequences, subset restrictions, observation data, conditional events, Natural interacting sampling idea : Use η n 1 or its empirical approx to sample wrt η n 1 Monte-Carlo/ Mean Field models : η n = Law(X n ) with Marov : X n 1 η n 1 X n 2 Interacting MCMC models : { } Use the occupation measures MCMC target η of an MCMC with target η n n 1
Feynman-Kac models Feynman-Kac distribution flows Wea representation: [f n test funct on a state space E n ] η n (f n ) = γ n(f n ) γ n (1) with γ n (f n ) = E f n (X n ) G p (X p ) 0 p<n ( ) A Key Formula: Z n = E 0 p<n G p(x p ) = 0 p<n η p(g p ) Path space models X n = (X 0,, X n) Examples G n [0, 1] particle absorption models G n = Observation lielihood function Filtering models G n = 1 An Conditional/Restriction models
Feynman-Kac models Nonlinear distribution flows Evolution equation: η n+1 = Φ n+1 (η n ) = Ψ Gn (η n )M n+1 With the only 2 transformations : X -Free Marov transport eq : [M n (x n 1, dx n ) from E n 1 into E n ] (η n 1 M n )(dx n ) := η n 1 (dx n 1 ) M n (x n 1, dx n ) E n 1 Bayes-Boltzmann-Gibbs transformation : Ψ Gn (η n )(dx n ) := 1 η n (G n ) G n(x n ) η n (dx n )
Static Boltzmann-Gibbs models Boltzmann-Gibbs distribution flows Target distribution flow : η n (dx) g n (x) λ(dx) Product hypothesis : g n = g n 1 G n 1 = η n = Ψ Gn 1 (η n 1 ) Running Ex: g n = 1 An with A n G n 1 = 1 An g n = e βnv with β n G n 1 = e (βn βn 1)V Problem : η n = Ψ Gn 1 (η n 1 ) = unstable equation
Static Boltzmann-Gibbs models Feynman-Kac modeling Choose M n (x, dy) st local fixed point eq η n = η n M n (Metropolis, Gibbs,) Stable equation : g n = g n 1 G n 1 = η n = Ψ Gn 1 (η n 1 ) = η n = η n M n = Ψ Gn 1 (η n 1 )M n = FK-model Feynman-Kac dynamical formulation (X n Marov M n ) f (x) g n (x) λ(dx) E f (X n ) G p (X p ) 0 p<n Interacting Metropolis/Gibbs/ stochastic algorithms
Mean field particle methods Mean field interpretation Nonlinear Marov models : Always K n,η (x, dy) Marov st η n = Φ n (η n 1 ) = η n 1 K n,ηn 1 =Law ( X n ) ie : P(X n dx n X n 1 ) = K n,ηn 1 (X n 1, dx n ) Mean field particle interpretation Marov chain ξ n = (ξn, 1, ξn N ) En N st η N n := 1 δ N ξ i n N η n 1 i N Particle approximation transitions ( 1 i N) ξn 1 i ξn i K n,η N n 1 (ξn 1, i dx n )
Mean field particle methods Discrete generation mean field particle model Schematic picture : ξ n E N n ξ n+1 E N n+1 ξ 1 n ξ i n ξ N n K n+1,η N n ξ 1 n+1 ξ i n+1 ξ N n+1 Rationale : η N n N η n = K n+1,η N n N K n+1,ηn = ξ i n almost iid copies of X n
Mean field particle methods Ex: Feynman-Kac distribution flows FK-Nonlinear Marov models : ɛ n = ɛ n (η n ) 0 st η n -ae ɛ n G n [0, 1] (ɛ n = 0 not excluded) K n+1,ηn (x, dz) = S n,ηn (x, dy) M n+1 (y, dz) S n,ηn (x, dy) := ɛ n G n (x) δ x (dy) + (1 ɛ n G n (x)) Ψ Gn (η n )(dy) Mean field genetic type particle model : Examples : ξ i n E n accept/reject/selection ξ i n E n proposal/mutation ξ i n+1 E n+1 G n = 1 A illing with uniform replacement M n -Metropolis/Gibbs moves G n -interaction function (subsets fitting or change of temperatures)
Mean field particle methods Mean field genetic type particle model : ξ 1 ṇ ξ i ṇ ξ N n S n,η N n ξ 1 n ξ i n ξ N n M n+1 ξ 1 n+1 ξ i n+1 ξ N n+1 Accept/Reject/Selection transition : S n,η N n (ξ i n, dx) := ɛ n G n (ξn) i δ ξ i n (dx) + ( 1 ɛ n G n (ξn) ) i N G n(ξ j P n ) j=1 N =1 Gn(ξ )δ ξ j (dx) n n Ex : G n = 1 A, ɛ n = 1 G n (ξ i n) = 1 A (ξ i n)
Mean field particle methods Path space models X n = (X 0,, X n) genealogical tree/ancestral lines η N n := 1 N = 1 δ N (ξ i 0,n,ξ1,n i,,ξi n,n ) N η n δ ξ i n 1 i N 1 i N Unbias particle approximations : γn N (1) = ηp N (G p ) N γ n (1) = η p (G p ) 0 p<n 0 p<n Ex G n = 1 A : γn N (1) = (success % at p) 0 p<n FK-Mean field particle models = sequential Monte Carlo, population Monte Carlo, particle filters, pruning, spawning, reconfiguration, quantum Monte carlo, go with the winner
Interacting Marov chain Monte Carlo models (i-mcmc) Objective Find a series of MCMC models X (n) := (X (n) ) 0 st Advantages η (n) = 1 + 1 η n 0 l δ X (n) l Use η (n) η n to define X (n+1) with target η n+1 Using η n the sampling η n+1 is often easier Improve the proposition step in any Metropolis type model with target η n+1 ( enters the stability prop of the flow η n ) Increases the precision at every time step But CLT variance often CLT variance mean field models Easy to combine with mean field stochastic algorithms
Interacting Marov chain Monte Carlo models (i-mcmc) Interacting Marov chain Monte Carlo models Find M 0 and a collection of transitions M n,µ st η 0 = η 0 M 0 and Φ n (µ) = Φ n (µ)m n,µ (X (0) ) 0 Marov chain M 0 Given X (n), we let X (n+1) Rationale : η (n) η n = Example : { with Marov transtions M n+1,η (n) Φ n+1 (η (n) ) Φ n+1(η n ) = η n+1 M n+1,ηn with fixed point η n+1 M n+1,η (n) = η (n+1) η n+1 M n,µ (x, dy) = Φ n (µ)(dy) X (n+1) rv Φ n+1 ( η (n) )
Interacting Marov chain Monte Carlo models (i-mcmc) ((n 1)-th chain) X (n 1) 0 X (n 1) 1 X (n 1) η (n 1) η n 1 (n-th chain) M n,η (n 1) X (n) 0 X (n) M n,ηn 1 X (n) +1
[MEAN FIELD PARTICLE MODEL] Nonlinear semigroup Φ p,n(η p) := η n Local fluctuation theorem : W N n := N hη N n Φn η N i n 1 W n Centered Gaussian field Local transport formulation : η 0 η 1 = Φ 1 (η 0 ) η 2 = Φ 0,2 (η 0 ) Φ 0,n (η 0 ) η0 N Φ 1 (η0 N ) Φ 0,2(η0 N ) Φ 0,n(η0 N ) η1 N Φ 2 (η1 N ) Φ 1,n(η1 N ) η2 N Φ 2,n (η2 N ) ηn 1 N Φ n(ηn 1 N ) ηn N Key decomposition formula : nx η N n ηn = [Φ q,n(η N q ) Φq,n(Φq(ηN q 1 ))] q=0 1 N nx W N q Dq,n First order decomp Φp,n(η) Φp,n(µ) (η µ)dp,n + (η µ) 2 q=0 Example Functional CLT : N hη N i nx n ηn W qd q,n q=0
[i-mcmc] Nonlinear sg Φ p,n(η p) = η n with a first order decomp : Φ p,n(η) Φ p,n(µ) (η µ)d p,n + (η µ) 2 Functional CLT for correlated/interacting MCMC models : hη (n) i nx η n q=0 p (2(n q))! V qd q,n (n q)! with `V q q 0 Centered Gaussian field E V q(f ) 2 = η q h(f η q(f )) 2i + 2 P» m 1 ηq (f η q(f )) Mq,η m (f η q 1 q(f )) Comparisons : [Mean field case] `W q q 0 Centered Gaussian field E W q(f ) 2 = η q 1 nk q,ηq 1 (f K q,ηq 1 (f )) 2o Case : K q,η(x, dy) = M q,η(x, dy) = Φ q(η)(dy) = (V q = W q) = [Mean field] > [i-mcmc]
Some references Interacting stochastic simulation algorithms Mean field and Feynman-Kac particle models : Feynman-Kac formulae Genealogical and interacting particle systems, Springer (2004) Refs joint wor with L Miclo A Moran particle system approximation of Feynman-Kac formulae Stochastic Processes and their Applications, Vol 86, 193-216 (2000) joint wor with L Miclo Branching and Interacting Particle Systems Approximations of Feynman-Kac Formulae Séminaire de Probabilités XXXIV, Lecture Notes in Mathematics, Springer-Verlag Berlin, Vol 1729, 1-145 (2000) Sequential Monte Carlo models : joint wor with Doucet A, Jasra A Sequential Monte Carlo Samplers JRSS B (2006) joint wor with A Doucet On a class of genealogical and interacting Metropolis models Sém de Proba 37 (2003)
Interacting stochastic simulation algorithms i-mcmc algorithms : joint wor with A Doucet Interacting Marov Chain Monte Carlo Methods For Solving Nonlinear Measure-Valued Eq, HAL-INRIA RR-6435, (Feb 2008) joint wor with B Bercu and A Doucet Fluctuations of Interacting Marov Chain Monte Carlo Models HAL-INRIA RR-6438, (Feb 2008) joint wor with C Andrieu, A Jasra, A Doucet Non-Linear Marov chain Monte Carlo via self-interacting approximations Tech report, Dept of Math, Bristol Univ (2007) joint wor with A Brocwell and A Doucet Sequentially interacting Marov chain Monte Carlo Tech report, Dept of Statistics, Univ of British Columbia (2007)