A Backward Particle Interpretation of Feynman-Kac Formulae

Transcription

1 A Backward Particle Interpretation of Feynman-Kac Formulae P. Del Moral Centre INRIA de Bordeaux - Sud Ouest Workshop on Filtering, Cambridge Univ., June 14-15th 2010 Preprints (with hyperlinks), joint works with A. Doucet & S.S. Singh: A Backward Particle Interpretation of Feynman-Kac Formulae HAL-INRIA RR-7019 (2009), to appear in M2AN (2010). Forward Smoothing Using Sequential Monte Carlo CUED/F-INFENG/TR.638. Cambridge University, Engineering Dpt. (2009). P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 1 / 26

2 Outline 1 Introduction, motivations 2 Nonlinear Markov models 3 Some convergence results 4 Additive functionals P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 2 / 26

3 Summary 1 Introduction, motivations Some notation Feynman-Kac measures Filtering and sensitivity analysis 2 Nonlinear Markov models 3 Some convergence results 4 Additive functionals P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 3 / 26

4 Some notation E measurable state space, P(E) proba. on E, B(E) bounded meas. functions (µ, f ) P(E) B(E) µ(f ) = µ(dx) f (x) M(x, dy) integral operator over E M(f )(x) = M(x, dy)f (y) [µm](dy) = µ(dx)m(x, dy) (= [µm](f ) = µ[m(f )] ) Bayes-Boltzmann-Gibbs transformation : G : E [0, [ with µ(g) > 0 Ψ G (µ)(dx) = 1 µ(g) G(x) µ(dx) E finite Vector-Matrix notation µ = [µ(1),..., µ(d)] and f = [f (1),..., f (d)] P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 4 / 26

5 Feynman-Kac measures Markov X n with transitions M n (x n 1, dx n ) on some state space E n. Potential functions G n : x n E n G n (x n ) [0, [ Feynman-Kac path measures: dq n := 1 G p (X p ) Z n dp n with P n := Law(X 0,..., X n ) 0 p<n The n-time marginals: f n B(E n ) η n (f n ) := γ n(f n ) γ n (1) with γ n (f n ) := E f n (X n ) G p (X p ) 0 p<n Updated measures Q n, η n, and γ n w.r.t. the product 0 p n P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 5 / 26

6 Example : Nonlinear filtering P ((X n, Y n ) d(x, y) (X n 1, Y n 1 )) := M n (X n 1, dx) g n (x, y) λ Y n (dy) Given the observation sequence Y = y with G n (x n ) = g n (x n, y n ) η n = Law(X n 0 p n Y p = y p ) γ n (1) = p n (y 0,..., y n ) ( density w.r.t. 0 p n λ Y ) p In path space settings smoothing and path estimation Q n = Law((X 0,..., X n ) 0 p n Y p = y p ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 6 / 26

7 Sensitivity analysis Expected Maximization θ M n (x n 1, dx n ) = p θ n(x n 1, x n ) λ X n (dx n ) and g θ n (x, y) λ Y n (dy) Parametric Feynman-Kac models : (G n, Q n, γ n, η n,...) (G θ n, Q θ n, γ θ n, η θ n,...) θ log γθ n(1) = Q θ n(f θ n ) with the additive functional and F θ n (x 0,..., x n ) := θ Q θ n(ϕ n ) = Q ( θ n Fn θ [ϕ n Q ) θ n(ϕ n )] 0 k n f θ k (x k 1, x k ) fk θ (x k 1, x p ) := log G p θ (x k ) + log pθ k (x k 1, x k ) θ θ Expected Maximization min entropy (additive fct. F θ n θ E n (θ, θ ) := Q θ n ( log (d Q θ n/d without derivatives) )) θ Q n Q θ n(fn θ ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 7 / 26

8 3 Key observations η n (f n ) := γ n(f n ) γ n (1) with γ n (f n ) := E f n (X n ) G p (X p ) 0 p<n Path space models: [ X n := (X 0,..., X n) & G n (X n ) := G n(x n)] = η n = Q n A multiplicative formula: Z n = E G p (X p ) = η p (G p ) 0 p<n 0 p<n Proof: Z n := γ n (1) = γ n 1 (G n 1 ) = η n 1 (G n 1 ) γ n 1 (1) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 8 / 26

9 3 Key observations Path space models: M n (X n 1, dx n ) = P (X n dx n X n 1 ) = H n (X n 1, x n ) λ n (dx n ) Q n (d(x 0,..., x n )) = η n (dx n ) M n,ηn 1 (x n, dx n 1 )... M 1,η0 (x 1, dx 0 ) with the backward Markov transitions : M n,ηn 1 (x n, dx n 1 ) G n 1 (x n 1 ) H n (x n 1, x n ) η n 1 (dx n 1 ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 9 / 26

10 Summary 1 Introduction, motivations 2 Nonlinear Markov models McKean distribution models Mean field particle interpretations The 4 types of particle approximation measures 3 Some convergence results 4 Additive functionals P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 10 / 26

11 Nonlinear Markov models Flows of Feynman-Kac measures A two step correction prediction model η n Updating-correction Prediction/Markov transport η n = Ψ Gn (η n ) η n+1 = η n M n+1 Selection nonlinear transport formulae Ψ Gn (η n ) = η n S n,ηn with, for any ɛ n = ɛ n (η n ) [0, 1] s.t. ɛ n G n 1 S n,ηn (x,.) := ɛ n G n (x) δ x + (1 ɛ n G n (x)) Ψ Gn (η n ) η n+1 = η n (S n,ηn M n+1 ) := η n K n+1,ηn P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 11 / 26

12 Nonlinear Markov chains η n = Law(X n )=Perfect sampling algorithm Nonlinear transport formulae : η n+1 = η n K n+1,ηn with the collection of Markov probability transitions : K n+1,ηn = S n,ηn M n+1 Local transitions : P(X n dx n X n 1 ) = K n,η n 1 (X n 1, dx n ) avec η n 1 = Law(X n 1 ) McKean measures (canonical process) : P n (d(x 0,..., x n )) = η 0 (dx 0 ) K 1,η0 (x 0, dx 1 )... K n,ηn 1 (x n 1, dx n ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 12 / 26

13 Mean field particle interpretations Sampling pb Mean field particle interpretations Markov Chain ξ n = (ξn, 1..., ξn N ) En N s.t. η N n := 1 δ N ξ i n N η n 1 i N Approximated local transitions ( 1 i N) ξn 1 i ξn i K n,η N n 1 (ξn 1, i dx n ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 13 / 26

14 Schematic picture : ξ n E N n ξ n+1 E N n+1 Rationale : ξ 1 n. ξ i n. ξ N n K n+1,η N n ξ 1 n+1. ξ i n+1. ξ N n+1 η N n N η n = K n+1,η N n N K n+1,ηn = ξ i i.i.d. copies of X Particle McKean measures : 1 N N δ (ξ i 0,...,ξn i ) N Law(X 0,..., X n ) i=1 P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 14 / 26

15 Some key advantages Mean field models=stochastic linearization/perturbation technique : η N n = η N n 1K n,η N n N W N n avec Wn N W n Centered Gaussian Fields. η n = η n 1 K n,ηn 1 stable No propagation of local sampling errors = Uniform control w.r.t. the time horizon No burning, no need to study the stability of MCMC models. Stochastic adaptive grid approximation Nonlinear system positive-benefic interactions. Simple and natural sampling algorithm. P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 15 / 26

16 Feynman-Kac models Genetic type stochastic algo. ξṇ 1 ξ 1 M n+1 n ξn+1 1. S ξṇ i n,η N n.. ξ n i ξn+1 i... ξn N ξn+1 N Acceptance/Rejection-Selection : [Geometric type clocks] ξ N n S n,η N n (ξ i n, dx) := ɛ n G n (ξn) i δ ξ i n (dx) + ( 1 ɛ n G n (ξn) ) i N G n(ξ j P n ) j=1 N k=1 Gn(ξk )δ ξ j (dx) n n Ex. : G n = 1 A G n (ξ i n) = 1 A (ξ i n) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 16 / 26

17 Interaction/branch. process 4 types of occupation measures (N = 3) = = = Current population 1 N Genealogical tree 1 N N i=1 N i=1 Unbias particle normalizing Constants Zn N := ηp N (G p ) 0 p<n δ ξ i n i-th individual at time n η n δ (ξ i 0,n,ξ i 1,n,...,ξi n,n) i-th ancestral line Q n 0 p<n η p (G p ) = Z n P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 17 / 26

18 Interaction/branch. process 4 types of occupation measures (N = 3) = = = Complete genealogical tree McKean measures 1 N N i=1 δ (ξ i 0,ξ i 1,...,ξi n) η 0(dx 0 )K 1,η0 (x 0, dx 1 )... K n,ηn 1 (x n 1, dx n ) Backward Feynman-Kac path measures [ elementary Matrices operations] Q N n (d(x 0,..., x n )) = ηn N (dx n ) M n,η N n 1 (x n, dx n 1 )... M 1,η N 0 (x 1, dx 0 ) η n (dx n ) M n,ηn 1 (x n, dx n 1 )... M 1,η0 (x 1, dx 0 ) = Q n (d(x 0,..., x n )) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 18 / 26

19 Equivalent Stochastic Algorithms : Genetic and evolutionary type algorithms. Spatial branching models. Sequential Monte Carlo methods. Population Monte Carlo models. Diffusion Monte Carlo (DMC), Quantum Monte Carlo (QMC),... Some botanical names = application domain areas : particle filters, bootsrapping, selection, pruning-enrichment, reconfiguration, cloning, go with the winner, spawning, condensation, grouping, rejuvenations, harmony searches, biomimetics, splitting, [(Meta)Heuristics] 1996 Feynman-Kac mean field particle model P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 19 / 26

20 Summary 1 Introduction, motivations 2 Nonlinear Markov models 3 Some convergence results Non asymptotic theorems Unnormalized models 4 Additive functionals P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 20 / 26

21 Asympt. theo. TCL,PGD, PDM (n,n) fixed some examples : Empirical processes : sup sup N E( η N n η n p F n ) < n 0 N 1 Concentration inequalities uniform w.r.t. time : sup P( ηn N (f n ) η n (f n ) > ɛ) c exp (Nɛ 2 )/(2 σ 2 ) n 0 + Guionnet sup n 0 (IHP 01) & Ledoux sup Fn (JTP 00) & Rio AAP 10 Propagations of chaos exansions (+Patras,Rubenthaler (AAP 09)) : P N n,q := Loi(ξ 1 n,..., ξ q n) η q n + 1 N 1 P n,q N k k P n,q + 1 N k+1 k+1 P N n,q with sup N 1 k+1 P N n,q tv < & sup n 0 1 P n,q tv c q 2. P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 21 / 26

22 Unnormalized models Un-bias particle approximation measures γn N (f n ) := ηn N (f n ) ηp N (G p ) 0 p<n Asymptotic theorms : fluctuations & deviations + A. Guionnet (AAP 99, SPA 98), + L. Miclo (SP 2000), + D. Dawson Non asymptotic theory : bias and variance estimates 1 Taylor type expansion (+Patras & Rubenthaler (AAP 09) : E ( (γ N n ) q (F ) ) =: Q N n,q(f ) = γ q n (F ) + 1 k (q 1)(n+1) 1 N k k Q n,q (F ) 2 Variance estimates (+Cerou & Guyader Hal-INRIA 08 & IPH 2010) : ( [γ N E n (f n ) γ n (f n ) ] ) 2 c n N γ n(1) 2 P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 22 / 26

23 Summary 1 Introduction, motivations 2 Nonlinear Markov models 3 Some convergence results 4 Additive functionals P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 23 / 26

24 Additive funct. (+ Doucet & Singh Hal-INRIA 09 & M2AN 2010) Path space models P n := Law(X 0,..., X n ) dq n := 1 G p (X p ) Z n dp n 0 p<n Hyp. : M n (x n 1, dx n ) = H n (x n 1, x n ) λ n (dx n ) Q n (d(x 0,..., x n )) = η n (dx n ) M n,ηn 1 (x n, dx n 1 )... M 1,η0 (x 1, dx 0 ) with the backward transitions : M p+1,η (x, dx ) G p (x ) H p+1 (x, x) η(dx ) Particle estimates complete genealogical tree : Q N n (d(x 0,..., x n )) = η N n (dx n ) M n,η N n 1 (x n, dx n 1 )... M 1,η N 0 (x 1, dx 0 ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 24 / 26

25 2 type of path-space estimates Complete genealogical tree = McKean meas. FK-Path space 1 N N δ (ξ i 0,...,ξn i ) N Loi(X 0,..., X n ) & Q N n N Q n i=1 Simple genealogical tree = FK-Path space N η N n = 1 N i=1 δ (ξ i 0,n,ξ i 1,n,...,ξi n,n ) N Q n = η n Main problem : Path degeneracy w.r.t. time horizon (as any genetic ancestral tree) Roughly : Uniform estimates linear estimates w.r.t. the time horizon P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 25 / 26

26 Some non asymptotic estimates Additive functional : F n (x 0,..., x n ) = 1 n p n f p (x p ) Bias estimate+uniform L p -bounds + variance (1/N 2 ) N E ( [(Q N n Q n )(F n )] 2) c 1/n +1/N }{{} bias term Uniform exponential concentration 1 log sup P N n 0 ( [Q N n Q n ](F n ) b ) + ɛ ɛ 2 /(2b 2 ) N P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 26 / 26