Concentration inequalities for Feynman-Kac particle models. P. Del Moral. INRIA Bordeaux & IMB & CMAP X. Journées MAS 2012, SMAI Clermond-Ferrand

Size: px

Start display at page:

Download "Concentration inequalities for Feynman-Kac particle models. P. Del Moral. INRIA Bordeaux & IMB & CMAP X. Journées MAS 2012, SMAI Clermond-Ferrand"

Muriel Jordan
5 years ago
Views:

1 Concentration inequalities for Feynman-Kac particle models P. Del Moral INRIA Bordeaux & IMB & CMAP X Journées MAS 2012, SMAI Clermond-Ferrand Some hyper-refs Feynman-Kac formulae, Genealogical & Interacting Particle Systems with appl., Springer (2004) Sequential Monte Carlo Samplers JRSS B. (2006). (joint work with A. Doucet & A. Jasra) A Backward Particle Interpretation of Feynman-Kac Formulae M2AN (2010). (joint work with A. Doucet & S.S. Singh) Concentration Inequalities for Mean Field Particle Models. Annals of Applied Probability (2011) (joint work with Emmanuel Rio). On the concentration of interacting processes. Foundations & Trends in Machine Learning [170p.] (2012). (joint work with Peng Hu & Liming Wu) [+ Refs] More references on the website delmoral/index.html [+ Links]

2 Feynman-Kac models Some basic notation Description of the models Some examples Ineracting particle interpretations Concentration inequalities

3 Basic notation P(E) probability meas., B(E) bounded functions on E. (µ, f ) P(E) B(E) µ(f ) = µ(dx) f (x) Q(x 1, dx 2 ) integral operators x 1 E 1 x 2 E 2 Q(f )(x 1 ) = Q(x 1, dx 2 )f (x 2 ) [µq](dx 2 ) = µ(dx 1 )Q(x 1, dx 2 ) (= [µq](f ) = µ[q(f )] )

4 Basic notation Boltzmann-Gibbs transformation G 0 µ(dx) Ψ G (µ)(dx) = 1 µ(g) G(x) µ(dx) When G 1 we have the (non unique) Markov transport equation with Ψ G (µ) = µs µ,g S µ,g (x, dy) = G(x) δ x (dy) + (1 G(x)) Ψ G (µ)(dy)

5 Feynman-Kac models Markov chain X n E n and a potential function G n : E n [0, [ dq n := 1 G Z n p (X p ) dp n with P n = Law(X 0,..., X n ) 0 p<n Transition/Excursions/Path spaces X n = (X n, X n+1) X n = X [T n,t n+1] X n = (X 0,..., X n) Continuous time models tn+1 X n = X [t n,t n+1] & G n (X n ) = exp V t (X t )dt t n

6 Some examples Confinements: X n Z d A & G n := 1 A. and Q n = Law ((X 0,..., X n ) X p A, 0 p < n) Z n = Proba (X p A, 0 p < n) SAW : X n = (X p) 0 p n & G n (X n ) = 1 X n {X 0,...,X n 1 } and Q n = Law ( (X 0,..., X n) X p X q, 0 p < q < n ) Z n = Proba ( X p X q, 0 p < q < n )

7 Some examples Filtering : Y n = H n (X n, V n ) & G n (x n ) := p Yn X n (y n x n ). and Q n = Law ((X 0,..., X n ) Y p = y p, 0 p < n) Z n = p Y0,...,Y n (y 0,..., y n ) Multilevel splitting : A n, with B non critical recurrent subset. T n := inf {t T n 1 : X t (A n B)} X n = (X t ) t [Tn,T n+1] & G n (X n ) = 1 An+1 (X T n+1 ) and ( ) Q n = Law X [T X 0,T n] hits A n 1 before B Z n = P(X hits A n 1 before B)

8 Some examples Absorption models : Xn c En c absorption (1 G n) X n c exploration M n+1 Xn+1 c Q n = Law((X0 c,..., Xn c ) T abs. n) & Z n = Proba ( T abs. n ) Quasi-invariant measures : (G n, M n ) = (G, M) & M µ-reversible 1 n log P ( T abs. n ) n λ = top spect. of Q(x, dy) = G(x)M(x, dy) [Frobenius theo] Q(h) = λh = λ eigenfunction (ground state) P(X c n dx T abs. > n) n 1 µ(h) h(x) µ(dx)

9 Some examples Doob h-processes X h : M h (x, dy) = 1 λ h 1 (x)q(x, dy)h(y) = Q(x, dy)h(y) Q(h)(x) = M(x, dy)h(y) M(h)(x) Q n (d(x 0,..., x n )) Proba((X h 0,..., X h n ) d(x 0,..., x n )) h 1 (x n )

10 Feynman-Kac models Ineracting particle interpretations Nonlinear evolution equation Mean field particle models Graphical illustration First order expansions Uniform concentration w.r.t. time Particle free energy Concentration inequalities

11 Flow of n-marginals [X n Markov with transitions M n ] η n (f ) = γ n (f )/γ n (1) with γ n (f ) := E f (X n ) G p (X p ) 0 p<n (γ n (1) = Z n ) Nonlinear evolution equation : η n+1 = Ψ Gn (η n )M n+1 Z n+1 = η n (G n ) Z n Nonlinear m.v.p. = Law of a Markov X n (perfect sampler) η n+1 = Φ n+1 (η n ) = η n (S n,ηn M n+1 ) = η n K n+1,ηn = Law(X n+1 )

12 Examples related to product models η n (dx) := 1 Z n { n p=0 h p (x) } λ(dx) with h p 0 3 illustrations: h p (x) = e (βp+1 βp)v (x) β p = η n (dx) = 1 e βnv (x) λ(dx) Z n h p (x) = 1 Ap+1 (x) A p = η n (dx) = 1 1 An (x) λ(dx) Z n h p (θ) = p(y p θ, y 0,..., y p 1 ) = η n (dθ) = p(θ y 0,..., y n ) For any MCMC transitions M n with target η n, we have η n+1 = η n+1 M n+1 = Ψ hn+1 (η n )M n+1 Feynman-Kac model

13 Mean field particle model = Markov ξ n = (ξ i n) 1 i N E N n P (ξ n+1 dx ξ n ) = K n+1,η N n (ξn, i dx i ) with ηn N = 1 N 1 i N and the (unbiased) particle normalizing constants Zn+1 N = ηn N (G n ) Zn N = ηp N (G p ) 0 p n 1 i N δ ξ i n

14 Mean field particle model = Markov ξ n = (ξ i n) 1 i N E N n P (ξ n+1 dx ξ n ) = K n+1,η N n (ξn, i dx i ) with ηn N = 1 N 1 i N and the (unbiased) particle normalizing constants Zn+1 N = ηn N (G n ) Zn N = ηp N (G p ) 0 p n 1 i N δ ξ i n [K n+1,ηn = S n,ηn M n+1 ] Sequential particle simulation technique G n -acceptance-rejection with recycling M n+1 -propositions Genetic type branching particle model ξ n = (ξn) i G n selection 1 i N ξ n = ( ξ n) i M n mutation 1 i N ξ n+1 = (ξn+1) i 1 i N

15 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

16 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

17 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

18 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

19 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

20 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

21 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

22 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

23 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

24 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

25 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

26 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

27 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

28 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

29 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

30 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

31 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

32 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

33 Graphical illustration : η n η N n := 1 N 1 i N δ ξ i n

34 How to use the full ancestral tree model? G n 1 (x n 1 )M n (x n 1, dx n ) hyp = H n (x n 1, x n ) ν n (dx n ) Q n (d(x 0,..., x n )) = η n (dx n ) M n,ηn 1 (x n, dx n 1 )... M 1,η0 (x 1, dx 0 ) }{{} η n(dx n) H n+1(x n,x n+1)

35 How to use the full ancestral tree model? G n 1 (x n 1 )M n (x n 1, dx n ) hyp = H n (x n 1, x n ) ν n (dx n ) Q n (d(x 0,..., x n )) = η n (dx n ) M n,ηn 1 (x n, dx n 1 )... M 1,η0 (x 1, dx 0 ) }{{} η n(dx n) H n+1(x n,x n+1) Particle approximation = Random stochastic matrices Q N n (d(x 0,..., x n )) = η N n (dx n ) M n,η N n 1 (x n, dx n 1 )... M 1,η N 0 (x 1, dx 0 )

36 How to use the full ancestral tree model? G n 1 (x n 1 )M n (x n 1, dx n ) hyp = H n (x n 1, x n ) ν n (dx n ) Q n (d(x 0,..., x n )) = η n (dx n ) M n,ηn 1 (x n, dx n 1 )... M 1,η0 (x 1, dx 0 ) }{{} η n(dx n) H n+1(x n,x n+1) Particle approximation = Random stochastic matrices Q N n (d(x 0,..., x n )) = η N n (dx n ) M n,η N n 1 (x n, dx n 1 )... M 1,η N 0 (x 1, dx 0 ) Ex.: Additive functionals f n (x 0,..., x n ) = 1 n+1 0 p n f p(x p ) Q N n (f n ) := 1 n p n η N n M n,η N n 1... M p+1,η N p (f p ) }{{} matrix operations

37 The example of the h-process and the absorption model Q n (d(x 0,..., x n )) P((X h 0,..., X h n ) d(x 0,..., x n )) h 1 (x n ) Invariant measure µ h = µ h M h & normalized additive functionals f n (x 0,..., x n ) = 1 n p n f (x p ) = Q n (f n ) n µ h (f ) If G = G θ depends on some θ R f := θ log G θ θ log 1 λθ n n + 1 θ log Zθ n+1 = Q n (f n ) NB : Similar expression when M θ depends on some θ R.

38 4 particle estimates Individuals ξ i n almost iid with law η n η N n = 1 N 1 i N δ ξ i n Path space models Ancestral lines almost iid with law Q n η N n := 1 N 1 i N δ ancestral linen(i) Backward particle model Q N n (d(x 0,..., x n )) = η N n (dx n ) M n,η N n 1 (x n, dx n 1 )... M 1,η N 0 (x 1, dx 0 ) Normalizing constants Z n+1 = η p (G p ) N Zn+1 N = ηp N (G p ) 0 p n 0 p n (Unbiased)

39 How & Why it works (Computer Sci.) Stochastic adaptive grid approximation. (Stats) Universal acceptance-rejection-recycling sampling schemes. (Probab) Stochastic linearization/perturbation technique. η n = Φ n (η n 1 ) η N n = Φ n ( η N n 1 ) + 1 N V N n Theorem: (V N n ) n N (V n ) n independent centered Gaussian fields. Φ p,n (η p ) = η n stable sg No propagation of local errors = Uniform control w.r.t. the time horizon New concentration inequalities for (general) interacting processes

40 Key idea = First order expansions Key telescoping decomposition η N n η n = n p=0 [ Φp,n (η N p ) Φ p,n ( Φp (η N p 1) )] First order expansion N [ Φp,n (η N p ) Φ p,n ( Φp (η N p 1 ))] = N [Φ p,n ( Φ p (η N p 1 ) + 1 N V N p ) ( Np 1 Φ p,n Φp (η ))] V N p D p,n + 1 N R N p,n with a predictable D p,n first order operator }{{} fluctuation term 2nd-order measure R N p,n }{{} bias-term

41 Stochastic perturbation model Stochastic perturbation model W η,n n := N [ η N n η n ] = 0 p n V N p D p,n + 1 N R N n Under some mixing condition on the limiting FK semigroups Φ p,n osc (D p,n (f )) Cte e (n p)α and E ( R N n (f ) m) Cte 2 m (2m)!/m! Uniform concentration estimates w.r.t. the time parameter

42 Particle free energy Multiplicative formulae Z N n = 0 p<n Taylor first order expansion η N p (G p ) = γ N n (1) N γ n (1) = x, y > 0 log y log x = log ( γ N n (1)/γ n (1) ) p<n (y x) x + t(y x) dt η p (G p ) = ( 0 p<n log η N p (G p ) log η p (G p ) ) = ( ( ) ) 0 p<n log η p (G p ) + 1 N Wp η,n (G p ) log η p (G p ) = 1 1 N 0 p<n 0 η p (G p ) + first order expansion [exercice] W η,n p (G p ) t N Wp η,n (G p ) dt

43 Feynman-Kac models Ineracting particle interpretations Concentration inequalities Current population models Particle free energy Genealogical tree models Backward particle models

44 Current population models Constants (c 1, c 2 ) related to (bias,variance), c universal constant Test funct. f n 1, (x 0, n 0, N 1). The probability of the event [ η N n η n ] (f ) c 1 N ( 1 + x + x ) + c 2 N x is greater than 1 e x. x = (x i ) 1 i d (, x] = d i=1 (, x i] cells in E n = R d. ( ) F n (x) = η n 1(,x] and Fn N (x) = ηn N ( ) 1(,x] The probability of the following event N F N n F n c d (x + 1) is greater than 1 e x.

45 Particle free energy models Constants (c 1, c 2 ) related to (bias,variance), c universal constant (x 0, n 0, N 1) Unbiased property E η n N (f n ) 0 p<n ηp N (G p ) = E f n (X n ) G p (X p ) 0 p<n For any ɛ {+1, 1}, the probability of the event ɛ n log ZN n c 1 Z n N ( 1 + x + x ) + c 2 N x is greater than 1 e x. note (0 ɛ 1 (1 e ɛ ) (e ɛ 1) 2ɛ) e ɛ zn z eɛ z N z 1 2ɛ

46 Genealogical tree models := η N n (in path space) Constants (c 1, c 2 ) related to (bias,variance), c universal constant f n test function f n 1, (x 0, n 0, N 1). The probability of the event [ η N n Q n ] (f ) c1 n + 1 N ( 1 + x + x ) + c2 (n + 1) N x is greater than 1 e x. F n = indicator fct. f n of cells in E n = ( ) R d0..., R dn The probability of the following event 0 p n d p sup ηn N (f n ) Q n (f n ) c (n + 1) f n F n is greater than 1 e x. N (x + 1)

47 Backward particle models Constants (c 1, c 2 ) related to (bias,variance), c universal constant. f n normalized additive functional with f p 1, (x 0, n 0, N 1) The probability of the event [ ] Q N 1 n Q n (fn ) c 1 N (1 + (x + x x)) + c 2 N(n + 1) f a,n normalized additive functional w.r.t. f p = 1 (,a], a R d = E n. The probability of the following event is greater than 1 e x. sup Q N n (f a,n ) Q n (f a,n ) c a R d d (x + 1) N

Mean field simulation for Monte Carlo integration. Part II : Feynman-Kac models. P. Del Moral

Mean field simulation for Monte Carlo integration Part II : Feynman-Kac models P. Del Moral INRIA Bordeaux & Inst. Maths. Bordeaux & CMAP Polytechnique Lectures, INLN CNRS & Nice Sophia Antipolis Univ.