Mean field simulation for Monte Carlo integration. Part I : Intro. nonlinear Markov models (+links integro-diff eq.) P. Del Moral

Transcription

1 Mean field simulation for Monte Carlo integration Part I : Intro. nonlinear Markov models (+links integro-diff eq.) P. Del Moral INRIA Bordeaux & Inst. Maths. Bordeaux & CMAP Polytechnique Lectures, INLN CNRS & Nice Sophia Antipolis Univ Some hyper-refs Mean field simulation for Monte Carlo integration. Chapman & Hall - Maths & Stats [600p.] (May 2013). Feynman-Kac formulae, Genealogical & Interacting Particle Systems with appl., Springer [573p.] (2004) Concentration Inequalities for Mean Field Particle Models. Ann. Appl. Probab (2011). (+ Rio). Coalescent tree based functional representations for some Feynman-Kac particle models Annals of Applied Probability (2009) (+ Patras, Rubenthaler) Particle approximations of a class of branching distribution flows arising in multi-target tracking. SIAM Control. & Opt. (2011). (joint work with Caron, Doucet, Pace) On the stability & the approximation of branching distribution flows, with applications to nonlinear multiple target filtering. Stoch. Analysis and Appl. (2011). (joint work with Caron, Pace, Vo). More references on the websitehttp:// delmoral/index.html [+ Links]

2 Contents Some basic notation Markov chains Linear evolution equations Finite state space models Monte Carlo methods Links with linear integro-diff. eq. Nonlinear Markov models Nonlinear evolution equations Mean field particle models Graphical illustration Links with nonlinear integro-diff. eq. How & Why it works? Some interpretations A stochastic perturbation analysis Uniform concentration inequalities

3 Some basic notation Markov chains Nonlinear Markov models How & Why it works?

4 Lebesgue integral on a measurable state space E (µ, f ) =(measure, function) (M(E) B b (E)) µ(f ) = µ(dx) f (x) Delta-Dirac Measure at a E µ = δ a δ a (f ) = f (x) δ a (dx) = f (a) Normalization of a positive measure µ M + (E) (when µ(1) 0) µ(dx) := µ(dx)/µ(1) = probability P(E)

5 Lebesgue integral on a measurable state space E (µ, f ) =(measure, function) (M(E) B b (E)) µ(f ) = µ(dx) f (x) Delta-Dirac Measure at a E µ = δ a δ a (f ) = f (x) δ a (dx) = f (a) Normalization of a positive measure µ M + (E) (when µ(1) 0) µ(dx) := µ(dx)/µ(1) = probability P(E) Example µ = 10 Law(X ) µ(1) = 10 & µ = Law(X ) & µ(f ) = E(f (X ))

6 Examples E={1,...,d} Matrix-Vector notation f (1) µ := [µ(1),..., µ(d)] and f :=. f (d) µ(f ) := d µ(i) f (i) i=1

7 Examples E={1,...,d} Matrix-Vector notation f (1) µ := [µ(1),..., µ(d)] and f :=. f (d) µ(f ) := d µ(i) f (i) i=1 X = 1 i N δ X i spatial Poisson point process with intensity µ N = Poisson random variable with E(N) = µ(1) and X i iid µ E(X (f )) = E(E(X (f ) N)) = E(N µ(f )) = µ(1) µ(f ) = µ(f )

8 Q(x, dy) integral operator from E into E Two operator actions : f B b (E ) Q(f ) B b (E) and µ M(E) µq M(E ) with Q(f )(x) = [µq](dx ) = Q(x, dx )f (x ) µ(dx)q(x, dx ) ( [µq](f ) := µ[q(f )] ) and the composition (Q 1 Q 2 )(x, dx ) = Q 1 (x, dx )Q 2 (x, dx )

9 Q(x, dy) integral operator from E into E Two operator actions : f B b (E ) Q(f ) B b (E) and µ M(E) µq M(E ) with Q(f )(x) = [µq](dx ) = Q(x, dx )f (x ) µ(dx)q(x, dx ) ( [µq](f ) := µ[q(f )] ) and the composition (Q 1 Q 2 )(x, dx ) = Q 1 (x, dx )Q 2 (x, dx ) Notation (used in variance descriptions) Q([f Q(f )] 2 )(x) := Q(x, dy) [f (y) Q(f )(x)] 2 Q Markov operator x Q(x, dy) P(E) notation : M or K (Markov transitions-kernel/stochastic matrices)

10 Finite state spaces E = {1,..., d} and E = {1,..., d }: Action on the right f (1) f :=. f (d ) Matrix operation B b (E ) Q(f ) = Q(f )(1). Q(f )(d) B b (E) Q(f ) = Q(f )(1). Q(f )(d) = Q(1, 1)... Q(1, d )..... Q(d, 1)... Q(d, d ) f (1). f (d )

11 Finite state spaces E = {1,..., d} and E = {1,..., d }: Action on the left µ = [µ(1),..., µ(d)] M(E) µq = [(µq)(1),..., (µq)(d )] M(E ) Matrix operation µq = [µ(1),..., µ(d)] Q(1, 1)... Q(1, d )..... Q(d, 1)... Q(d, d )

12 Boltzmann-Gibbs transformation : G 0 s.t. µ(g) > 0 Ψ G (µ)(dx) = 1 µ(g) G(x) µ(dx)

13 Boltzmann-Gibbs transformation : G 0 s.t. µ(g) > 0 Ψ G (µ)(dx) = 1 µ(g) Bayes rule (with fixed observation y) G(x) µ(dx) µ(dx) = p(x)dx and G(x) = p(y x) Ψ G (µ)(dx) = 1 p(y) p(y x) p(x)dx = p(x y) dx

14 Boltzmann-Gibbs transformation : G 0 s.t. µ(g) > 0 Ψ G (µ)(dx) = 1 µ(g) Bayes rule (with fixed observation y) G(x) µ(dx) µ(dx) = p(x)dx and G(x) = p(y x) Restriction Ψ G (µ)(dx) = 1 p(y) p(y x) p(x)dx = p(x y) dx µ(dx) = P(X dx) = p(x)dx and G(x) = 1 A (x) Ψ G (µ)(dx) = 1 P(X A) 1 A(x) p(x)dx = P(X dx X A)

15 Boltzmann-Gibbs transformation : G 0 s.t. µ(g) > 0 Ψ G (µ)(dx) = 1 µ(g) G(x) µ(dx) Markov transport equation Ψ G (µ)(dy) = µ(dx)s µ (x, dy) Ψ G (µ) = µs µ Example 1 : (G 1) accept/reject/recycling/interacting jumps S µ (x, dy) = G(x)δ x (dy) + (1 G(x)) Ψ G (µ)(dy)

16 Boltzmann-Gibbs transformation : G 0 s.t. µ(g) > 0 Ψ G (µ)(dx) = 1 µ(g) G(x) µ(dx) Markov transport equation Ψ G (µ)(dy) = µ(dx)s µ (x, dy) Ψ G (µ) = µs µ Example 1 : (G 1) accept/reject/recycling/interacting jumps Note : S µ (x, dy) = G(x)δ x (dy) + (1 G(x)) Ψ G (µ)(dy) S 1 N 1 j N δ (X i, dy) X j = G(X i )δ X i (dy) + (1 G(X i )) 1 j N G(X j ) 1 k N G(X k ) δ X j (dy)

17 Other examples of transport equations Ψ G (µ) = µs µ Example 2 : ɛ µ s.t. ɛ µ G 1 µ a.e. S µ (x, dy) = ɛ µ G(x) δ x (dy) + (1 ɛ µ G(x)) Ψ G (µ)(dy) Example 3 : a s.t. G > a S µ (x, dy) = Example 4 : G a µ(g) δ x(dy) + ( 1 a ) µ(g) Ψ G a (µ)(dy) S µ (x, dy) = α(x) δ x (dy) + (1 α(x)) Ψ [G G(x)]+ (µ)(dy) with the acceptance rate α(x) = µ[g G(x)]/µ(G)

18 Some basic notation Markov chains Linear evolution equations Finite state space models Monte Carlo methods Links with linear integro-diff. eq. Nonlinear Markov models How & Why it works?

19 Example: Markov chain X n+1 = F n (X n, W n ) η 0 = Law(X 0 ) and M n (x n 1, dx n ) = P (X n dx n X n 1 = x n 1 ) η 0 M 1 = Law(X 1 ) and M n (f )(x) = E(f (X n ) X n 1 = x)

20 Example: Markov chain X n+1 = F n (X n, W n ) η 0 = Law(X 0 ) and M n (x n 1, dx n ) = P (X n dx n X n 1 = x n 1 ) η 0 M 1 = Law(X 1 ) and M n (f )(x) = E(f (X n ) X n 1 = x) as well as (M 1... M n )(x 0, dx n ) = P(X n dx n X 0 = x 0 ) η 0 M 1... M n = Law(X n ) E={1,...,d} Matrix-Vector notation

21 Markov chain models X n+1 = F n (X n, W n ) Linear evolution equations Law(X n ) := η n η n+1 = η n M n+1 E={1,...,d} simple matrix computations M n+1 (1, 1)... M n+1 (1, d) η n+1 = [η n (1),..., η n (d)]..... M n+1 (d, 1)... M n+1 (d, d)

22 Markov chain models X n+1 = F n (X n, W n ) Linear evolution equations Law(X n ) := η n η n+1 = η n M n+1 E={1,...,d} simple matrix computations M n+1 (1, 1)... M n+1 (1, d) η n+1 = [η n (1),..., η n (d)]..... M n+1 (d, 1)... M n+1 (d, d) Note : Same equations for X n E n = {1,..., d n } with M n+1 R (dn dn+1) +

23 (Crude) Monte Carlo methods : Markov chain X n+1 = F n (X n, W n ) E n Linear evolution equations Law(X n ) := η n η n+1 = η n M n+1 N iid copies/samples X i n+1 = F n(x i n, Wi n ), 1 i N ηn N := 1 δ N X i n N η n 1 i N ηn N (f ) = 1 f (X i N n) N η n (f ) = E(f (X n )) 1 i N

24 (Crude) Monte Carlo methods : Markov chain X n E n with transitions M n Path space models P N n := 1 N Historical process 1 i N X n = (X 0,..., X n ) E n+1 = δ (X i 0,...,X i n ) N P n = Law(X 0,..., X n ) 0 p n E p with Markov transition M n Same model as before and η N n := 1 N P n+1 = η n+1 = η n M n+1 1 i N with N iid copies of the historical process δ X i n = P N n N P n = η n X i n = (X i 0,..., X i n)

25 Diffusion type & discrete generation models (d=1) Markov chain with Gaussian perturbations X n+1 = F n (X n, W n ) = X n + a n (X n ) + b n (X n ) W n with the Gaussian random variable P(W n dw) = 1 exp ( w 2 ) 2π 2 Diffusion limits when (t n+1 t n ) = 0 & (a n, b n, X n ) = ( a t n, b t n, X t n ) dx t = a t(x t )dt + b t(x t )dw t

26 Diffusion type & discrete generation models (d=1) Diffusion model dx t = a t(x t )dt + b t(x t )dw t Taylor expansion (Ito s formula) f (X t + dx t ) f (X t ) = f (X t ) dx t f (X t ) dx t dx t +...

27 Diffusion type & discrete generation models (d=1) Diffusion model Taylor expansion (Ito s formula) dx t = a t(x t )dt + b t(x t )dw t f (X t + dx t ) f (X t ) = f (X t ) dx t f (X t ) dx t dx t +... with the infinitesimal generator dw t= dt = L t (f )(X t ) dt + f (X t ) b t(x t )dw t }{{} Martingale (E(.) = 0) L t (f ) = a t f (b t) 2 2 f Weak parabolic PDE η t = Law(X t ) d dt η t(f ) = η t (L t (f ))

28 Two-steps jump-diffusion type model (d=1) X n X n+1/2 = F n (X n, W n ) X n+1 = ɛ n+1 X n+1/2 + (1 ɛ n+1 ) Y n+1 ( ) ɛ n+1 e Vn(X n+1/2) δ e Vn(X n+1/2) δ 0 Y n+1 P n (X n+1/2, dy)

29 Two-steps jump-diffusion type model (d=1) X n X n+1/2 = F n (X n, W n ) X n+1 = ɛ n+1 X n+1/2 + (1 ɛ n+1 ) Y n+1 ( ) ɛ n+1 e Vn(X n+1/2) δ e Vn(X n+1/2) δ 0 Y n+1 P n (X n+1/2, dy) Jump-diffusion limits (a n, b n, V n, P n, X n ) = ( ) a t n, b t n, V t n, P t n, X t n between jumps = diffusion as before, and jumps times T n+1 = inf { t T n : t T n V s (X s ) ds e n } e n iid expo(λ = 1)

30 Two-steps jump-diffusion type model (d=1) with the diffusion part and the jump part M n+1 J n+1 X n X n+1/2 X n+1 M n+1 (f )(x) f (x) + L diff t (f )(x) J n+1 (f )(x) = e V tn (x) f (x) + (1 e V tn (x) ) P t n (f )(x) = f (x) + V t n (x) [ P t n (f )(x) f (x) ] }{{} =L jump t (f )(x)

31 Two-steps jump-diffusion type model (d=1) with the diffusion part and the jump part M n+1 J n+1 X n X n+1/2 X n+1 M n+1 (f )(x) f (x) + L diff t (f )(x) J n+1 (f )(x) = e V tn (x) f (x) + (1 e V tn (x) ) P t n (f )(x) = f (x) + V t n (x) [ P t n (f )(x) f (x) ] }{{} =L jump t (f )(x)

32 Two-steps jump-diffusion type model (d=1) with the diffusion part and the jump part M n+1 J n+1 X n X n+1/2 X n+1 M n+1 (f )(x) f (x) + L diff t (f )(x) J n+1 (f )(x) = e V tn (x) f (x) + (1 e V tn (x) ) P t n (f )(x) = f (x) + V t n (x) [ P t n (f )(x) f (x) ] }{{} =L jump t (f )(x) [ MJ = (I + L diff ) (I + L jump ) I + (L diff + L jump ) ]

33 Two-steps jump-diffusion type model (d=1) with the diffusion part and the jump part M n+1 J n+1 X n X n+1/2 X n+1 M n+1 (f )(x) f (x) + L diff t (f )(x) J n+1 (f )(x) = e V tn (x) f (x) + (1 e V tn (x) ) P t n (f )(x) = f (x) + V t n (x) [ P t n (f )(x) f (x) ] }{{} =L jump t (f )(x) [ MJ = (I + L diff ) (I + L jump ) I + (L diff + L jump ) ] = Weak integro-differential equation with the infinitesimal generator η t = Law(X t ) d dt η t(f ) = η t (L t (f )) L t (f )(x) = a t(x) f (x)+ 1 2 (b t(x)) 2 2 f (x)+v t (x) (f (y) f (x)) P t(x, dy)

34 Some basic notation Markov chains Nonlinear Markov models Nonlinear evolution equations Mean field particle models Graphical illustration Links with nonlinear integro-diff. eq. How & Why it works?

35 Mean field models : Markov chain X n+1 = F n (X n, η n, W n ) Nonlinear evolution equations Law(X n ) := η n η n+1 = Φ n+1 (η n ) := η n K n+1,ηn with Markov transitions K n,η indexed by the symplex η S (d 1) R d + K n+1,ηn (x n, dx n+1 ) = P (X n+1 dx n+1 X n = x n )

36 Mean field models : Markov chain X n+1 = F n (X n, η n, W n ) Nonlinear evolution equations Law(X n ) := η n η n+1 = Φ n+1 (η n ) := η n K n+1,ηn with Markov transitions K n,η indexed by the symplex η S (d 1) R d + K n+1,ηn (x n, dx n+1 ) = P (X n+1 dx n+1 X n = x n ) When E = {1,..., d} simple matrix computations K n+1,ηn (1, 1)... K n+1,ηn (1, d) η n+1 = [η n (1),..., η n (d)]..... K n+1,ηn (d, 1)... K n+1,ηn (d, d)

37 Example 1 : Markov chain X n+1 = F n (X n, η n, W n ) E := { 1, 1} McKean s two velocities of gases : F n (X n, η n, W n ) = X n 1 Wn η n(1) + ( X n ) 1 Wn>η n(1) X n+1 = ɛ n X n with the Bernoulli random variable P(ɛ n = 1) = 1 P(ɛ n = 1) = η n (1) Nonlinear equation on the symplex S (0) = [0, 1] : η n+1 (1) = η n (1) 2 + η n ( 1) 2 = η n (1) 2 + (1 η n (1)) 2

38 Example 1 : Markov chain X n+1 = F n (X n, η n, W n ) E := { 1, 1} McKean s two velocities of gases : F n (X n, η n, W n ) = X n 1 Wn η n(1) + ( X n ) 1 Wn>η n(1) X n+1 = ɛ n X n with the Bernoulli random variable P(ɛ n = 1) = 1 P(ɛ n = 1) = η n (1) Nonlinear equation on the symplex S (0) = [0, 1] : η n+1 (1) = η n (1) 2 + η n ( 1) 2 = η n (1) 2 + (1 η n (1)) 2 n 1/2 if η 0 (1) {0, 1}

39 Example 2 : G n (i)( ex. = e Vn(i) ) ]0, 1] & (M n+1 (i, j)) i,j=1,...,d Markov transition K n+1,ηn (i, j) = G n (i) M n+1 (i, j)+(1 G n (i)) k η n (k)g n (k) l η n(l)g n (l) M n+1(k, j)

40 Example 2 : G n (i)( ex. = e Vn(i) ) ]0, 1] & (M n+1 (i, j)) i,j=1,...,d Markov transition K n+1,ηn (i, j) = G n (i) M n+1 (i, j)+(1 G n (i)) k η n (k)g n (k) l η n(l)g n (l) M n+1(k, j) k η n(k) K n+1,ηn (k, j) = k η n(k)g n (k) M n+1 (k, j) + [1 l η n(l)g n (l)] k = k η n(k)g n(k) l ηn(l)gn(l) M n+1(k, j) η n(k)g n(k) l ηn(l)gn(l) M n+1(k, j)

41 Example 2 : G n (i)( ex. = e Vn(i) ) ]0, 1] & (M n+1 (i, j)) i,j=1,...,d Markov transition K n+1,ηn (i, j) = G n (i) M n+1 (i, j)+(1 G n (i)) k η n (k)g n (k) l η n(l)g n (l) M n+1(k, j) k η n(k) K n+1,ηn (k, j) = k η n(k)g n (k) M n+1 (k, j) + [1 l η n(l)g n (l)] k = k η n(k)g n(k) l ηn(l)gn(l) M n+1(k, j) k η n(k)g n (k)m n+1 (k, j) η n(k)g n(k) l ηn(l)gn(l) M n+1(k, j)

42 Example 2 : G n (i)( ex. = e Vn(i) ) ]0, 1] & (M n+1 (i, j)) i,j=1,...,d Markov transition K n+1,ηn (i, j) = G n (i) M n+1 (i, j)+(1 G n (i)) k η n (k)g n (k) l η n(l)g n (l) M n+1(k, j) k η n(k) K n+1,ηn (k, j) = k η n(k)g n (k) M n+1 (k, j) + [1 l η n(l)g n (l)] k = k η n(k)g n(k) l ηn(l)gn(l) M n+1(k, j) k η n(k)g n (k)m n+1 (k, j) η n(k)g n(k) l ηn(l)gn(l) M n+1(k, j) G n (1) M n+1 (1, 1)... M n+1 (1, d) η n+1 η n G n (d) M n+1 (d, 1)... M n+1 (d, d) }{{}}{{} Multiplication Markov transport

43 Example 2 : G n (i)( ex. = e Vn(i) ) ]0, 1] & (M n+1 (i, j)) i,j=1,...,d Markov transition η n+1 = [Ψ Gn (η n )(1),..., Ψ Gn (η n )(d)] M n+1 (1, 1)... M n+1 (1, d)..... M n+1 (d, 1)... M n+1 (d, d) with the Boltzmann-Gibbs transformation Ψ Gn : η M(E) Ψ Gn (η) M(E) defined by Ψ Gn (η)(i) = 1 η n (G n ) G n(i) η(i) Nonlinear evolution eq. on the (d 1)-symplex η n S (d 1) R d + η n+1 = Φ n+1 (η n ) := Ψ Gn (η n )M n+1

44 Example 2 : G n (i)( ex. = e Vn(i) ) ]0, 1] & (M n+1 (i, j)) i,j=1,...,d Markov transition η n+1 = [Ψ Gn (η n )(1),..., Ψ Gn (η n )(d)] M n+1 (1, 1)... M n+1 (1, d)..... M n+1 (d, 1)... M n+1 (d, d) with the Boltzmann-Gibbs transformation Ψ Gn : η M(E) Ψ Gn (η) M(E) defined by Ψ Gn (η)(i) = 1 η n (G n ) G n(i) η(i) Nonlinear evolution eq. on the (d 1)-symplex η n S (d 1) R d + η n+1 = Φ n+1 (η n ) := Ψ Gn (η n )M n+1 Feynman-Kac models

45 Mean field models : Markov X n+1 = F n (X n, η n, W n ) E = R d Nonlinear evolution equations Law(X n ) := η n η n+1 = Φ n+1 (η n ) := η n K n+1,ηn Mean field particle-simulation models: N almost iid copies/samples X i n+1 = F n(x i n, ηn n, Wi n ), 1 i N with the occupation measure of the system at time n : η N n := 1 N 1 i N δ X i n N η n

46 Mean field models : Markov X n+1 = F n (X n, η n, W n ) E = R d Nonlinear evolution equations Law(X n ) := η n η n+1 = Φ n+1 (η n ) := η n K n+1,ηn Mean field particle-simulation models: N almost iid copies/samples X i n+1 = F n(x i n, ηn n, Wi n ), 1 i N with the occupation measure of the system at time n : η N n := 1 N 1 i N δ X i n N η n [by induction on n] ηn+1 N = 1 δ N X i n+1 with Xn+1 i F n (Xn, i η n, Wn) i ηn+1 N N η n+1 1 i N

47 An abstract mathematical model Nonlinear evolution equation η n+1 = Φ n+1 (η n ) := η n K n+1,ηn Nonlinear Markov model interpretation (not unique) K n+1,ηn (x n, dx n+1 ) = P (X n+1 dx n+1 X n = x n ) with Law(X n ) := η n Mean field particle-simulation model N X i n X i n+1 = r.v. with law K n+1,η N n (X i n, dx) where η N n = 1 N i=1 δ X i n

48 Local sampling errors Nonlinear evolution equation Mean field particle model η n+1 = Φ n+1 (η n ) V N n+1 := N ( η N n+1 Φ n+1 ( η N n )) η N n+1 = Φ n+1 ( η N n ) + 1 N V N n+1 Note f : osc(f ) := sup x,y (f (x) f (y)) 1 E ( ηn+1(f N ) ηn N ) N E ([ ηn+1 N ( )] Φ n+1 η N n (f ) 2 ηn N ) = ηn N ( ) K n+1,η N n (f ) = Φ n+1 η N n (f ) [ = ηn N [ K n+1,η N n f Kn+1,η N n (f ) ] ] 2 1

49 McKean measures : Markov chain X n E n with transitions K n,ηn 1 P N n := 1 N 1 i N with the McKean measures P n (d(x 0,..., x n )) = η 0 (dx 0 ) δ (X i 0,...,X i n ) N P n = Law(X 0,..., X n ) 1 p n K p,ηp 1 (x p 1, dx p )

50 McKean measures : Markov chain X n E n with transitions K n,ηn 1 P N n := 1 N 1 i N with the McKean measures P n (d(x 0,..., x n )) = η 0 (dx 0 ) δ (X i 0,...,X i n ) N P n = Law(X 0,..., X n ) 1 p n K p,ηp 1 (x p 1, dx p ) Historical process X n = (X 0,..., X n ) E n+1 = 0 p n E p with Markov transition K n,ηn 1

51 McKean measures : Markov chain X n E n with transitions K n,ηn 1 P N n := 1 N 1 i N with the McKean measures P n (d(x 0,..., x n )) = η 0 (dx 0 ) δ (X i 0,...,X i n ) N P n = Law(X 0,..., X n ) 1 p n K p,ηp 1 (x p 1, dx p ) Historical process X n = (X 0,..., X n ) E n+1 = 0 p n E p with Markov transition K n,ηn 1 Same mathematical model as before and η N n := 1 N P n+1 = η n+1 = η n K n+1,ηn 1 i N δ X i n = P N n N P n = η n with almost N iid copies of the historical process X i n = (X i 0,..., X i n).

52 Example 1 : Markov chain X n+1 = F n (X n, η n, W n ) E = R d=1 McKean-Vlasov diffusion type & discrete generation models X n+1 = F n (X n, η n, W n ) = a n (X n, η n ) + b n (X n, η n ) W n with the Gaussian random variable P(W n dw) = 1 exp ( w 2 ) 2π 2

53 Example 1 : Markov chain X n+1 = F n (X n, η n, W n ) E = R d=1 McKean-Vlasov diffusion type & discrete generation models X n+1 = F n (X n, η n, W n ) = a n (X n, η n ) + b n (X n, η n ) W n with the Gaussian random variable P(W n dw) = 1 exp ( w 2 ) 2π 2 Two-steps jump type model X n X n+1/2 = F n (X n, η n, W n ) X n+1 = ɛ n+1 X n+1/2 + (1 ɛ n+1 ) Y n+1 with ( ) ɛ n+1 e Vη (Xn+1/2) n+1/2 δ e Vη (X n+1/2 n+1/2) δ 0 Y n+1 P ηn+1/2 (X n+1/2, dy)

54 Example 1 : Markov chain X n+1 = F n (X n, η n, W n ) R d=1 McKean-Vlasov diffusion type & discrete generation models a n (X n, η n ) = η n (dy) a n(x n, y) & b n (X n, η n ) = η n (dy) b n(x n, y)

55 Example 1 : Markov chain X n+1 = F n (X n, η n, W n ) R d=1 McKean-Vlasov diffusion type & discrete generation models a n (X n, η n ) = η n (dy) a n(x n, y) & b n (X n, η n ) = η n (dy) b n(x n, y) X i n+1 = F n (X i n, η N n, W i n) = 1 N N a n(x n, i Xn) j + 1 N j=1 N b n(x n, i Xn) j Wn i j=1

56 Example 1 : Markov chain X n+1 = F n (X n, η n, W n ) R d=1 McKean-Vlasov diffusion type & discrete generation models a n (X n, η n ) = η n (dy) a n(x n, y) & b n (X n, η n ) = η n (dy) b n(x n, y) X i n+1 = F n (X i n, η N n, W i n) = 1 N N a n(x n, i Xn) j + 1 N j=1 N b n(x n, i Xn) j Wn i In the same vein : Two-steps mean field jump type model Xn i Xn+1/2 i = F n(xn, i ηn N, Wn) i Xn+1 i = ɛ i n+1 Xn+1/2 i +( 1 ɛ i n+1) Y i n+1 j=1 with ( ɛ i n+1 e V η n+1/2 N (X i n+1/2 ) δ e V η n+1/2 N (X i )) n+1/2 δ 0 Yn+1 i P η N (X i n+1/2 n+1/2, dy)

57 Example 2 : Back to the multiplication-transport model η n+1 = Ψ Gn (η n )M n+1 Nonlinear transport equation Ψ Gn (µ) = µs n,µ with S n,µ (x,.) = G n (x)δ x + (1 G n (x)) Ψ Gn (µ)

58 Example 2 : Back to the multiplication-transport model η n+1 = Ψ Gn (η n )M n+1 Nonlinear transport equation Ψ Gn (µ) = µs n,µ with S n,µ (x,.) = G n (x)δ x + (1 G n (x)) Ψ Gn (µ) Nonlinear Markov chain model η n+1 = Φ n+1 (η n ) := η n K n+1,ηn with K n+1,ηn := S n,ηn M n+1

59 Example 2 : Back to the multiplication-transport model η n+1 = Ψ Gn (η n )M n+1 Nonlinear transport equation Ψ Gn (µ) = µs n,µ with S n,µ (x,.) = G n (x)δ x + (1 G n (x)) Ψ Gn (µ) Nonlinear Markov chain model η n+1 = Φ n+1 (η n ) := η n K n+1,ηn with K n+1,ηn := S n,ηn M n+1 Mean field particle model = Genetic type interacting jump model X i n S n,η N n X n i M n+1 }{{} K n+1,η N n i Xn+1 with S n,η N n (X i n, dy) = G n (X i n)δ X i n (dy)+(1 G(X i n)) 1 j N G n (X j n) 1 k N G n(x k n ) δ X j n (dy)

60 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

61 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

62 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

63 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

64 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

65 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

66 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

67 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

68 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

69 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

70 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

71 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

72 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

73 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

74 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

75 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

76 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

77 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

78 Graphical illustration : η n η N n := 1 N 1 i N δ X i n

79 Some questions Law(Xn i acceptance) = η n? Convergence of the occupation measures of the genealogical trees? Use of the occupation measures of the complete ancestral tree? Convergence of the product of success proportions? Feynman-Kac integration models on path spaces

80 Some questions Law(Xn i acceptance) = η n? Convergence of the occupation measures of the genealogical trees? Use of the occupation measures of the complete ancestral tree? Convergence of the product of success proportions? Feynman-Kac integration models on path spaces Part II & Part III (Application areas) & Part V (Theoretical results)

81 Some questions Law(Xn i acceptance) = η n? Convergence of the occupation measures of the genealogical trees? Use of the occupation measures of the complete ancestral tree? Convergence of the product of success proportions? Feynman-Kac integration models on path spaces Part II & Part III (Application areas) & Part V (Theoretical results) Nonlinear evolution equations on M(E n ) γ n+1 = γ n Q n+1,γn

82 Some questions Law(Xn i acceptance) = η n? Convergence of the occupation measures of the genealogical trees? Use of the occupation measures of the complete ancestral tree? Convergence of the product of success proportions? Feynman-Kac integration models on path spaces Part II & Part III (Application areas) & Part V (Theoretical results) Nonlinear evolution equations on M(E n ) γ n+1 = γ n Q n+1,γn Part IV (Multiple object filtering)

83 Some questions Law(Xn i acceptance) = η n? Convergence of the occupation measures of the genealogical trees? Use of the occupation measures of the complete ancestral tree? Convergence of the product of success proportions? Feynman-Kac integration models on path spaces Part II & Part III (Application areas) & Part V (Theoretical results) Nonlinear evolution equations on M(E n ) γ n+1 = γ n Q n+1,γn Part IV (Multiple object filtering) Link with nonlinear integro-differential equations?

84 Nonlinear diffusion type models (d=1) X n+1 = F n (X n, η n, W n ) = X n + a n (X n, η n ) + b n (X n, η n ) W n with the Gaussian random variable P(W n dw) = 1 exp ( w 2 ) 2π 2

85 Nonlinear diffusion type models (d=1) X n+1 = F n (X n, η n, W n ) = X n + a n (X n, η n ) + b n (X n, η n ) W n with the Gaussian random variable P(W n dw) = 1 exp ( w 2 ) 2π 2 Diffusion limits when (t n+1 t n ) = 0 & (a n, b n, X n, η n ) = ( a t n, b t n, X t n, η t n ) dx t = a t(x t, η t)dt + b t(x t, η t)dw t

86 Nonlinear diffusion type models (d=1) X n+1 = F n (X n, η n, W n ) = X n + a n (X n, η n ) + b n (X n, η n ) W n with the Gaussian random variable P(W n dw) = 1 exp ( w 2 ) 2π 2 Diffusion limits when (t n+1 t n ) = 0 & (a n, b n, X n, η n ) = ( a t n, b t n, X t n, η t n ) dx t = a t(x t, η t)dt + b t(x t, η t)dw t Weak parabolic nonlinear PDE η t = Law(X t ) d dt η t(f ) = η t(l t,η t (f )) with the infinitesimal generator L t,η t (f )(x) = a t(x, η t) f (x) (b t(x, η t)) 2 2 f (x)

87 Two-steps jump-diffusion type model (d=1) X n X n+1/2 = F n (X n, η n, W n ) X n+1 = ɛ n+1 X n+1/2 + (1 ɛ n+1 ) Y n+1 ( ) ɛ n+1 e Vη (X n+1/2 n+1/2) δ e Vη (X n+1/2 n+1/2) δ 0 Y n+1 P ηn+1/2 (X n+1/2, dy)

88 Two-steps jump-diffusion type model (d=1) X n X n+1/2 = F n (X n, η n, W n ) X n+1 = ɛ n+1 X n+1/2 + (1 ɛ n+1 ) Y n+1 ( ) ɛ n+1 e Vη (X n+1/2 n+1/2) δ e Vη (X n+1/2 n+1/2) δ 0 Y n+1 P ηn+1/2 (X n+1/2, dy) Jump-diffusion limits ) (a n, b n, V n, P n, X n, η n ) = (a tn, b tn, V ηtn, P ηtn, X tn, η tn between jumps = nonlinear diffusion as before, and jumps times { t } T n+1 = inf t T n : V s (X s, η s) ds e n e n iid expo(λ = 1) T n

89 Two-steps jump-diffusion type model (d=1) X n X n+1/2 = F n (X n, η n, W n ) X n+1 = ɛ n+1 X n+1/2 + (1 ɛ n+1 ) Y n+1 ( ) ɛ n+1 e Vη (X n+1/2 n+1/2) δ e Vη (X n+1/2 n+1/2) δ 0 Y n+1 P ηn+1/2 (X n+1/2, dy) Jump-diffusion limits ) (a n, b n, V n, P n, X n, η n ) = (a tn, b tn, V ηtn, P ηtn, X tn, η tn between jumps = nonlinear diffusion as before, and jumps times { t } T n+1 = inf t T n : V s (X s, η s) ds e n e n iid expo(λ = 1) T n Weak integro-differential equation d dt η t(f ) = η t(l t,η t (f )) L t,η t (f )(x) = a t(x, η t) f (x) (b t(x, η t)) 2 2 f (x) + V η (x) (f (y) f (x)) P t η (x, dy) t

90 An abstract mathematical model Nonlinear evolution equation P(E) d dt η t(f ) = η t (L t,ηt (f )) Mean field particle-simulation model= Markov E N with generator L t (ϕ)(x 1,..., x N ) = L (i) (ϕ)(x 1,..., x i,..., x N ) t, 1 N 1 j N δ x j 1 i N

91 Local sampling errors Nonlinear evolution equation dη t (f ) = η t (L t,ηt (f )) dt

92 Local sampling errors Nonlinear evolution equation dη t (f ) = η t (L t,ηt (f )) dt Mean field particle model (useless in pratice) dη N t (f ) = η N t (L t,η N t (f )) dt + 1 N dm N t (f ) with a sequence of martingales M N t with predictable increasing process M N (f ) t = t 0 ηs N Γ Ls,η (f, f ) ds N s where Γ L stands for the carré du champ operator Γ L (f, f ) := L([f f (x)] 2 )(x) = L(f 2 )(x) 2f (x)l(f )(x)

93 Some basic notation Markov chains Nonlinear Markov models How & Why it works? Some interpretations A stochastic perturbation analysis Uniform concentration inequalities

94 How & Why it works? (Biology) Individual based models (IBM). (Physics) Mean field approximation schemes. (Computer Sciences) Stochastic adaptive grid approximation. (Stats) Universal acceptance-rejection-recycling sampling schemes. (Probab) Stochastic linearization/perturbation technique. η n = Φ n (η n 1 ) η N n = Φ n ( η N n 1 ) + 1 N V N n Theorem: (V N n ) n N (V n ) n independent centered Gaussian fields. Φ p,n (η p ) = η n stable sg No propagation of local errors = Uniform control w.r.t. the time horizon New concentration inequalities for (general) interacting processes

95 Stochastic perturbation analysis = Taylor expansion Gâteaux derivative of Φ n : P(E n 1 ) P(E n ) ɛ Φ n(η + ɛµ)(f ) ɛ=0 = µ[d η Φ n (f )] with the first order linear/integral operator d η Φ n : f B b (E n ) d η Φ n (f ) B b (E n 1 )

96 Stochastic perturbation analysis = Taylor expansion Gâteaux derivative of Φ n : P(E n 1 ) P(E n ) ɛ Φ n(η + ɛµ)(f ) ɛ=0 = µ[d η Φ n (f )] with the first order linear/integral operator d η Φ n : f B b (E n ) d η Φ n (f ) B b (E n 1 ) Φ n (µ) Φ n (η) (µ η) d η Φ n

97 Stochastic perturbation analysis = Taylor expansion Gâteaux derivative of Φ n : P(E n 1 ) P(E n ) ɛ Φ n(η + ɛµ)(f ) ɛ=0 = µ[d η Φ n (f )] with the first order linear/integral operator d η Φ n : f B b (E n ) d η Φ n (f ) B b (E n 1 ) Φ n (µ) Φ n (η) (µ η) d η Φ n Differential rule d η (Φ n+1 Φ n ) = d η Φ n d Φn(η)Φ n+1 Semigroup derivatives Φ p,n (µ) Φ p,n (η) (µ η) d η Φ p,n

98 Key idea = First order expansions Key telescoping decomposition η N n η n = n p=0 [ Φp,n (η N p ) Φ p,n ( Φp (η N p 1) )] First order expansion N [ Φp,n (η N p ) Φ p,n ( Φp (η N p 1 ))] = N [Φ p,n ( Φ p (η N p 1 ) + 1 N V N p ) ( Np 1 Φ p,n Φp (η ))] V N p D p,n + 1 N R N p,n with a predictable D p,n first order operator }{{} fluctuation term 2nd-order measure R N p,n }{{} bias-term

99 Stochastic perturbation model Stochastic perturbation model W η,n n := N [ η N n η n ] = 0 p n V N p D p,n + 1 N R N n Under some mixing condition on the limiting semigroups Φ p,n osc (D p,n (f )) Cte e (n p)α and E ( R N n (f ) m) Cte 2 m (2m)!/m! Uniform concentration estimates w.r.t. the time parameter

100 Uniform concentration inequalities Constants (c 1, c 2 ) related to (bias,variance), c finite constant Test functions/observables f n 1, (x 0, n 0, N 1). When E n = R d : F n (y) := η n ( 1(,y] ) and F N n (y) := η N n ( 1(,y] ) with y R d The probability of any of the following events is greater than 1 e x. η N n η n (fn ) c 1 N ( 1 + x + x ) + c 2 N x sup [ η N ] p η p (fp ) c x log (n + e)/n 0 p n F N n F n c d (x + 1)/N

101 Coalescent tree based expansions Weak propagation of chaos Taylor s type expansions: P (N,q) n = Law of the first q ancestral lines (q N) = Qn q + ( ) 1 1 N l d l P (q) n + O N m+1 with signed measures d l P (q) n 1 l m expressed in terms of coalescent trees.

102 Coalescent tree based expansions Weak propagation of chaos Taylor s type expansions: P (N,q) n = Law of the first q ancestral lines (q N) = Qn q + ( ) 1 1 N l d l P (q) n + O N m+1 with signed measures d l P (q) n 1 l m expressed in terms of coalescent trees. Romberg-Richardson interpolation: For any order l 1 1 m l ( 1) l m m! m l (l m)! P(mN,q) n = Q q n + O(1/N l )