An introduction to particle simulation of rare events

Transcription

1 An introduction to particle simulation of rare events P. Del Moral Centre INRIA de Bordeaux - Sud Ouest Workshop OPUS, 29 juin 2010, IHP, Paris P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 1 / 33

2 Outline 1 Introduction, motivations 2 The heuristic of particle methods 3 Particle Feynman-Kac models 4 Stochastic models and methods 5 Some convergence results 6 Some references P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 2 / 33

3 Summary 1 Introduction, motivations Some rare event problems Stochastic models Importance sampling techniques 2 The heuristic of particle methods 3 Particle Feynman-Kac models 4 Stochastic models and methods 5 Some convergence results 6 Some references P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 3 / 33

4 Rare event analysis Stochastic process X Rare event A : Proba(X A) & Law((X 0,..., X t ) X A) engineering/physics/biology/economics/finance : Finance : ruin/default probabilities, financial crashes, eco. crisis,... engineering : networks overload, breakdowns, engines failures,... Physics : climate models, directed polymer conformations, particle in absorbing medium, ground states of Schroedinger models. Statistics : tail probabilities, extreme random values. Combinatorics : Complex enumeration problems. Process strategies Rare event Control and prediction. X t = F t (X t 1, W t ) Law((W 0,..., W t ) X A) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 4 / 33

5 Stochastic models Only 2 Ingredients 1 Physical/biological/financial process : queuing network, portfolio, volatility process, stock market evolutions, interacting/exchange economic models... 1 Potential function (energy type, indicator, restrictions): critical level crossing, penalties functions, constraints subsets, performance levels, long range dependence... Objectives Estimation of the probability of the rare event. Computing the full distributions of the path of the process evolving in the critical regime prediction control. P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 5 / 33

6 Twisted Monte Carlo methods P(X A) = Find Q s.t. Q(A) 1 Elementary Monte Carlo estimate X i iid Q dp P(A) := dq (x) 1 A(x) Q(dx) P N (A) := 1 N Variance Drawbacks dp dq (x) 1 A(x) P(dx) 1 i N dp dq (X i ) 1 A (X i ) (ex. cross-entropy : Q {Q a, a A}) Huge variance if Q badly chosen optimal choice Q(dx) 1 A (x)p(dx). Need to twist the original reference process X. Stochastic evolution X = (X 0,..., X n ) dp n dq n (X ) := n k=0 p k (X k X k 1 ) q k (X k X k 1 ) degenerate product martingale w.r.t. n P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 6 / 33

7 Summary 1 Introduction, motivations 2 The heuristic of particle methods Flow of optimal twisted measures 5 Examples of flow of target measures 3 types of occupation measures Some equivalent stochastic algorithms 3 Particle Feynman-Kac models 4 Stochastic models and methods 5 Some convergence results 6 Some references P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 7 / 33

8 Flow of measure with increasing sampling complexity Rare event = cascade/series of intermediate less-rare events ( energy levels, physical gateways, index level crossings). Conditional probability flow = flow of optimal twisted measures n η n = Law(process series of n intermediate events) Rare event probabilities= Normalizing constants. Particle methods (Sampling a genealogical type default tree model % success or default) Explorations/Local search propositions of the solution space. Branching-Selection individuals critical regimes. P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 8 / 33

9 5 Examples of flow of target measures 1 η n = Law((X 0,..., X n ) 0 p n X p A p ) 2 η n (dx) e βnv (x) λ(dx) with β n 3 η n (dx) 1 An (x) λ(dx) with A n 4 η n = Law K π ((X 0,..., X n ) X n = x n ). 5 η n = Law( X hits B n X hits B n before A) 5 particle heuristics : 1 M n -local moves individual selections A n i.e. G n = 1 An 2 MCMC local moves η n = η n M n individual selections G n = e (βn+1 βn)v 3 MCMC local moves η n = η n M n individual selections G n = 1 An+1 4 M-local moves Selection G(x 1, x 2 ) = π(dx2)k(x2,dx1) π(dx 1)M(x 1,dx 2) 5 M n -local moves Selection-branching on upper/lower levels B n. P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 9 / 33

10 Interaction/branch. process 3 types of occupation measures (N = 3) = = = N Current population 1 N i=1 δ ξ i n i-th individual at time n N Genealogical tree model 1 N i=1 δ (ξ i 0,n,ξ i 1,n,...,ξi n,n) i-th ancestral line Complete genealogical tree model 1 N N i=1 δ (ξ i 0,ξi 1,...,ξi n) Empirical mean potentials [success % (G n = 1 A )] 1 N N G n (ξn) i i=1 P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 10 / 33

11 Equivalent Stochastic Algorithms : Genetic and evolutionary type algorithms. Spatial branching models, subset splitting techniques. Sequential Monte Carlo methods, population Monte Carlo models. Diffusion Monte Carlo (DMC), Quantum Monte Carlo (QMC),... Some botanical names = application domain areas : bootsrapping, selection, pruning-enrichment, reconfiguration, cloning, go with the winner, spawning, condensation, grouping, rejuvenations, harmony searches, biomimetics, splitting, [(Meta)Heuristics] 1996 Feynman-Kac mean field particle model P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 11 / 33

12 Summary 1 Introduction, motivations 2 The heuristic of particle methods 3 Particle Feynman-Kac models Some notation Asymptotic Analysis Some workout examples Sensitivity analysis 4 Stochastic models and methods 5 Some convergence results 6 Some references P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 12 / 33

13 Notation E measurable state space, P(E) proba. on E, B(E) bounded meas. functions (µ, f ) P(E) B(E) µ(f ) = µ(dx) f (x) M(x, dy) integral operator over E M(f )(x) = M(x, dy)f (y) [µm](dy) = µ(dx)m(x, dy) (= [µm](f ) = µ[m(f )] ) Bayes-Boltzmann-Gibbs transformation : G : E [0, [ with µ(g) > 0 Ψ G (µ)(dx) = 1 µ(g) G(x) µ(dx) E finite Matrix notations µ = [µ(1),..., µ(d)] and f = [f (1),..., f (d)] P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 13 / 33

14 Infinite Population N = Feynman-Kac measures (G n, M n ) Current population: η N n (f ) := 1 N N i=1 f (ξ i n) N η n (f ) := γ n(f ) γ n (1) with [Potential functions G n ] & [X n Markov chain transitions M n ] γ n (f ) := E f n (X n ) G p (X p ) 0 p<n Genealogical tree occupation measures: 1 N N i=1 δ (ξ i 0,n,ξ i 1,n,...,ξi n,n ) N dq n := 1 Z n 0 p<n G p (X p ) dp n Normalizing constants: 0 p<n ηn p (G p ) N Z n = γ n (1) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 14 / 33

15 Feynman-Kac models ALL of the above heuristics Example : G n = 1 An and X n Markov chain Normalizing constants: γ n (1) = P (n-th rare event) = P(X p A p, 0 p < n) n-th time marginals: η n = Law(X n n-th rare event)= Law(X n X p A p, 0 p < n) Path space measures: Q n = Law((X p ) 0 p n n-th rare event)= Law((X p ) 0 p n X p A p, p < n) Note : Including the n-th rare event level set Updated measures γ n, η n,, Q n with 0 p n P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 15 / 33

16 Another detailed static example Restriction of measures : A n (Ex.: A n = [a n, [ tails probab.) η n (dx) = 1 Z n 1 An (x) λ(dx) Feynman-Kac representation : η n (f ) := γ n(f ) γ n (1) with γ n (f ) := E f n (X n ) 1 Ap+1 (X p ) 0 p<n and Note : P (X n dx n X n 1 = x n 1 ) = M n (x n 1, dx n ) with η n = η n M n Z n = λ (A n ) = λ ( 1 An 1 An 1 ) λ ( 1 An 1 ) }{{} η n 1(1 An ) (A 0=E) Z n 1 = 0 p<n η p ( 1Ap+1 ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 16 / 33

17 Sensitivity analysis θ M n (x n 1, dx n ) = p θ n(x n 1, x n ) λ X n (dx n ) and G n (x) = G θ n (x) Parametric Feynman-Kac models : (G n, Q n, γ n, η n,...) (G θ n, Q θ n, γ θ n, η θ n,...) θ log γθ n(1) = Q θ n(f θ n ) with the additive functional and F θ n (x 0,..., x n ) := θ Q θ n(ϕ n ) = Q ( θ n Fn θ [ϕ n Q ) θ n(ϕ n )] 0 k n f θ k (x k 1, x k ) fk θ (x k 1, x p ) := log G p θ (x k ) + log pθ k (x k 1, x k ) θ θ P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 17 / 33

18 Additive functionals (with Doucet & Singh Hal-INRIA july 09) Path space models P n := Law(X 0,..., X n ) dq n := 1 G p (X p ) Z n dp n 0 p<n Hyp. : M n (x n 1, dx n ) = H n (x n 1, x n ) λ n (dx n ) Q n (d(x 0,..., x n )) = η n (dx n ) M n,ηn 1 (x n, dx n 1 )... M 1,η0 (x 1, dx 0 ) with the backward transitions : M p+1,η (x, dx ) G p (x ) H p+1 (x, x) η(dx ) Particle estimates Ancestral tree or the complete genealogical tree : Q N n (d(x 0,..., x n )) = η N n (dx n ) M n,η N n 1 (x n, dx n 1 )... M 1,η N 0 (x 1, dx 0 ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 18 / 33

19 Summary 1 Introduction, motivations 2 The heuristic of particle methods 3 Particle Feynman-Kac models 4 Stochastic models and methods McKean distribution models Mean field particle interpretations 5 Some convergence results 6 Some references P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 19 / 33

20 Nonlinear Markov models Flows of Feynman-Kac measures A two step correction prediction model η n Updating-correction Prediction/Markov transport η n = Ψ Gn (η n ) η n+1 = η n M n+1 Selection nonlinear transport formulae Ψ Gn (η n ) = η n S n,ηn with, for any ɛ n = ɛ n (η n ) [0, 1] s.t. ɛ n G n 1 S n,ηn (x,.) := ɛ n G n (x) δ x + (1 ɛ n G n (x)) Ψ Gn (η n ) η n+1 = η n (S n,ηn M n+1 ) := η n K n+1,ηn P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 20 / 33

21 Nonlinear Markov chains η n = Law(X n )=Perfect sampling algorithm Nonlinear transport formulae : η n+1 = η n K n+1,ηn with the collection of Markov probability transitions : K n+1,ηn = S n,ηn M n+1 Local transitions : P(X n dx n X n 1 ) = K n,η n 1 (X n 1, dx n ) with η n 1 = Law(X n 1 ) McKean measures (canonical process) : P n (d(x 0,..., x n )) = η 0 (dx 0 ) K 1,η0 (x 0, dx 1 )... K n,ηn 1 (x n 1, dx n ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 21 / 33

22 Mean field particle interpretations Sampling pb Mean field particle interpretations Markov Chain ξ n = (ξn, 1..., ξn N ) En N s.t. η N n := 1 δ N ξ i n N η n 1 i N Approximated local transitions ( 1 i N) ξn 1 i ξn i K n,η N n 1 (ξn 1, i dx n ) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 22 / 33

23 Schematic picture : ξ n E N n ξ n+1 E N n+1 Rationale : ξ 1 n. ξ i n. ξ N n K n+1,η N n ξ 1 n+1. ξ i n+1. ξ N n+1 η N n N η n = K n+1,η N n N K n+1,ηn = ξ i i.i.d. copies of X Particle McKean measures : 1 N N δ (ξ i 0,...,ξn i ) N Law(X 0,..., X n ) i=1 P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 23 / 33

24 Feynman-Kac models Genetic type stochastic algo. ξṇ 1 ξ 1 M n+1 n ξn+1 1. S ξṇ i n,η N n.. ξ n i ξn+1 i... ξn N ξn+1 N Acceptance/Rejection-Selection : [Geometric type clocks] ξ N n S n,η N n (ξ i n, dx) := ɛ n G n (ξn) i δ ξ i n (dx) + ( 1 ɛ n G n (ξn) ) i N G n(ξ j P n ) j=1 N k=1 Gn(ξk )δ ξ j (dx) n n Ex. : G n = 1 A G n (ξ i n) = 1 A (ξ i n) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 24 / 33

25 Some key advantages Mean field models=stochastic linearization/perturbation technique : η N n = η N n 1K n,η N n N W N n with Wn N W n Centered Gaussian Fields. η n = η n 1 K n,ηn 1 stable No propagation of local sampling errors = Uniform control w.r.t. the time horizon No burning, no need to study the stability of MCMC models. Stochastic adaptive grid approximation Nonlinear system positive-benefic interactions. Simple and natural sampling algorithm. P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 25 / 33

26 Summary 1 Introduction, motivations 2 The heuristic of particle methods 3 Particle Feynman-Kac models 4 Stochastic models and methods 5 Some convergence results Non asymptotic theorems Unnormalized models Additive functionals 6 Some references P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 26 / 33

27 Asympt. theo. TCL,PGD, PDM (n,n) fixed some examples : Empirical processes : sup sup N E( η N n η n p F n ) < n 0 N 1 Concentration inequalities uniform w.r.t. time : sup P( ηn N (f n ) η n (f n ) > ɛ) c exp (Nɛ 2 )/(2 σ 2 ) n 0 + Guionnet sup n 0 (IHP 01) & Ledoux sup Fn (JTP 00) & Rio AAP 10 Propagations of chaos exansions (+Patras,Rubenthaler (AAP 09)) : P N n,q := Law(ξ 1 n,..., ξ q n) η q n + 1 N 1 P n,q N k k P n,q + 1 N k+1 k+1 P N n,q with sup N 1 k+1 P N n,q tv < & sup n 0 1 P n,q tv c q 2. P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 27 / 33

28 Unnormalized models Un-bias particle approximation measures γn N (f n ) := ηn N (f n ) ηp N (G p ) 0 p<n Asymptotic theorms : fluctuations & deviations + A. Guionnet (AAP 99, SPA 98), + L. Miclo (SP 00), + D. Dawson (01) Non asymptotic theory : bias and variance estimates 1 Taylor type expansion (+Patras & Rubenthaler (AAP 09) : E ( (γ N n ) q (F ) ) =: Q N n,q(f ) = γ q n (F ) + 1 k (q 1)(n+1) 1 N k k Q n,q (F ) 2 Variance estimates (+Cerou & Guyader Hal-INRIA 08 & IPH 2010) : ( [γ N E n (f n ) γ n (f n ) ] ) 2 c n N γ n(1) 2 P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 28 / 33

29 Additive functionals F n (x 0,..., x n ) = 1 n p n f p (x p ) Bias estimate+uniform L p -bounds + variance (1/N 2 ) N E ( [(Q N n Q n )(F n )] 2) c 1/n +1/N }{{} bias term Uniform exponential concentration 1 log sup P N n 0 ( [Q N n Q n ](F n ) b ) + ɛ ɛ 2 /(2b 2 ) N P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 29 / 33

30 Summary 1 Introduction, motivations 2 The heuristic of particle methods 3 Particle Feynman-Kac models 4 Stochastic models and methods 5 Some convergence results 6 Some references P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 30 / 33

31 some references + http links Particle methods & Sequential Monte Carlo Feynman-Kac formulae. Genealogical and interacting particle systems, Springer (2004) Refs. (with L. Miclo) Branching and Interacting Particle Systems Approximations of Feynman-Kac Formulae. Sém. de Proba. (00). (with Doucet A. et Jasra A.) Sequential MC Samplers. JRSS B (2006). (with A. Doucet) On a class of genealogical and interacting Metropolis models. Sém. de Proba. 37 (2003). (with F. Patras, S. Rubenthaler) Coalescent tree based functional representations for some Feynman-Kac particle models, AAP (09) (with A. Doucet and S. S. Singh) A Backward Particle Interpretation of Feynman-Kac Formulae (HAL-INRIA 2009) to appear in M2AN (2010). (with F. Cerou et A. Guyader) A non asymptotic variance theorem for unnormalized Feynman-Kac particle models (HAL-INRIA 2008), to appear in the Annals of IHP (2010) P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 31 / 33

32 Rare event analysis Particle simulation of twisted measures (with J. Garnier) Genealogical Particle Analysis of Rare events. Annals of Applied Probab., 15-4 (2005). (with J. Garnier) Simulations of rare events in fiber optics by interacting particle systems. Optics Communications, Vol. 267 (2006). Branching processes (with P. Lezaud) Branching and interacting particle interpretation of rare event proba.. Stochastic Hybrid Systems : Theory and Safety Critical Applications, eds. H. Blom and J. Lygeros. Springer (2006). (with F. Cerou, Le Gland F., Lezaud P.) Genealogical Models in Entrance Times Rare Event Analysis, Alea, Vol. I, (2006). Proceedings Conf. RESIM 2006 (with A. M. Johansen et A. Doucet) Sequential Monte Carlo Samplers for Rare Events (with F. Cerou, A. Guyader, F. LeGland, P. Lezaud et H. Topart) Some recent improvements to importance splitting P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 32 / 33

33 Absorption models (with L. Miclo) Particle Approximations of Lyapunov Exponents Connected to Schrodinger Operators and Feynman-Kac Semigroups. ESAIM Probability & Statistics, vol. 7, pp (2003). (with A. Doucet) Particle Motions in Absorbing Medium with Hard and Soft Obstacles. Stochastic Analysis and Applications, vol. 22 (2004). + recent preprints (with A. Doucet and S. S. Singh) Forward Smoothing Using Sequential Monte Carlo Technical Report 638. Cambridge University Engineering Department (2009). (with A. Doucet et A. Jasra) On Adaptive Resampling Procedures for Sequential Monte Carlo Methods (HAL-INRIA 2008). P. Del Moral (INRIA) INRIA Bordeaux-Sud Ouest 33 / 33