Sequential Monte Carlo Methods in High Dimensions

Transcription

1 Sequential Monte Carlo Methods in High Dimensions Alexandros Beskos Statistical Science, UCL Oxford, 24th September 2012 Joint work with: Dan Crisan, Ajay Jasra, Nik Kantas, Andrew Stuart Imperial College, Singapore, UCL, Warwick Presentation based on submitted papers: On the stability of Sequential Monte Carlo methods in high dimensions (12) Error bounds and normalizing constants for Sequential Monte Carlo in high dimensions (12)

2 Background Perceived idea in DA/SMC areas that solving the full Bayesian problem for practical DA applications is unfeasible. Due to weight degeneracy happening very fast. So, standard practice is to apply Kalman-FIlter- type or variational-type methods using Gaussian approximations. Yet, there have been new attempts trying to confront weight degeneracy for SMC from DA community (e.g. van Leeuwen s talk, Chorin et al. (10)). Talk will show some efforts towards this direction from group from (mainly) SMC community.

3 Structure of the Talk Main Part: Cost of an SMC sampler in high dimensions. Secondary Part: A smoothing problem for the Navier-Stokes equation.

4 Outline 1 MCMC asymptotics as d 2 SMC Asymptotics as d : No Resampling 3 SMC Asymptotics as d : With Resampling 4 Navier Stokes Smoothing

5 Context N is number of particles; d is dimension of state space. Standard theory for SMC has looked at asymptotics as number of particles N. The context d is not as well studied or understood; there has been some work mainly in DA community: Snyder et al. (08), Bickel et al. (08), Bengtsson et al. (08), Quang et al. (11). Analytical results as d have provided powerful insights into the behaviour of MCMC algorithms: Roberts & Rosenthal (01). A detailed investigation should also benefit SMC methods.

6 Set-Up Assume iid target distribution in R d : Π(x 1:d ) = d j=1 { π(x j ) = exp d j=1 What is the cost of algorithms for large d? Is there some limit as d? } g(x j ) Can such results be used to tune and optimise algorithms?

7 Outline 1 MCMC asymptotics as d 2 SMC Asymptotics as d : No Resampling 3 SMC Asymptotics as d : With Resampling 4 Navier Stokes Smoothing

8 MCMC Algorithms Simulate ergodic Markov chain, invariant under Π(x 1:d ), up to equilibrium: x (1), x (2),..., E.g., Random-Walk Metropolis (RWM) will propose: x pr = x + h N(0, I d ) and x pr will be accepted with probability a d (x, x pr ) = 1 Π(x pr ) Π(x)

9 Limit for RWM Scale step-size to control acceptance probability: h = l 2 /d Indeed, we have a(l) = lim d E [ a d (x, x pr ) ] (0, 1) Sqeeze time at MCMC trajectory by 1/d: x (0) x (1) x (2) 1/d Trajectory of first co ordinate

10 Result + Utilisation Theorem (Roberts et al., 97) ([ t d ]) Continuous time process x 1 converges weakly to the solution of SDE: dx dt for speed function: = 1 2 s(l) (log π) (x) + s(l) dw dt s(l) = l 2 a(l), We should maximize s(l). Surprisingly, for "all" targets d j=1 π(x j): a(l opt ) = 0.234

11 Further Directions Further investigations involve: Independent x j with varying standard deviations (Bédard). Non-independent co-ordinates: change of measure from independent (Beskos et al., 09); short-length dependencies. Non-local algorithms: Hybrid Monte-Carlo (Beskos et al., 11).

12 Outline 1 MCMC asymptotics as d 2 SMC Asymptotics as d : No Resampling 3 SMC Asymptotics as d : With Resampling 4 Navier Stokes Smoothing

13 Context: Static We have target distribution: Π(x 1:d ) = d j=1 and will use particles, N, from: for some small φ 1 > 0. { π(x j ) = exp Π 1 (x 1:d ) = Π(x 1:d ) φ 1 d j=1 Direct Importance Sampling would require (Bickel, Snyder, etc.): N = O(κ d ), κ > 1. } g(x j )

14 Annealed Importance Sampling Neal (01); Chopin (02); Del Moral et al. (06). We work with the sequence of distributions: Π n (x) Π(x) φn, n = 1, 2,..., d where we have chosen φ n φ n 1 = 1 φ 1 d (so φ d 1). N particles start off with x (i) 0 Π 1 and evolve according to: d K n (x (i) n 1, dx n) = k n (x (i) n 1,j, dx n,j) j=1 such that π n k n = π n where π n (x j ) exp{ φ n g(x j )}.

15 Weights The unnormalised particle weights are as follows: W (i) n = W (i) t φ 1 n 1 Π n (x (i) n 1 ) Π n 1 (x (i) n 1 ) That is, after l d (t) = (1 φ 1 )/d steps: log W (i) t = 1 φ 1 d l d d (t) { (i) g(x n,j ) π n(g) } j=1 n=1 We will look at the stability of log W (i) t as d.

16 The Patricle Evolution Dynamics Co-ordinates evolve independently via k n (x n 1,j, dx n,j ). x 2 x 5 x 1 x 3 x 4 Trajectory of i th particle, j th co ordinate ( 1 φ ) / d 1 φ 1 φ 2 φ 3 φ 4 φ 5 1 One can think of a continuum of targets and densities: k s (x, dx ), π s (x j ) exp{ s g(x j )}.

17 Statement of One of Results Theorem: Under conditions, we have that as d : log W (i) t B σ 2 φ1 :t where B is a Brownian motion. The asymptotic variance is: t σφ 2 1 :t = (1 φ { 1) π s ĝ2 s k s (ĝs 2 ) } ds. φ 1 log W (i) 1 stabilise as d for fixed N.

18 Comments Consider ESS t = ( N i=1 W (i) t ) 2. N (i) i=1 (W t ) 2 One can also obtain, as d : ESS 1 [ N i=1 ex i ] 2 iid N, X i N(0, σ 2 φ1 i=1 e2x :1 ) i One can find that: lim E [ ESS 1] 1 + (N 1)e 3σ2 φ1 :1. d and that: ESS 1 N N,d exp{ σ 2 φ 1 :1 }.

19 Comments Recall that: σ 2 φ 1 :t = (1 φ 1) t φ 1 π s { ĝ2 s k s (ĝ 2 s ) } ds. Here, ĝ s is the solution to the Poisson equation: Note also that: g(x) π s (g) = ĝ s (x) k s (ĝ s )(x) π { ĝ 2 k(ĝ 2 ) } is the asymptotic variance in the standard CLT for geometric MCMC Markov chains.

20 Conditions for Theorem (A1) i. Minorisation condition uniformly in s: There exists set C, constant θ (0, 1) and probability law ν so that C is (1, θ, ν)-small w.r.t. k s. ii. Geometric Ergodicity uniformly in s: k s V (x) λv (x) + b I C (x), with λ < 1, b > 0 and C as above, for all s [φ 1, 1]. (A2) Controlled Perturbations of {k s }: k t k s V M t s.

21 (Very Rough) Sketch of Proof We look at the process: Z t,d = 1 φ 1 d l d (t) n=1 We have the decomposition: l d (t) n=1 { g(x (i) n,j ) π n(g) } = l d (t) n=1 { g(x (i) n,j ) π n(g) } { ĝn (x (i) n,j ) k n(ĝ n )(x (i) n 1,j ) } +R (i) d,t with the first term providing a Martingale Functional CLT and the second vanishing (when divided with d).

22 More Comments Proposition: We also have that: x (i) d,j π So, O(d) MCMC steps exactly enough to attain correct distribution. Overall cost O(N d 2 ).

23 Outline 1 MCMC asymptotics as d 2 SMC Asymptotics as d : No Resampling 3 SMC Asymptotics as d : With Resampling 4 Navier Stokes Smoothing

24 Dynamic Resampling? Analysis of algorithms under dynamic resampling is tough. Del Moral et al. (11) notice that as N dynamic (stochastic) resampling times coincide with deterministic ones with high probability. In our case, we can identify deterministic instances {t k (d)} matching the dynamic resampling times with probability at least 1 M N. We carry out the analysis doing resampling at {t k (d)}. We can also identify limits as d : t k (d) t k

25 Deterministic Resampling In particular, we have that (essentially): t 1 = inf{t [φ 1, 1] : e σ2 φ 1 :t < α}. t k = inf{t [t k 1, 1] : e σ2 t k 1 :t < α} We have defined here: σ 2 v:t = (1 φ 1 ) t v π s { ĝ2 s k s (ĝ 2 s ) } ds. No of resampling instances converges to m <.

26 Statement of Result Resampling forces dependence between particles and co-ordinates. Theorem: Under the stated conditions we have that, for instances s k (d) s k (t k 1, t k ). log W tk 1 (d):s k (d) N(0, σ 2 t k 1 :s k ) Overall, understanding of behavior of algorithm as d boils down to σ 2 v:t. Manifestation of effect of resampling.

27 A Comment on Proof Construct martingale under filtration {G j,d } d j=1 so that: G 0,d = σ(all particles, just after resampling) G 1,d = G 0,d σ(1st coordinate) G 2,d = G 1,d σ(2nd coordinate). Exploit conditional independence given G 0,d. Technique used in other applied probability applications (e.g. joint asymptotics in Monte Carlo + datasize).

28 Outline 1 MCMC asymptotics as d 2 SMC Asymptotics as d : No Resampling 3 SMC Asymptotics as d : With Resampling 4 Navier Stokes Smoothing

29 Navier Stokes Dynamics Consider NS dynamics on [0, L] [0, L], describing the evolution of the velocity v = v(x, t) of incompressible fluid: u t ν u + (u ) u + p = f u = 0 u(x, 0) = u 0 (x) with ν the viscosity, p the pressure, f the forcing. We assume periodic boundary conditions.

30 Spectral Domain Natural basis here is {ψ k } k Z 2 /{0} such that: where k = (k 2, k 1 ). So that we can expand: ψ k (x) = k k exp{ i 2π L k x} u(x) = k Z 2 /{0} for Fourier coefficients u k = u, ψ k. u k ψ k (x)

31 Bayesian Framework We observe u(x, t) with error (Eulerian case): Y s = ( u(x m, s δ) ) M + N(0, Σ) ; 1 s T m=1 We set a prior on u 0 : u 0 Π 0 = N(0, ( ) α ) We need to learn about the posterior: Π(u 0 Y ) L(Y u 0 ) Π 0 (u 0 ) for likelihood L(Y u 0 ) = e 1 2 T s=1 Ys Gs(u 0) 2 Σ, with "observation operator" u 0 G s (u 0 ) = ( u(x m, s δ) ) M m=1.

32 Non-Sparsity of Model Graph Π(u Y ) is a high-dimensional target. High dimensional posteriors arise frequently in Bayesian applications in statistics (e.g. Bayesian hierarchical modeling), and many times are successfully dealt with. This is mainly due to intrinsic conditional independencies in the model structure allowing for local computations when applying a Gibbs sampler. Such a structure is not present in our DA set-up, making it a non-standard challenging computational problem.

33 Learning from Posterior Law & Stuart (12) use a RWM-type MCMC algorithm. It proposes: u 0,pr = ρ u ρ 2 Z for noise Z Π 0, accepted will probability: 1 L(Y u 0,pr ) L(Y u 0 ) This is relevant for off-line setup, and was used to check robustness of practical approximate algorithms. Algorithm needed ρ 1 to give good acceptance probabilities, and could tackle some scenarios (state space made of 64 2 Fourier coefficients).

34 A Mixing Issue The proposal also writes as: with u 0,pr, ψ k = ρ u 0, ψ k + 1 ρ 2 Z, ψ k Re{ Z, ψ k } N(0, 1 2 ( 4π2 L 2 k 2 ) α ) Scale of noise is ideally tuned to the prior distribution, but badly tuned to the posterior. A-posteriori, "low" Fourier coefficients may have much smaller variances than a-priori, which explains ρ 1. But this destroys the mixing of "medium" Fourier coefficients, proposing very small steps relatively to their size.

35 An Improved SMC Sampler Better samplers could be build by sequentially assimilating data, and by sequentially adapting the scaling of Z. A Weight-Move Algorithm (Chopin, 02): 1 Assume collection of particles {u (i) 0 }N i=1 from Π(u 0 Y 1:s ). 2 Weight as W (i) = e 1 2 Y s+1 G s+1 (u (i) 0 ) 2 Σ 3 Resample; particles {u (i) 0 }N i=1 now represent Π(u 0 Y 1:(s+1) ). 4 Move particles according to kernel K s+1 (u (i) 0, du) invariant under Π(u 0 Y 1:(s+1) ). Importantly, current particle representation of Π(u 0 Y 1:(s+1) ) can be used to tune kernel K s+1 (u (i) 0, du).

36 Tuning of Kernel For instance, one can build K s+1 by now proposing: with u (i) 0,pr, ψ k = ρ u (i) 0, ψ k + 1 ρ 2 ξ Re{ξ} N(0, σ 2 ) and σ 2 the particle estimate of the marginal variance of Re{ u 0, ψ k } under the target Π(u 0 Y 1:(s+1) ). Chopin (02) also recommend Independence Samplers, effective in the presence of asymptotic normality. Still more time to do the numerics...

37 Discussion Described algorithm will need N realizations of NS dynamics from 0 to sδ, repeated for s = 1, 2,... T. Parallelisation over the N particles will be critical. SMC methods slowly making inroads in high-dimensional DA applications. Interaction between DA + SMC communities could provide platform for further advances.

38 References Bengtsson, Bickel and Li (08) Curse-of-dimensionality revisited: collapse of the PF in very large scale systems In Probability and statistics: essays in honor of David A. Freedman. Bickel, Li and Bengtsson (08) Sharp failure rates for the bootstrap particle filter in high dimensions In Pushing the limits of contemporary statistics: contributions in honor of Jayanta K. Ghosh. Chopin (02) A sequential particle filter method for static models Biometrika. Chorin, Morzfeld and Tu (10) Implicit particle filters for Data Assimilation Submitted. Del Moral, Doucet, and Jasra (11) On adaptive resampling procedures for sequential Monte Carlo methods To appear in Bernoulli.

39 References Del Moral, Doucet, and Jasra (06) Sequential Monte Carlo samplers J. R. Stat. Soc. Ser. B Stat. Methodol.. Law, Stuart (12) Evaluating Data Assimilation algorithms Submitted. Neal (01) Annealed importance sampling Stat. Comput.. Snyder, Bengtsson, Bickel and Anderson (08) Obstacles to high-dimensional particle filtering Monthly Weather Review van Leeuwen (10) Nonlinear Data Assimilation in geosciences: an extremely efficient particle filter Quart. J. of the R. Meteor. Soc.

40 References Beskos, Pillai, Roberts, Sanz-Serna and Stuart (11) Optimal tuning of the Hybrid Monte Carlo algorithm To appear in Bernoulli. Beskos, Roberts and Stuart (09) Optimal scalings for local Metropolis-Hastings chains on non-product targets in high dimensions Ann. Appl. Probab.. Quang, Musso, and Le Gland (10) An insight into the issue of dimensionality in particle filtering In Information Fusion (FUSION). Roberts, Gelman and Gilks (97) Weak convergence and optimal scaling of random walk Metropolis algorithms Ann. Appl. Probab.. Roberts and Rosenthal (01) Optimal scaling for various Metropolis-Hastings algorithms Statist. Sci.. Roberts and Rosenthal (98) Optimal scaling of discrete approximations to Langevin diffusions J. R. Stat. Soc. Ser. B Stat. Methodol..