Nested Sequential Monte Carlo Methods

Transcription

1 Nested Sequential Monte Carlo Methods Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön Linköping University, Sweden The University of Cambridge, UK Uppsala University, Sweden ICML 2015 Presented by: Qinliang Su Aug. 17, 2016

2 Outline 1 Introduction 2 Review of Sequential Monte Carlo (SMC) 3 Nested SMC 4 Nesting of Nested SMC 5 Experiments

3 C Introduction Review Christian of Sequential A. Naesseth Monte Carlo (SMC) August Nested 31, 2015SMC Nesting 4 of Nested SMC Experiments cle filters in high dimension Introduction (1) Known to perform poorly in high (say, d 10) dimensions. ex) Spatio-temporal model: g(y t x t ) = d k=1 g(y t,k x t,k ). X1 X2 X3 X4 X5 X6 Transition: x k x k 1 h(x k x k 1 ) Measurement: y k x k g(y k x k ) f(x t x t 1 ) is typically an extremely bad proposal distribution in HD. Goal: at each time step k, use some samples to approximate the posterior k p(x 1:k y 1:k ) h(x 1 )g(y 1 x 1 ) h(x t x t 1 )g(y t x t ) Does a better proposal distribution improve our result?, t=2 and then estimate the expectation E p [f (x 1:k )] as Eˆp [f (x 1:k )] = f (x 1:k )ˆp(x 1:k )dx 1:k

4 Introduction (2) Nested Sequential Monte Carlo Methods This paper is interested in the settings: i) x k is high-dimensional, i.e. x k R d with d 1; Christian A. Naesseth Linköping University, Linköping, Sweden ii) There are local dependcency strucuture among x 1:k, both Fredrik Lindsten The University spatially of Cambridge, and Cambridge, temporally United Kingdom article filters in high dimension sted SMC Christian A. Naesseth August 31, Thomas B. Schön Uppsala University, Uppsala, Sweden Two Known examples: to perform poorly in high (say, d 10) dimensions. ex) Spatio-temporal model: g(y t x t ) = d k=1 g(y t,k x t,k ). Abstract X1 X2 X3 X4 X5 X6 We propose nested sequential Monte Carlo (NSMC), a methodology to sample from sequences of probability distributions, even where the random variables are high-dimensional. NSMC generalises the SMC framework by requiring only approximate, properly weighted, samples from the SMC proposal distribution, while still resulting in a correct SMC algorithm. Furthermore, f(x t x t 1 NSMC ) is typically can in itself be anused extremely to pro- bad proposal distribution in HD. duce such properly weighted samples. Consequently, one NSMC sampler can be used to con- CHRISTIAN.A.NAESSETH@LIU.SE FREDRIK.LINDSTEN@ENG.CAM.AC.UK THOMAS.SCHON@IT.UU.SE k 1 k k + 1 Figure 1. Example of a spatio-temporal model where π k(x 1:k) is given by a k 2 3 undirected graphical model and x k R 2 3.

5 Outline 1 Introduction 2 Review of Sequential Monte Carlo (SMC) 3 Nested SMC 4 Nesting of Nested SMC 5 Experiments

6 High-level Descriptions of SMC Procedures (at time step k): i) Select one sequence from existing ones {X1:k 1 i }N i=1, Nested SMC Christian A. Naesseth August 31, denoted as X j 1:k 1 ii) The Draw bootstrap a samplefilter Xk i from proposal distribution q(x k X j 1:k 1 ), and set X1:k i = (X j The bootstrap particle 1:k 1 filter, X approximates k i ) as the new p(x t sample y 1:t ) by iii) Assign the new sample a weight Wk i = p(x 1:k i y 1:k ) N q(x 1:k i ), due to p N (x t y 1:t ) := W t i the mismatch between the proposal l W l δ X i t pdf t (x t ). and true pdf i=1 Weighting Resampling Propagation Weighting Resampling With the pair {X Resampling: 1:k i, W {(X k i }N t 1, i=1 i, the Wt 1)} i posterior N i=1 {( X is t 1, i approximated 1/N)} N i=1. as N Propagation: p(x i Xt i f(x t X t 1). i Wk i 1:k y 1:k ) N Weighting: Wt i = g(y t i=1 Xt). i i=1 W δ i X i (x i 1:k 1:k) k The key is how to choose {(Xt, i the Wt i proposal )} N distribution i=1

7 Bootstrap Particle Filter Proposal pdf is chosen to be the transition pdf, i.e., q(x k X j 1:k 1 ) = h(x k X j k 1 ) Under this proposal, the weight can be easily computed as W i k = f (X i k X j k 1 )g(y k X i k ) f (X i k X j k 1 ) = g(y k X i k ) Bootstrap PF performs poorly in high dimensions (d > 10) - Mismatch between the proposal and target distirbutions - Weight callapse, i.e. weights are dominated by only one weight Despite of its simplicty, h(x t x t 1 ) is a bad proposal distribution

8 Fully Adapted SMC (1) The proposal pdf is chosen to adapt to the target distribution Let π k (x 1:k )= 1 Z πk π k (x 1:k ) be the target pdf. The proposal pdf is designed as q k (x k x 1:k 1 )= 1 Z qk (x 1:k 1 ) q k(x k x 1:k 1 ), where q k (x k x 1:k 1 ) = π k(x 1:k ) π k 1 (x 1:k 1 ), [= g(y k x k )h(x k x k 1 )] Under this proposal pdf, the weight becomes W i k = Z q k (x 1:k 1 )

9 Fully Adapted SMC (2) Nested SMC Christian A. Naesseth August 31, D MRF Nested SMC implementation (I/III) Example: 2D MRF x 4 x 1 x 2 x 3 x 5 x 6 x 4,1 x 4,2 x 4,3 x 4,4 Target Optimal pdf: proposals π(x 1:k ) given = 1 by: Z πk φ 1 (x 1 ) k s=2 φ s(x s )Ψ s (x s 1, x s ) q t (x t x t 1 ) = φ t (x t )ψ t (x t 1, x t ) Proposal pdf: q { k (x k x k 1 ) = φ k (x k )Ψ d d }{ k (x k 1, x k ) d } Weight: = Z qk G t,k (x k 1 (x t,k )) = φm(x k t,k 1 k )Ψ k,(x x t,k k 1 ), x k )dxψ(x k t 1,k, x t,k ) k=1 k=2 k=1

10 Fully Adapted SMC (3) Algorithm 2: - Select one sequence from {X1:k 1 i }N i=1 with probability Z qk (X1:k 1 proportional to i ) N i=1 Zq k (X 1:k 1 i ), denoted as X j 1:k 1 ; - Draw X i k from q k( X j 1:k 1 ) and let X i 1:k = (X j 1:k 1, X i k ) Repeat above algorithm N times, we obtain samples {X1:k i }N i=1, and obtain π k (x 1:k ) 1 N δ N X i (x 1:k ) 1:k i=1 However, exact computation of Z qk and sampling from q k ( X j 1:k 1 ) are often impossible in practice.

11 Outline 1 Introduction 2 Review of Sequential Monte Carlo (SMC) 3 Nested SMC 4 Nesting of Nested SMC 5 Experiments

12 Nested SMC (1) Relaxing the exact computation and sampling requirements in fully adapted SMC... Definition 1 (Properly weighted sample) Let q(x) = 1 Z q q(x). A (random) pair (X, W ) R R + is properly weighted w.r.t. q( ) if E (X,W ) [f (X)W ] = Z q E q [f (x)] for all measurable functions f (x) The exact pair (X, W ) with X q(x) and W = Z q is a special case of properly weighted samples.

13 Nested SMC (2) (A1) Let Q be a class, and let q =Q(q, M). Assume that: i) The construction of q returns a member variable Ẑ q = q.getz(); ii) Q has a member function Simulate( ) which returns a (possibly random) variable X = q.simulate() iii) (X, Ẑq) is properly weighted w.r.t. q()

14 Nested SMC (3) Replace the exact Z q and X in fully adapted SMC with q.getz() and q.simulate() Algorithm 3: - Initialize q i = Q(q k ( X1:k 1 i ), M) for i = 1, 2,, N - Set Ẑ i q k = q i.getz() for i = 1, 2,, N - Repeat N times - Select one element from {1, 2, s,, N} with Ẑ probabilities s q k N ; denote the selected index as j s=1 Ẑ q s k - Draw X i k = qj.simulate() let X i 1:k = (X j 1:k 1, X i k )

15 Nested SMC (4) Theorem 1 Assume Q satisfies condition (A1). Then, the generated samples from nested SMC satisfies ( ) N 1/2 1 N f (X1:k i N ) π D k(f ) N (0, Σ M k (f )), D i=1 where means converges in distribution. As long as (q.getz, q.simulate()) is properly weighted, the expectation estimated from nested SMC converges to the exact expectation π k (f ) as N increases

16 Outline 1 Introduction 2 Review of Sequential Monte Carlo (SMC) 3 Nested SMC 4 Nesting of Nested SMC 5 Experiments

17 Nested SMC Nesting of SMC (1) Christian A. Naesseth August 31, D MRF Nested SMC implementation (I/III) x1 x2 x3 x4 x5 x6 x4,1 x4,2 x4,3 Optimal proposals given by: { Z πk can be estimated as: Ẑπ k = Ẑπ k 1 1 } N N i=1 Ẑ q i k, where Ẑ i q k = q i.getz(). Theorem 2 q t(x t x t 1) = φ t(x t)ψ t(x t 1, x t) { d d }{ d } = G t,k(x t,k) m(x t,k 1, x t,k) ψ(x t 1,k, x t,k) k=1 k=2 The pair (X1:k i, Ẑ π i k ) is properly weighted w.r.t. π k ( ), in which X1:k i is drawn with Algorithm 3. x4,4 Implication: using nested SMC, properly weighted samples w.r.t. 2D MRF π k ( ) can be obtained from the properly weighted samples w.r.t. 1D MRF q k ( ) k=1

18 Nested SMC Nesting of Nested (2) Christian A. Naesseth August 31, D MRF Nested SMC implementation (I/III) x1 x2 x3 x4 x5 x6 x4,1 Nested Sequential x4,2 Monte Carlo Methods x4,3 x4,4 Christian A. Naesseth Optimal proposals given by: (q i Linköping.Simulate, q i University, Linköping, Sweden.GetZ) qt(xt xt 1) is= φt(xt)ψt(xt 1, properly xt) weighted w.r.t. 1D MRF q( ) Fredrik Lindsten { d d }{ d } FREDRIK.LINDSTEN@ENG.CAM.AC.UK The University of Cambridge, = Gt,k(xt,k) Cambridge, m(xt,k 1, United xt,k) Kingdomψ(xt 1,k, xt,k) k=1 k=2 k=1 (X1:k i, Ẑ π i k ) Thomas is properly B. Schön THOMAS.SCHON@IT.UU.SE weighted w.r.t. 2D MRF π( ) Uppsala University, Uppsala, Sweden (X1:k i, Ẑ π i k ) is properlyabstract weighted w.r.t. We propose nested sequential Monte Carlo 2D MRF π( ) (NSMC), a methodology to sample from sequences of probability distributions, even where the random variables are high-dimensional. Draw samples NSMCfrom generalises 3D the SMC MRF framework by requiring only approximate, properly weighted, samples from the SMC proposal distribution, while still resulting in a correct SMC algorithm. Furthermore, NSMC can in itself be used to produce such properly weighted samples. Consequently, one NSMC sampler can be used to construct an efficient high-dimensional proposal distribution for another NSMC sampler, and this k 1 k k Conclusion: One nested SMC sampler can be used as the proposal distribution for another nested SMC tarteting at higher dimensional distributions for some sequence of probability densities Figure 1. Example of a spatio-temporal model where πk(x1:k) is given by a k 2 3 undirected graphical model and xk R 2 3.

19 Outline 1 Introduction 2 Review of Sequential Monte Carlo (SMC) 3 Nested SMC 4 Nesting of Nested SMC 5 Experiments

20 Nested SMC Christian A. Naesseth August 31, Experiments (1) Particle filters in high dimension Known to perform poorly in high (say, d 10) dimensions. 1) Gaussian State Space Model ex) Spatio-temporal model: g(y t x t ) = d k=1 g(y t,k x t,k ). X 1 X 2 X 3 X 4 X 5 X 6 f(x t x t 1 ) is typically an extremely bad proposal Figure: Gaussian state space model in form of 2D MRF of size d t distribution in HD. Does a better proposal distribution improve our result? The transition and measurement pdfs are all Gaussian Two-level Nested SMC

21 Experiments (2) Nested Sequential Monte Carlo Methods d = 50 d = 100 d = ESS NSMC NSMC NSMC ST-PF ST-PF 100 ST-PF Bootstrap k k k ure 2. Median (over dimension) ESS (4) and 15 85% percentiles (shaded region). The results are based on 100 independent run GaussianFigure: MRF with dimension Mediand. effective sample size (ESS) and 15% 85% percentiles. N = 500 and M = 2d with 100 independent runs. x k x k 1, y k ) is not explicitly used. simulate data from this model for k = 1,..., 100 for ferent values of d = dim(x k ) {50, 100, 200}. ( The act filtering marginals are computed using the Kalman er. We compare with both theess(x ST-PF and k,l standard ) (bootap) E PF. e results are evaluated based on the effective sample size SS, see e.g. Fearnhead et al. (2010b)) defined as, tails of the model are given. The transition pro bility p(x k x k 1 ) is a localised Gaussian mixture the measurement ]) probability p(y k x k ) is t-distribu The model dimension 1 is d = Beskos et (2014a) report improvements for ST-PF over both the b strap σ PF and the block PF by Rebeschini & van H del (2015). k,l 2 We use N = M = 100 for both ST and NSMC (the special structure of this model imp that there is no significant computational overhead f [ (ˆx k,l µ k,l ) 2

22 C (the 2) Non-Gaussian special structure State Space of this Model model implies - The transition pdf p(x k x k 1 ) is Gaussian mixture - The measurement pdf p(y k x k ) is t-distribution e is no significant computational overhead from backward ) and the 30 p PF is 25 = re 3 we 15 e ESS (4), report improvements for ST-PF over both the bootand the block PF by Rebeschini & van Han- Experiments (3) 5). We use N = M = 100 for both ST-PF d accord- Carpenter 999). The the bootis close ESS 10 5 NSMC ST-PF Bootstrap 100 k Figure: Median ESS and 15% 85% percentiles.

23 Nested Sequential Monte Carlo Methods Introduction Review of Sequential Monte Carlo (SMC) Nested SMC Nesting of Nested SMC Experiments Experiments (4) Linköping, Sweden bridge, Cambridge, United Kingdom ppsala, Sweden Abstract 3) Spatio-Temporal Model-Drought Detection ted sequential Monte Carlo odology to sample from sebility distributions, even where iables are high-dimensional. s the SMC framework by reroximate, properly weighted, e SMC proposal distribution, g in a correct SMC algorithm. C can in itself be used to proly weighted samples. Conse- C sampler can be used to conhigh-dimensional proposal disther NSMC sampler, and this orithm can be done to an arbiallows us to consider complex onal models using SMC. We motivate the efficacy of our apfiltering problems with dimenof 100 to k 1 k k + 1 Figure 1. Example of a spatio-temporal model where πk(x1:k) is given by a k 2 3 undirected graphical model and xk R Hidden states: 0 (normal) or 1 (drought) at different locations and for some years sequence of probability densities π k(x 1:k) = Z 1 πk - Measurements: precipitation πk(x1:k), k 1, (2) with normalisation constants Z πk = π k(x 1:k)dx 1:k. Note that x 1:k := (x 1,..., x k) X k. The typical scenario that we consider is the well-known problem of inference in time series or state space models (Shumway & Stoffer, 2011; Cappé et al., 2005). Here the index k corresponds to time and we want to process some observations y 1:k in a

24 Nr of drought location Introduction Review of Sequential Monte Carlo (SMC) Nested SMC Experiments (5) Nesting of Nested SMC Experiments p( Nested Sequential Monte Carlo Methods 1920 Year 1930 p( p( North America region N 60 N Nr of drought locations Nr of drought locations N 40 N W W 0.4 p(x = 1) > 0.5 p(x = 1) > p(x = 1) > Year 1930 p(x = 1) > 0.9 p(x = 1) > 0.7 p(x = 1) > N Year North America region 60 N 40 N Sahel region 110 W N W America 1939 # of drought locations EstimateNorth of p(x k,i = 1) for locations of North America in 1939 of North Americawith in 1939 Figure 4. Top: Number of locations estimated p(x N W 140 W W 80 W North America 1939 for all sites over a span of 3 years. All results for N = W N N N N N N ornorth in America an abnormal state 1North (drought) America 1941 Measurem 110 W 80 W W 80 W 0.0