Monte Carlo Approximation of Monte Carlo Filters

Transcription

1 Monte Carlo Approximation of Monte Carlo Filters Adam M. Johansen et al. Collaborators Include: Arnaud Doucet, Axel Finke, Anthony Lee, Nick Whiteley 7th January 2014

2 Context & Outline Filtering in State-Space Models: SIR Particle Filters [GSS93] Rao-Blackwellized Particle Filters [AD02, CL00] Block-Sampling Particle Filters [DBS06] Exact Approximation of Monte Carlo Algorithms: Particle MCMC [ADH10] SMC 2 [CJP13] Approximated Rao-Blackwellized Particle Filters [CSOL11] Exactly-approximated RBPFs [JWD12] Approximating the BSPF Local SMC [JD14]

3 Particle MCMC MCMC algorithms which employ SMC proposals [ADH10] SMC algorithm as a collection of RVs Values Weights Ancestral Lines Construct MCMC algorithms: With many auxiliary variables Exactly invariant for distribution on extended space Standard MCMC arguments justify strategy SMC 2 employs the same approach within an SMC setting. What else does this allow us to do with SMC?

4 Ancestral Trees t=1 t=2 t=3 a 1 3 =1 a 4 3 =3 a 1 2 =1 a 4 2 =3 b 2 3,1:3 =(1, 1, 2) b 4 3,1:3 =(3, 3, 4) b 6 3,1:3 =(4, 5, 6)

5 SMC Distributions We ll need the SMC Distribution: ( ) an L+2:n, x n L+1:n, k; x n L = ψ M n,l [ M i=1 ] q(x i n L+1 xn L ) n p=n L+2 [ M ( r(a p w p 1 ) q x i p x ai p p 1) ] r(k w n ) and the conditional SMC Distribution: ( ) ψ n,l M ã k n L+2:n, x k n L+1:n ; x n L b k n L+1:n 1, k, x k n L+1:n = ( q x b k n,n L+1 n L+1 x n L r p=n L+2 i=1 ψn,l M (ã n L+2:n, x n L+1:n, k; x n L ) ) [ n ) ( ( bk n,p w p 1 q x b k n,p p x b n n,p 1 p 1 ) ] r(k w n )

6 A (Rather Broad) Class of Hidden Markov Models y 1 x 1 z 1 x 2 z 2 y 2 x 3 z 3 y 3 Unobserved Markov chain {(X n, Z n )} transition f. Observed process {Y n } conditional density g. Density: n p(x 1:n, z 1:n, y 1:n ) = f 1 (x 1, z 1 )g(y 1 x 1, z 1 ) f(x i, z i x i 1, z i 1 )g(y i x i, z i ). i=2

7 Formal Solutions Filtering and Prediction Recursions: p(x n, z n y 1:n 1 )g(y n x n, z n ) p(x n, z n y 1:n ) = p(x n, z n y 1:n 1 )g(y n x n, z n)d(x n, z n) p(x n+1, z n+1 y 1:n ) = p(x n, z n y 1:n )f(x n+1, z n+1 x n, z n )d(x n, z n ) Smoothing: p((x, z) 1:n y 1:n ) p((x, z) 1:n 1 y 1:n 1 )f((x, z) n (x, z) n 1 )g(y n (x, z) n )

8 A Simple SIR Filter Algorithmically, at iteration n: Given {Wn 1 i, (X, Z)i 1:n 1 } for i = 1,..., N: Resample, obtaining { 1 N, ( X, Z) i 1:n 1 }. Sample (X, Z) i n q n ( ( X, Z) i n 1) Weight W i n f((x,z)i n ( X, Z) i n 1 )g(yn (X,Z)i n) q n((x,z) i n ( X, Z) i n 1 ) Actually: Resample efficiently. Only resample when necessary....

9 A Rao-Blackwellized SIR Filter Algorithmically, at iteration n: Given {W X,i n 1, (Xi 1:n 1, p(z 1:n 1 X i 1:n 1, y 1:n 1)} Resample, obtaining { 1 N, ( X i 1:n 1, p(z 1:n 1 X i 1:n 1, y 1:n 1))}. For i = 1,..., N: Sample Xn i q n ( X n 1) i Set X1:n i ( X 1:n 1, i Xn). i Weight Wn X,i p(xi n,yn X i n 1 ) q n(xn i X n 1 i ) Compute p(z1:n y 1:n, X1:n). i Requires analytically tractable substructure.

10 An Approximate Rao-Blackwellized SIR Filter Algorithmically, at iteration n: Given {W X,i n 1, (Xi 1:n 1, p(z 1:n 1 X i 1:n 1, y 1:n 1)} Resample, obtaining { 1 N, ( X i 1:n 1, p(z 1:n 1 X i 1:n 1, y 1:n 1))}. For i = 1,..., N: Sample Xn i q n ( X n 1) i Set X1:n i ( X 1:n 1, i Xn). i Weight Wn X,i p(xi n,yn X i n 1 ) q n(xn i X n 1 i ) Compute p(z1:n y 1:n, X1:n). i Is approximate; how does error accumulate?

11 Exactly Approximated Rao-Blackwellized SIR Filter At time n = 1 Sample, X i 1 qx ( y 1 ). Sample, Z i,j 1 q z ( X i 1, y 1). Compute and normalise the local weights w z 1 ( X i 1, Z i,j 1 ) := p(xi 1, y 1, Z i,j ( 1 ) q z ), W z,i,j X1 i, y 1 := 1 Z i,j 1 define p(x i 1, y 1 ) := 1 M M j=1 w z 1 ( X i 1, Z i,j 1 Compute and normalise the top-level weights w x 1 ( ) X i p(x i 1 := 1, y 1) q ( x X1 i y ), W x,i 1 := 1 w z 1 ( X i 1, Zi,j 1 ) M k=1 wz 1 (X i 1, Zi,k 1 ). ( X i 1 ) w1 x N k=1 wx 1 ( X k 1 ). At times n 2, resample and do essentially the same again... )

12 Toy Example: Model We use a simulated sequence of 100 observations from the model defined by the densities: (( ) ( ) [ ]) x µ(x 1, z 1 ) =N ; z (( ) ( ) [ ]) xn xn f(x n, z n x n 1, z n 1 ) =N ;, z n z n ( ( ) [ ]) xn σ 2 g(y n x n, z n ) =N y n ;, x 0 z n 0 σz 2 Consider IMSE (relative to optimal filter) of filtering estimate of first coordinate marginals.

13 Approximation of the RBPF Mean Squared Filtering Error 10 1 N = 10 N = 20 N = 40 N = N = Number of Lower Level Particles, M For σ 2 x = σ 2 z = 1.

14 Computational Performance 10 0 Mean Squared Filtering Error N = 10 N = 20 N = 40 N = 80 N = Computational Cost, N(M+1) For σ 2 x = σ 2 z = 1.

15 Computational Performance Mean Squared Filtering Error N = 10 N = 20 N = 40 N = 80 N = Computational Cost, N(M+1) For σ 2 x = 10 2 and σ 2 z =

16 What About Other HMMs / Algorithms? Returning to: x 1 x 2 x 3 x 4 x 5 x 6 y 1 y 2 y 3 y 4 y 5 y 6 Unobserved Markov chain {X n } transition f. Observed process {Y n } conditional density g. Density: n p(x 1:n, y 1:n ) = f 1 (x 1 )g(y 1 x 1 ) f(x i x i 1 )g(y i x i ). i=2

17 Block Sampling: An Idealised Approach At time n, given x 1:n 1 ; discard x n L+1:n 1 : Sample from q(x n L+1:n x n L, y n L+1:n ). Weight with W (x 1:n ) = Optimally, p(x 1:n y 1:n ) p(x 1:n L y 1:n 1 )q(x n L+1:n x n L, y 1:n L+1:n ) q(x n L+1:n x n L, y n L+1:n ) =p(x n L+1:n x n L, y n L+1:n ) W (x 1:n ) p(x 1:n L y 1:n ) p(x 1:n L y 1:n 1 ) =p(y n x 1:n L, y n L+1:n 1 ) Typically intractable; auxiliary variable approach in [DBS06].

18 Local Particle Filtering: Current Trajectories

19 Local Particle Filtering: First Particle

20 Local Particle Filtering: SMC Proposal

21 Local Particle Filtering: CSMC Auxiliary Proposal

22 Local SMC Propose from: U n 1 1:M (b 1:n 2, k)p(x 1:n 1 y 1:n 1 )ψn,l(a M n L+2:n, x n L+1:n, k; x n L ) ψ n 1,L 1 M (ã k n L+2:n 1, x k n L+1:n 1 ; x ) n L b n L+2:n 1, x n L+1:n 1 Target: 1:M (b 1:n L, b k n,n L+1:n 1, k)p(x 1:n L, x b k n L+1:n ( a k n L+2:n, x k n L+1:n ; x n L b k n,n L+1:n, x b U n ψ M n,l ψ M n 1,L 1 (ã n L+2:n 1, x n L+1:n 1, k; x n L ). Weight: Zn L+1:n / Z n L+1:n 1. n,n L+1:n y 1:n) k n,n L+1:n n L+1:n )

23 Proposal: q(x t x t 1, y t ) = f(x t x t 1) Stochastic Volatility Bootstrap Local SMC Model: f(x i x i 1 ) =N ( φx i 1, σ 2) g(y i x i ) =N ( 0, β 2 exp(x i ) ) Top Level: Local SMC proposal. Stratified resampling when ESS< N/2. Local SMC Proposal: Weighting: W (x t 1, x t ) f(x t x t 1 )g(y t x t ) f(x t x t 1 ) = g(y t x t ) Resample residually every iteration.

24 SV Exchange Rata Data 5 Exchange Rate Data Observed Values n

25 SV Bootstrap Local SMC: M= N = 100, M = 100 Average Number of Unique Values L = 2 L = 4 L = 6 L = n

26 SV Bootstrap Local SMC: M= N = 100, M = 1000 Average Number of Unique Values L = 2 L = 4 L = 6 L = 10 L = 14 L = n

27 SV Bootstrap Local SMC: M= N=100, M=10,000 Average Number of Unique Values L = 2 L = 4 L = 6 L = 10 L = 14 L = 18 L = 22 L = n

28 Some Heuristics Recent calculations suggest that, under appropriate assumptions, at fixed cost (2L 1) M N: Optimal L is determined solely by the mixing of the HMM. Optimal M is a linear function of L. N can then be obtained from M, L and available budget. In practice: L can be chosen using pilot runs, and M fine-tuned once L is chosen.

29 In Conclusion SMC can be used hierarchically. Software implementation is not difficult [Joh09, Zho13]. The Rao-Blackwellized particle filter can be approximated exactly Can reduce estimator variance at fixed cost. Attractive for distributed/parallel implementation. Allows combination of different sorts of particle filter. Can be combined with other techniques for parameter estimation etc.. The optimal block-sampling particle filter can be approximated exactly Requiring only simulation from the transition and evaluation of the likelihood Easy to parallelise Low storage cost

30 References I [AD02] [ADH10] C. Andrieu and A. Doucet. Particle filtering for partially observed Gaussian state space models. Journal of the Royal Statistical Society B, 64(4): , C. Andrieu, A. Doucet, and R. Holenstein. Particle Markov chain Monte Carlo. Journal of the Royal Statistical Society B, 72(3): , [CJP13] N. Chopin, P. Jacob, and O. Papaspiliopoulos. SMC 2. Journal of the Royal Statistical Society B, 75(3): , [CL00] R. Chen and J. S. Liu. Mixture Kalman filters. Journal of the Royal Statistical Society B, 62(3): , [CSOL11] [DBS06] [GSS93] [JD14] T. Chen, T. Schön, H. Ohlsson, and L. Ljung. Decentralized particle filter with arbitrary state decomposition. IEEE Transactions on Signal Processing, 59(2): , February A. Doucet, M. Briers, and S. Sénécal. Efficient block sampling strategies for sequential Monte Carlo methods. Journal of Computational and Graphical Statistics, 15(3): , N. J. Gordon, S. J. Salmond, and A. F. M. Smith. Novel approach to nonlinear/non-gaussian Bayesian state estimation. IEE Proceedings-F, 140(2): , April A. M. Johansen and A. Doucet. Hierarchical particle sampling for intractable state-space models. CRiSM working paper, University of Warwick, In preparation. [Joh09] A. M. Johansen. SMCTC: Sequential Monte Carlo in C++. Journal of Statistical Software, 30(6):1 41, April [JWD12] A. M. Johansen, N. Whiteley, and A. Doucet. Exact approximation of Rao-Blackwellised particle filters. In Proceedings of 16th IFAC Symposium on Systems Identification, Brussels, Belgium, July IFAC, IFAC. [Zho13] Y. Zhou. vsmc: Parallel sequential Monte Carlo in C++. Mathematics e-print , ArXiv, 2013.

31 Key Identity = ψ M n,l (a n L+2:n, x n L+1:n, k; x n L ) p(x n L+1:n x n L, y n L+1:n ) ψ n,l M (a k ( ) [ q x bk n,n L+1 n L+1 x n n L =Ẑn L+1:n/p(y n L+1:n x n L ) n L+2:n, x k r ( ( ) b k n,p w p 1 q p=n L+2 p(x n L+1:n x n L, y n L+1:n ) n L+1:n, k; x n L... ) ) ] r(k w n ) x bk n,p p x bn n,p 1 p 1