WEIGHTING A RESAMPLED PARTICLES IN SEQUENTIAL MONTE CARLO (EXTENDED PREPRINT) L. Martino, V. Elvira, F. Louzada

Transcription

1 WEIGHTIG A RESAMLED ARTICLES I SEQUETIAL MOTE CARLO (ETEDED RERIT) L. Martino, V. Elvira, F. Louzaa Dep. of Signal Theory an Communic., Universia Carlos III e Mari, Leganés (Spain). Institute of Mathematical Sciences an Computing, Universiae e São aulo, São Carlos (Brazil). ABSTRACT The Sequential Importance Resampling (SIR) metho is the core of the Sequential Monte Carlo (SMC) algorithms (a.k.a., particle filters). In this work, we point out a suitable choice for weighting properly a resample particle. This observation entails several theoretical an practical consequences, allowing also the esign of novel sampling schemes. Specifically, we escribe one theoretical result about the sequential estimation of the marginal likelihoo. Moreover, we suggest a novel resampling proceure for SMC algorithms calle partial resampling, involving only a subset of the current clou of particles. Clearly, this scheme attenuates the aitional variance in the Monte Carlo estimators generate by the use of the resampling. Inex Terms Importance Sampling; Sequential Importance Resampling; Sequential Monte Carlo; article Filtering. 1. ITRODUCTIO Sequential Monte Carlo (SMC) methos have become essential tools for Bayesian analysis in statistical signal processing 2, 8, 9, 12, 18. SMC algorithms (a.k.a., particle filters) are base on the importance sampling technique 5, 7, 6, 11, 16, 22 an its sequential version known as Sequential Importance Sampling (SIS) 10, 13. Another essential piece of SMC is the application of resampling proceures 3, 10. The combination of SIS an resampling is often referre as Sequential Importance Resampling (SIR). Since the unnormalize importance weight of a resample particle cannot be compute analytically using the stanar IS weight efinition, in the classical SIR formulation, the users consier only the estimators involving normalize weights. The concept of the unnormalize weight of a resample particle is usually not consiere, i.e., its computation is avoie an omitte 8, 9, 10. In this work, we introuce a proper unnormalize importance weight for a particle resample from a set of weighte samples, efine as the arithmetic mean of the importance weights of these samples. This weight choice is proper accoring to the Liu s efinition 13, Section since it provies unbiase IS estimators, as shown in this work. The introuction of this unnormalize proper weight for a resample particle entails several interesting consequences from a practical an theoretical point of view. For instance, this weight efinition has been alreay applie implicitly or heuristically in ifferent works: in parallel particle filters 4, 19, 20 an parallel SMC schemes, e.g., the islan particle- ouble bootstrap metho 25, 26, or unawarely in certain classes of MCMC algorithms 1, 14 (where one particle is chosen among a set of caniates via resampling, before be testing as possible future state of the chain), as we can infer from the iscussion in 17. Similarly approaches have been implicitly use in the so-calle α-smc 27 an este-smc methos 21. Here, we also escribe two aitional consequences. First, we highlight that all the estimators erive in the SIS approach can also be employe in SIR using the weight efinition of a resample particle introuce here. We show it consiering the estimation of the marginal likelihoo (a.k.a., Bayesian evience or partition function) 10, 18, 22. In SIS, there are two possible estimators of the marginal likelihoo which are completely equivalent 18. Using the proper unnormalize weight for a resample particle, we show that we can employ two equivalent estimators of the marginal likelihoo also in SIR. They coincie with the estimators in SIS as special case, when no resampling is applie. Furthermore, we escribe an alternative resampling proceure for particle filtering algorithms, calle partial resampling, involving only a subset of the This is an extension of the work 15. This work has been supporte by the ERC grant an AoF grant , the Spanish government through the OTOSiS (TEC R), by the Grant 2014/ of the São aulo Research Founation (FAES) an by the Grant / of the ational Council for Scientific an Technological Development (Cq).

2 current population of particles. This scheme attenuates the loss of iversity in the population an the aitional variance in the Monte Carlo estimators generate ue to the application of the resampling steps. 2. IMORTACE SAMLIG Let us enote the target probability ensity function (pf) as π(x) 1 Z π(x) (known up to a normalizing constant) with x x 1:D x 1, x 2,..., x D R D η, where x R η for all 1,..., D. We consier the Monte Carlo approximation of complicate integrals involving the target π(x) an a square-integrable function h(x), e.g., I E π h() h(x) π(x)x, (1) where π(x). In general, generating samples irectly from the target π(x) is impossible. Thus, one usually consiers a (simpler) proposal pf, q(x). The expression below E π h() E q h()w(), 1 h(x) π(x) q(x)x, (2) Z q(x) where w(x) π(x) q(x) : R, suggests an alternative proceure. Inee, we can raw samples (also calle particles) x 1,..., x from q(x), 1 an then assign to each sample the following unnormalize weights w(x n ) π(x n), n 1,...,. (3) q(x n ) If the target function π(x) is normalize, i.e., Z 1, π(x) π(x), a natural (unbiase) IS estimator 13, 22 is efine as Î 1 w(x n )h(x n ), Î I, (4) where x n q(x), n 1,...,. If the normalizing constant Z is unknown, efining the normalize weights, w(x n ) w(x n ) w(x, n 1,...,, (5) i) an alternative self-normalize (biase) IS estimator 13, 22 is I Moreover, an unbiase estimator of marginal likelihoo, Z π(x)x, is given by w(x n )h(x n ), I I. (6) Ẑ 1 w(x i ), Ẑ Z, (7) where we have avoie the subinex, in orer to simplify the notation in the rest of the work. 1 We assume that q(x) > 0 for all x where π(x) 0, an q(x) has heavier tails than π(x).

3 2.1. Concept of proper weighte sample Although the weights of Eq. (3) are broaly use in the literature, the concept of a properly weighte sample, suggeste in 22, Section 14.2 an in 13, Section 2.5.4, can be use to construct more general weights. More specifically, following the efinition in 13, Section 2.5.4, a set of weighte samples is consiere proper with respect to the target π if, for any square integrable function h, E q w(x n )h(x n ) ce π h(x n ), n {1,..., }, (8) where c is a constant value, also inepenent from the inex n, an the expectation of the left han sie is performe, in general, w.r.t. to the joint pf of w(x) an x, i.e., q(w, x). amely, the weight w(x), (for a given value of x), coul even be consiere a ranom variable. 3. IMORTACE WEIGHT OF A RESAMLED ARTICLE Let us consier the following multinomial resampling proceure 3, 8, 9: 1. Draw particles x n q(x) an weight them with w(x n ) π(xn) q(x n), with n 1,...,. 2. Draw one particle x {x 1,..., x } from the iscrete probability mass where w(x n ) w(xn) w(xi). π(x x 1: ) w(x n )δ(x x n ), (9) Question 1. What is the istribution of the resample particle x (not conitione to x 1: )? We can easily write its corresponing ensity as q(x) q(x i ) π(x x 1: )x 1:. (10) where π is given in Eq. (9). However, the integral above cannot be compute analytically. Question 2. Can we obtain a proper importance weight associate to the resample particle x? As a consequence of the previous observations, we are not able to evaluate the corresponing stanar importance weight, w( x) π(ex) eq(ex). For solving this issue, let us consier resample particles x 1,..., x inepenently obtaine by the resampling proceure above. In SMC an aaptive IS applications, 5, 7, 8, 9, the unnormalize importance weights of x 1,..., x are not usually neee, but only the normalize ones. Thus, a well known proper strategy 8, 9, 10 in this case is to consier an, as a consequence, the normalize weights are w( x 1 ) w( x 2 )... w( x ), (11) w( x 1 ) w( x 2 )... w( x ) 1. (12) The reason why this approach is suitable lies on the Liu s efinition of proper importance weights in Section 2.1. Inee, consiering the ranom variable π(x x 1: ), we have E bπ h( ) x 1: h(x) π(x x 1: )x, w(x n )h(x n ) I. (13) where x n q(x) for n 1,...,, are consiere fixe in the expectation E bπ h( ) x 1:. ow, let us resample M times. The self-normalize IS estimator using the M resample particles is Ĩ M 1 M M h( x m ) E bπh( ) x 1: I. (14) M m1

4 Hence, we have Ĩ M I M I, (15) ue to Eqs. (13)-(14). This proves that the choice w( x m ) 1 M, for all m 1,..., M, is proper by Liu s efinition. However, for several theoretical an practical reasons (some of them iscusse below), it is interesting to efine also a proper unnormalize importance weight of a resample particle. Let us consier the following efinition. Definition 1. A proper choice for an unnormalize importance weight value (following Section 2.1) of a resample particle x {x 1,..., x } is ρ( x) ρ( x x 1: ) Ẑ 1 w(x i ). (16) Inee, in this case, we have where Q(x, x 1: ) π(x x 1: ) q(x i). E eq(x,x1: ) ρ(x x 1:)h(x) ce π h(x), (17) roof. We show that Eq. (17) hols. ote that E eq(x,x1: ) ρ(x x 1:)h(x) h(x)ρ(x x 1: ) Q(x, x 1: )xx 1:, (18) h(x)ρ(x x 1: ) π(x x 1: ) q(x i ) xx 1:. (19) Recalling that π(x x 1: ) 1 w(x n )δ(x x n ) w(x n) 1 Ẑ w(x n )δ(x x n ), (20) w(x n )δ(x x n ), (21) where Ẑ Ẑ(x 1:) 1 E eq(x,x1: ) ρ(x x 1: )h(x) w(x n). Recalling also w(x n ) π(xn) q(x n), we can rearrange the expectation above as where x j x 1,..., x j 1, x j+1,..., x. The we have, E eq(x,x1: ) ρ(x x 1:)h(x) h(x) ρ(x x 1: ) w(x) q(x i ) x j x, (22) 1 Ẑ h(x) π(x) i j 1 ρ(x x 1: ) 1 Ẑ q(x i ) x j x, (23) i j h(x)π(x) ρ(x x 1: ) 1 q(x i ) x j x, (24) 1 Ẑ i j

5 If we choose ρ(x x 1: ) Ẑ an replace in the expression above, we obtain E eq(x,x1: ) ρ(x x 1: )h(x) h(x)π(x) Ẑ 1 1 Ẑ q(x i ) x j x, (25) i j h(x)π(x) 1 x, (26) h(x)π(x)x (27) ce π h(x), (28) where c Z. Consiering inepenent resample particles, ote that with this efinition we again have ρ( x 1 ) ρ( x 2 )... ρ( x ), so that also ρ( x n ) 1, for all n 1,...,, enoting with ρ( x n) the corresponing normalize weights. Remark 1. The previous efinition allows is to estimate Z using the resample particles as well. Inee, Z 1 ρ( x i ) 1 ( ) Ẑ Ẑ, (29) is an unbiase estimator of Z (equivalent to Ẑ). 4. ALICATIO I SIR Let recall x x 1:D x 1, x 2,..., x D R D η where x R η for all 1,..., D an let us consier a target pf π(x) factorize as π(x) π(x) γ 1 (x 1 ) D γ (x x 1: 1 ), (30) where γ 1 (x 1 ) is a marginal pf an γ (x x 1: 1 ) are conitional pfs. We also enote the joint probability of x 1,..., x, where 2 π (x 1: ) 1 Z π (x 1: ), (31) π (x 1: ) γ 1 (x 1 ) γ j (x j x 1:j 1 ). Clearly, π(x) π D (x 1:D ). We can also consier a proposal pf ecompose in the same fashion, j2 q(x) q 1 (x 1 )q 2 (x 2 x 1 ) q D (x D x 1:D 1 ). In a batch IS scheme, given the n-th sample x n x (n) 1:D q(x), we assign the importance weight w(x n ) π(x n) q(x n ) γ 1(x (n) 1 )γ 2(x (n) 2 x(n) 1 ) γ D(x (n) D x(n) 1:D 1 ) q 1 (x (n) 1 )q 2(x (n) 2 x(n) 1 ) q D(x (n) D x(n) 1:D 1 ).

6 The previous expression suggests a recursive proceure for computing the importance weights. Inee, in a sequential Importance sampling (SIS) approach 8, 9, we can write where we have set 1 1 β(n) 1 π(x(n) 1 ) q(x (n) 1 ) an j, n 1,...,, (32) γ (x (n) x(n) 1: 1 ) q (x (n) x(n) 1: 1 ), (33) for 2,..., D. Clearly, w(x n ) D. The estimator of the normalizing constant Z π R η (x 1: )x 1: at the -th iteration is Ẑ 1 1 Ẑj Again, Z Z D an Ẑ ẐD. However, an alternative formulation is often use 9, 10 Z j 1 β(n) j w(n) j, w(n) j 1 Ẑ1 j2 Ẑ j 1 1 1, (34). (35) j Ẑ 2 Ẑ Ẑ1... Ẑ 1 Ẑ Ẑ. (36) 1 Therefore, in SIS, Ẑ in Eq. (34) an Z in Eq. (36) are equivalent formulations of the same estimator of Z Estimators of the marginal likelihoo in SIR Sequential Importance Resampling (SIR) 13, 22, 23, 24 combines the SIS approach with the application of the resampling proceure escribe in Section 3. Consiering the proper importance weight of a resample particle given in Definition 1 an recalling that the weight at the -th iteration, we obtain the following recursion, 1 1 an for 2,..., D, where 1 β(n), (37) 1 { (n) w 1, without resampling at ( 1)-th it., Ẑ 1, with resampling at ( 1)-th it., (38) i.e., if a resampling is applie at ( 1)-th iteration then ξ (n) 1 Ẑ 1, n 1,...,. Remark 2. Using the Definition 1 an the recursive efinition of the weights in Eqs. (37)-(38), Ẑ an Z are both consistent an equivalent estimators of the marginal likelihoo, also in SIR. amely, the two estimators are Ẑ 1, Z j 1 β(n) j (39) are equivalent, Ẑ Z. For instance, if the resampling is applie at each iteration, they become 1 Z j, (40)

7 an clearly coincie. 1 Ẑ Ẑ 1 1 j, (41) Remark 3. Let us focus on the marginal likelihoo estimators at the final iteration, i.e., Ẑ ẐD an Z Z D. Without using the Definition 1 an the recursive efinition of the weights in Eqs. (37)-(38), the only estimator of the marginal likelihoo that can be properly compute in SIR is Z, that involves only the computation of the normalize weights (omitting the values of the corresponing unnormalize ones). 5. ARTIAL RESAMLIG The core of Sequential Monte Carlo methos is the SIR approach 8, 9, 23. amely, the weights are constructe recursively as in (32) an resampling steps, involving all the particles, are applie at some iterations. The combination of both, SIS an resampling schemes, is possible in the stanar SIR approach only if the entire set of particles is employe in the resampling 8, 9, so that the assumption ρ( x 1 ) ρ( x 2 )... ρ( x ), (42) is enough for computing I an Z, since w( x n ) 1 for all n {1,..., }. If Definition 1 is use an then the recursive expression (37)-(38) is applie, we can efine a resampling proceure involving only a subset of particles, as escribe in the following. We consier to apply a partial resampling scheme at the -th iteration: 1. Choose ranomly without replacement a subset of M samples, containe within the set of particles {x (1) R {x (j1),..., x (j M ) },,..., x() }. Let us enote also the set {x (1) of the particles which o not take part in the resampling.,..., x() }\R, 2. Give the set R, resample with replacement M particles accoring to the normalize weights w (jm) m 1,..., M, obtaining R { x (1),..., x(m) }. Clearly, R R. 3. For all the resample particles in R set w (m) 1 M m k1 w(jm) M k1 w(j k ), for w (j k), (43) for m 1,..., M, whereas the unnormalize importance weights of the particles in remains invariant. 4. Go forwar to the iteration + 1 of the SIR metho, using the recursive formula (32). The proceure above is vali since yiels proper weighte samples by Liu s efinition. If M, it coincies with the traitional resampling proceure. This approach can reuce the loss of iversity ue to the application of the resampling 8, 9, COCLUSIOS In this work, we have introuce a proper choice of the unnormalize weight assigne to a resample particle. This choice entails several theoretical an practical consequences. We have escribe two of them, regaring (1) the estimation of the marginal likelihoo an, (2) the application of a partial resampling involving only a subset of the clou of particles, within SIR techniques. Other novel algorithms (base on the partial resampling perspective) an theoretical consequences (also affecting well-known the MCMC techniques, such as the particle Metropolis-Hastings metho 1, 17, an parallel SMC implementations) will be highlighte out in an extene version of this work.

8 References 1 C. Anrieu, A. Doucet, an R. Holenstein. article Markov chain Monte Carlo methos. Journal of the Royal Statistical Society B, 72(3): , M. S. Arulumpalam, S. Maskell,. Goron, an T. Klapp. A tutorial on particle filters for online nonlinear/non-gaussian Bayesian tracking. IEEE Transactions Signal rocessing, 50(2): , February M. Bolić,. M. Djurić, an S. Hong. Resampling algorithms for particle filters: A computational complexity perspective. EURASI Journal on Avances in Signal rocessing, 2004(15): , ovember M. Bolić,. M. Djurić, an S. Hong. Resampling algorithms an architectures for istribute particle filters. IEEE Transactions Signal rocessing, 53(7): , July M. F. Bugallo, L. Martino, an J. Coraner. Aaptive importance sampling in signal processing. Digital Signal rocessing, (47):36 49, O. Cappe an T. Ryen E. Moulines. Inference in Hien Markov Moels. Springer, O. Cappé, A. Guillin, J. M. Marin, an C.. Robert. opulation Monte Carlo. Journal of Computational an Graphical Statistics, 13(4): , M. Djurić, J. H. Kotecha, J. Zhang, Y. Huang, T. Ghirmai, M. F. Bugallo, an J. Míguez. article filtering. IEEE Signal rocessing Magazine, 20(5):19 38, September A. Doucet,. e Freitas, an. Goron, eitors. Sequential Monte Carlo Methos in ractice. Springer, ew York (USA), A. Doucet an A. M. Johansen. A tutorial on particle filtering an smoothing: fifteen years later. technical report, V. Elvira, L. Martino, D. Luengo, an M. F. Bugallo. Generalize multiple importance sampling. ariv: , F. Gustafsson, F. Gunnarsson,. Bergman, U. Forssell, J. Jansson, R. Karlsson, an.-j. orlun. article filters for positioning, navigation an tracking. IEEE Transactions Signal rocessing, 50(2): , February J. S. Liu. Monte Carlo Strategies in Scientific Computing. Springer, J. S. Liu, F. Liang, an W. H. Wong. The multiple-try metho an local optimization in metropolis sampling. Journal of the American Statistical Association, 95(449): , March L. Martino, V. Elvira, an F. Louzaa. Weighting a resample particle in Sequential Monte Carlo. IEEE Statistical Signal rocessing Workshop, (SS), 122:1 5, L. Martino, V. Elvira, D. Luengo, an J. Coraner. An aaptive population importance sampler: Learning from the uncertanity. IEEE Transactions on Signal rocessing, 63(16): , L. Martino, F. Leisen, an J. Coraner. On multiple try schemes an the article Metropolis-Hastings algorithm. vira: , L. Martino, J. Rea, V. Elvira, an F. Louzaa. Cooperative parallel particle filters for on-line moel selection an applications to urban mobility. vira: , J. Míguez. Analysis of parallelizable resampling algorithms for particle filtering. Signal rocessing, 87(12): , J. Miguez an M. A. Vazquez. A proof of uniform convergence over time for a istribute particle filter. Signal rocessing, 122: , C. A. aesseth, F. Linsten, an T. B. Schon. este Sequential Monte Carlo methos. roceeings of theinternational Conference on Machine Learning, 37:1 10, 2015.

9 22 C.. Robert an G. Casella. Monte Carlo Statistical Methos. Springer, D. B. Rubin. A noniterative sampling/importance resampling alternative to the ata augmentation algorithm for creating a few imputations when fractions of missing information are moest: the SIR algorithm. Journal of the American Statistical Association, 82: , D. B. Rubin. Using the sir algorithm to simulate posterior istributions. in Bayesian Statistics 3, as Bernaro, Degroot, Linley, an Smith. Oxfor University ress, Oxfor, 1988., C. Verg, C. Dubarry,. Del Moral, an E. Moulines. On parallel implementation of sequential Monte Carlo methos: the islan particle moel. Statistics an Computing, 25(2): , C. Verg,. Del Moral, E. Moulines, an J. Olsson. Convergence properties of weighte particle islans with application to the ouble bootstrap algorithm. ariv: , pages 1 39, Whiteley, A. Lee, an K. Heine. On the role of interaction in sequential Monte Carlo algorithms. Bernoulli, 22(1): , 2016.