WEIGHTING A RESAMPLED PARTICLE IN SEQUENTIAL MONTE CARLO. L. Martino, V. Elvira, F. Louzada

Transcription

1 WEIGHTIG A RESAMPLED PARTICLE I SEQUETIAL MOTE CARLO L. Martino, V. Elvira, F. Louzaa Dep. of Signal Theory an Communic., Universia Carlos III e Mari, Leganés (Spain). Institute of Mathematical Sciences an Computing, Universiae e São Paulo, São Carlos (Brazil). ABSTRACT The Sequential Importance Resampling (SIR) metho is the core of the Sequential Monte Carlo (SMC) algorithms (a.k.a., particle filters). In this work, we point out a suitable choice for weighting properly a resample particle. This observation entails several theoretical an practical consequences, allowing also the esign of novel sampling schemes. Specifically, we escribe one theoretical result about the sequential estimation of the marginal likelihoo. Moreover, we suggest a novel resampling proceure for SMC algorithms calle partial resampling, involving only a subset of the current clou of particles. Clearly, this scheme attenuates the aitional variance in the Monte Carlo estimators generate by the use of the resampling. Inex Terms Importance Sampling; Sequential Importance Resampling; Sequential Monte Carlo; Particle Filtering. 1. ITRODUCTIO Sequential Monte Carlo (SMC) methos have become essential tools for Bayesian analysis in statistical signal processing 2, 6, 7, 10, 15. SMC algorithms are base on the importance sampling technique 5, 4, 9, 12, 16 an its sequential version known as Sequential Importance Sampling (SIS) 8, 11. Another essential piece of SMC is the application of resampling proceures 3, 8. The combination of SIS an resampling is often referre as Sequential Importance Resampling (SIR). Since the unnormalize importance weight of a resample particle cannot be compute analytically using the stanar IS weight efinition, in the classical SIR formulation, the users consier only the estimators involving normalize weights. The concept of the unnormalize weight of a resample particle is usually not consiere, i.e., its computation is avoie an omitte 6, 7, 8. In this work, we propose a proper unnormalize importance weight for a resample particle which provies unbiase IS estimators, following the Liu s efinition of a proper This work has been supporte by the ERC grant an AoF grant , the Spanish government through the OTOSiS (TEC R), by the Grant 2014/ of the São Paulo Research Founation (FAPESP) an by the Grant / of the ational Council for Scientific an Technological Development (CPq). weighte sample 11, Section The introuction of this unnormalize proper weight for a resample particle yiels several interesting consequences. Here, we escribe two of them. First, we point out that all the estimators erive in the SIS approach can also be employe in SIR using the introuce weight efinition of a resample particle. We show it consiering the estimation of the marginal likelihoo (a.k.a., Bayesian evience or partition function) 8, 15, 16. In SIS, there are two possible estimators of the marginal likelihoo which are completely equivalent 15. Using the proper unnormalize weight for a resample particle, we show that also in SIR we can employ two equivalent estimators of the marginal likelihoo. They coincie with the estimators in SIS as special case, when no resampling is applie. Furthermore, we escribe an alternative resampling proceure for SMC algorithms, calle partial resampling, involving only a subset of the current population of particles. This scheme attenuates the aitional variance in the Monte Carlo estimators generate by the use of resampling steps. Moreover, it can reuce the loss of iversity in the population of particle ue to the application of the resampling. 2. IMPORTACE SAMPLIG Let us enote the target probability ensity function (pf) as π(x) 1 Z π(x) (known up to a normalizing constant) with x x 1:D x 1, x 2,..., x D R D η, where x R η for all 1,..., D. We consier the Monte Carlo approximation of complicate integrals involving the target π(x) an a square-integrable function h(x), e.g., I E π h() h(x) π(x)x, (1) where π(x). In general, generating samples irectly from the target π(x) is impossible. Thus, one usually consiers a (simpler) proposal pf, q(x). Inee, the equality E π h() E q h()w(), 1 h(x) π(x) q(x)x, (2) Z q(x)

2 where w(x) π(x) q(x) : R, suggests an alternative proceure. Inee, we can raw samples (also calle particles) x 1,..., x from q(x), 1 an then assign to each sample the following weights w(x n ) π(x n), n 1,...,. (3) q(x n ) If the target function π(x) is normalize, i.e., Z 1, π(x) π(x), a natural (unbiase) IS estimator 11, 16 is efine as Î 1 w(x n )h(x n ) I, (4) where x n q(x), n 1,...,. If the normalizing constant Z is unknown, efining the normalize weights, w(x n ) w(x n ) w(x, n 1,...,, (5) i) an alternative self-normalize (biase) IS estimator 11, 16 is I w(x n )h(x n ) I. (6) Moreover, an unbiase estimator of marginal likelihoo, Z π(x)x, is given by Ẑ 1 w(x i ) 2.1. Concept of proper weighte sample Z. (7) Although the weights of Eq. (3) are broaly use in the literature, the concept of a properly weighte sample, suggeste in 16, Section 14.2 an in 11, Section 2.5.4, can be use to construct more general weights. More specifically, following the efinition in 11, Section 2.5.4, a set of weighte samples is consiere proper with respect to the target π if, for any square integrable function h, E q w(x n )h(x n ) ce π h(x n ), n {1,..., }, (8) where c is a constant value, also inepenent from the inex n, an the expectation of the left han sie is performe, in general, w.r.t. to the oint pf of w(x) an x, i.e., q(w, x). amely, the weight w(x), (for a given value of x), coul even be consiere a ranom variable. 3. IMPORTACE WEIGHT OF A RESAMPLED PARTICLE Let us consier the following multinomial resampling proceure 3, 6, 7: 1 We assume that q(x) > 0 for all x where π(x) 0, an q(x) has heavier tails than π(x). 1. Draw particles x n q(x) an weight them with w(x n ) π(xn) q(x n), with n 1,...,. 2- Draw one particle x {x 1,..., x } from the iscrete probability mass where w(x n ) π(x x 1: ) w(x n )δ(x x n ), (9) w(xn) P w(xi). Question 1. What is the istribution of the resample particle x (not conitione to x 1: )? We can easily write its corresponing ensity as q(x) q(x i ) π(x x 1: )x 1:. (10) where π is given in Eq. (9). However, the integral above cannot be compute analytically. Question 2. Can we obtain a proper importance weight associate to the resample particle x? As a consequence of the previous observations, we are not able to evaluate the corresponing stanar importance weight, w( x) π(ex) eq(ex). For solving this issue, let us consier resample particles x 1,..., x inepenently obtaine by the resampling proceure above. In SMC an aaptive IS applications, 4, 5, 6, 7, the unnormalize importance weights of x 1,..., x are not usually neee, but only the normalize ones. Thus, a well known proper strategy 6, 7, 8 in this case is to set w( x 1 ) w( x 2 )... w( x ), (11) an, as a consequence, the normalize weights are w( x 1 ) w( x 2 )... w( x ) 1. (12) The reason why this approach is suitable lies on the Liu s efinition of proper importance weights in Section 2.1. Inee, consiering the ranom variable π(x x 1: ), we have E bπ h( ) x 1: h(x) π(x x 1: )x, w(x n )h(x n ) I. (13) where x n q(x) for n 1,...,, are consiere fixe in the expectation E bπ h( ) x 1:. Thus, The self-normalize IS estimator using the resample particles is Ĩ 1 h( x n ) E bπh( ) x 1: I. (14)

3 Hence, we can write for Ĩ I I, (15) ue to Eqs. (13)-(14). This is the proof that the choice w( x n ) 1, for all n, is proper by Liu s efinition. However, for several theoretical an practical reasons (some of them iscusse below), it is interesting to efine also a proper unnormalize importance weight of a resample particle. Let us consier the following efinition. Definition 1. A proper choice for an unnormalize importance weight (following Section 2.1) of a resample particle x {x 1,..., x } is w( x) Ẑ 1 w(x i ). (16) Consiering inepenent resample particles, ote that with this efinition we again have w( x 1 ) w( x 2 )... w( x ), so that also w( x n ) 1, for all n 1,...,. The unnormalize weight in Eq. (16) suggests that a resample particle x conenses the statistical information provie by x 1,..., x. Remark 1. The previous efinition allows is to estimate Z using the resample particles as well. Inee, Z 1 w( x i ) 1 is an unbiase estimator of Z (equivalent to Ẑ). 4. APPLICATIO I SIR ( ) Ẑ Ẑ, (17) Let recall x x 1:D x 1, x 2,..., x D R D η where x R η for all 1,..., D an let us consier a target pf π(x) factorize as π(x) π(x) γ 1 (x 1 ) D γ (x x 1: 1 ), (18) 2 where γ 1 (x 1 ) is a marginal pf an γ (x x 1: 1 ) are conitional pfs. We also enote the oint probability of x 1,..., x, where π (x 1: ) 1 Z π (x 1: ), (19) π (x 1: ) γ 1 (x 1 ) γ (x x 1: 1 ). 2 Clearly, π(x) π D (x 1:D ). We can also consier a proposal pf ecompose in the same fashion, q(x) q 1 (x 1 )q 2 (x 2 x 1 ) q D (x D x 1:D 1 ). In a batch IS scheme, given the n-th sample x n x (n) 1:D q(x), we assign the importance weight w(x n ) π(x n) q(x n ) γ 1(x (n) 1 )γ 2(x (n) 2 x(n) 1 ) γ D(x (n) D x(n) 1:D 1 ) q 1 (x (n) 1 )q 2(x (n) 2 x(n) 1 ) q D(x (n) D x(n) 1:D 1 ). The previous expression suggests a recursive proceure for computing the importance weights. Inee, in a sequential Importance sampling (SIS) approach 6, 7, we can write 1 β(n) where we have set 1 1 π(x(n) 1 ) q(x (n), n 1,...,, (20) 1 ) an γ (x (n) x(n) 1: 1 ) q (x (n) x(n) 1: 1 ), (21) for 2,..., D. Clearly, w(x n ) D. The estimator of the normalizing constant Z π R η (x 1: )x 1: at the -th iteration is Ẑ 1 1 Ẑ 1 1, (22). (23) Again, Z Z D an Ẑ ẐD. However, an alternative formulation is often use 7, 8 Z 1 β(n) w(n), w(n) 1 Ẑ1 2 Ẑ 1 Ẑ 2 Ẑ Ẑ1... Ẑ 1 Ẑ Ẑ. (24) 1 Therefore, in SIS, Ẑ in Eq. (22) an Z in Eq. (24) are equivalent formulations of the same estimator of Z Estimators of the marginal likelihoo in SIR Sequential Importance Resampling (SIR) 11, 16 combines the SIS approach with the application of the resampling proceure escribe in Section 3. Consiering the proper importance weight of a resample particle given in Definition 1 an

4 recalling that the weight at the -th iteration, we obtain the following recursion, 1 1 an for 2,..., D, where 1 1 β(n), (25) { (n) w 1, without resampling at ( 1)-th iter., Ẑ 1, with resampling at ( 1)-th iter., (26) i.e., if a resampling is applie at ( 1)-th iteration then ξ (n) 1 Ẑ 1, n 1,...,. Remark 2. Using the Definition 1 an the recursive efinition of the weights in Eqs. (25)-(26), Ẑ an Z are both consistent an equivalent estimators of the marginal likelihoo, also in SIR. amely, the two estimators are Ẑ 1, Z 1 β(n) (27) are equivalent, Ẑ Z. For instance, if the resampling is applie at each iteration, they become Z 1 Ẑ Ẑ 1 an clearly coincie. 1 1, (28), (29) Remark 3. Let us focus on the marginal likelihoo estimators at the final iteration, i.e., Ẑ ẐD an Z Z D. Without using the Definition 1 an the recursive efinition of the weights in Eqs. (25)-(26), the only estimator of the marginal likelihoo that can be properly compute in SIR is Z, that involves only the computation of the normalize weights (omitting the values of the corresponing unnormalize ones). 5. PARTIAL RESAMPLIG The core of Sequential Monte Carlo methos is the SIR approach. amely, the weights are constructe recursively as in (20) an resampling steps, involving all the particles, are applie at some iterations. The combination of both, SIS an resampling schemes, is possible in the stanar SIR approach only if the entire set of particles is employe in the resampling 6, 7, so that the assumption w( x 1 ) w( x 2 )... w( x ), (30) is enough for computing I an Z, since w( x n ) 1 for all n {1,..., }. If Definition 1 is use an then the recursive expression (25)-(26) is applie, we can efine a resampling proceure involving only a subset of particles, as escribe in the following. We consier to apply a partial resampling scheme at the -th iteration: 1. Choose ranomly without replacement a subset of M samples, R {x (1),..., x ( M ) }, containe within the set of particles {x (1),..., x() }. Let us enote also the set {x (1),..., x() }\R, of the particles which o not take part in the resampling. 2. Give the set R, resample with replacement M particles accoring to the normalize weights w (m) w (m) P M k1 w( k ) { x (1), for m 1,..., M, obtaining R,..., x(m) }. Clearly, R R. 3. For all the resample particles in R set w (m) 1 m w ( k), (31) M k1 for m 1,..., M, whereas the unnormalize importance weights of the particles in remains invariant. 4. Go forwar to the iteration + 1 of the SIR metho, using the recursive formula (20). The proceure above is vali since yiels proper weighte samples by Liu s efinition. If M, it coincies with the traitional resampling proceure. This approach can reuce the loss of iversity ue to the application of the resampling 6, 7, COCLUSIOS In this work, we have introuce a proper choice of the unnormalize weight assigne to a resample particle. This choice entails several theoretical an practical consequences. We have escribe two of them, regaring (1) the estimation of the marginal likelihoo an, (2) the application of a partial resampling involving only a subset of the clou of particles, within SIR techniques. Other novel algorithms (base on the partial resampling perspective) an theoretical consequences (also affecting well-known the MCMC techniques, such as the particle Metropolis-Hastings metho 1, 14, an parallel SMC implementations) will be highlighte out in an extene version of this work. Several numerical simulations will be also inclue.

5 References 1 C. Anrieu, A. Doucet, an R. Holenstein. Particle Markov chain Monte Carlo methos. J. R. Statist. Soc. B, 72(3): , M. S. Arulumpalam, S. Maskell,. Goron, an T. Klapp. A tutorial on particle filters for online nonlinear/non-gaussian Bayesian tracking. IEEE Transactions Signal Processing, 50(2): , February M. Bolić, P. M. Durić, an S. Hong. Resampling algorithms for particle filters: A computational complexity perspective. EURASIP Journal on Avances in Signal Processing, 2004(15): , ovember M. F. Bugallo, L. Martino, an J. Coraner. Aaptive importance sampling in signal processing. Digital Signal Processing, (47):36 49, O. Cappé, A. Guillin, J. M. Marin, an C. P. Robert. Population Monte Carlo. Journal of Computational an Graphical Statistics, 13(4): , P. M. Durić, J. H. Kotecha, J. Zhang, Y. Huang, T. Ghirmai, M. F. Bugallo, an J. Míguez. Particle filtering. IEEE Signal Processing Magazine, 20(5):19 38, September A. Doucet,. e Freitas, an. Goron, eitors. Sequential Monte Carlo Methos in Practice. Springer, ew York (USA), A. Doucet an A. M. Johansen. A tutorial on particle filtering an smoothing: fifteen years later. technical report, V. Elvira, L. Martino, D. Luengo, an M. F. Bugallo. Generalize multiple importance sampling. ariv: , F. Gustafsson, F. Gunnarsson,. Bergman, U. Forssell, J. Jansson, R. Karlsson, an P.-J. orlun. Particle filters for positioning, navigation an tracking. IEEE Transactions Signal Processing, 50(2): , February J. S. Liu. Monte Carlo Strategies in Scientific Computing. Springer, L. Martino, V. Elvira, D. Luengo, an J. Coraner. An aaptive population importance sampler: Learning from the uncertanity. IEEE Transactions on Signal Processing, 63(16): , L. Martino, V. Elvira, D. Luengo, an J. Coraner. Layere aaptive importance sampling. ariv: , L. Martino, F. Leisen, an J. Coraner. On multiple try schemes an the particle metropolis-hastings algorithm. vira: , L. Martino, J. Rea, V. Elvira, an F. Louzaa. Cooperative parallel particle filters for on-line moel selection an applications to urban mobility. vira: , C. P. Robert an G. Casella. Monte Carlo Statistical Methos. Springer, 2004.