The aim Part 3 2(58)

Transcription

1 The aim Part (8 Part - onlinear state inference using sequential Monte Carlo The aim in part is to introduce the particle filter and the particle smoother. Division of Automatic Control Linköping University Sweden This will be done by first introducing the importance sampler and then show how this foundation can be used to derive a first working particle filter. Outline (8 Rejection sampling (I/VII 4(8. Basic sampling methods a Rejection sampling b Importance sampling. Particle filtering. Particle smoothing a Forward filtering backward smoothing (FFBSm b Forward filtering backward simulation (FFBSi Rejection sampling is a Monte Carlo method that produce i.i.d. samples from a target distribution π(z = π(z C π, where π(z can be evaluated and C π is a normalization constant. Key idea: Generate random numbers uniformly from the area under the graph of the target distribution π(z. Just as hard as the original problem, but what if...

2 Rejection sampling (II/VII (8 Rejection sampling (III/VII 6(8 Assumptions: z Bq( z π( z ubq( z Generate a sample z from a proposal distribution q(z and a sample u U[, ]. The sample z is then an i.i.d. sample from the target if u π( z Bq( z. It is easy to sample from q(z.. There exists a constant B such that π(z Bq(z, z Z.. The support of q(z includes the support of π(z, i.e., q(z > when π(z >. Algorithm Rejection sampling (RS. Sample z q(z.. Sample u U[, ].. If u π( z accept z as a sample from π(z and go to. Bq( z 4. Otherwise, reject z and go to. Rejection sampling (IV/VII 7(8 The procedure can be used with multivariate densities in the same way. The rejection rate depends on B, choose B as small as possible, while still satisfying π(z Bq(z, z Z. Choosing a good proposal distribution q(z is very important. Rejection sampling is used to construct fast particle smoothers via backward simulation. Rejection sampling (V/VII 8(8 Task: Generate i.i.d. samples from the following distribution, π(z = ( e z sin(6z + cos(z sin(4z +, C π Bq( z π( z Solution: Use rejection sampling where q(z = (z, and B =. ubq( z z

3 Rejection sampling (VI/VII 9(8 Rejection sampling (VII/VII ( (a samples 4 4 (b samples 4 4 (c samples 4 4 (d Showing rejected (red and accepted (blue samples Importance sampling (I/VIII (8 Algorithm Importance sampler (IS. Generate i.i.d. samples {z i } i= from the proposal distribution q(z.. Compute the importance weights w i = π(zi q(z i, i =,...,.. ormalize the importance weights w i = wi j= wj, i =,...,. Importance sampling (II/VIII (8 An alternative interpretation of IS IS does not provide samples from the target distribution, but the samples {z i } i= together with the normalised weights {wi } i= provides an empirical approximation of the target distribution, π(z = i= w i δ z i(z. When this approximation is inserted into I(g(z = g(zπ(zdz the resulting estimate is Î (g(z = i= w(z i g(z i.

4 Importance sampling (III/VIII (8 Importance sampling (IV/VIII 4(8 Let us revisit the same problem (scalar LGSS used in illustrating the EM algorithm, x t+ = θx t + v t, y t = x t + e t, ( vt e t (( (.,.. p(x = (x,.. The true parameter value for θ is given by θ =.9. We use p(θ = ( θ µ θ, σ θ as prior distribution for θ. The identification problem is now to determine the parameter θ on the basis of the observations y :T and the above model, using the IS algorithm. The result will be an estimate of the posterior distribution p(θ y :T. The importance sampler will target π(θ = p(θ y :T = p(y :T θp(θ p(y :T p(y :T θp(θ. Chose the proposal distribution to be the same as the prior, ( q(θ = θ µ θ, σθ. The importance weights are then computed according to i.e., the likelihood. w i = π(θi q(θ i = p(y :T θ i, i =,...,, Importance sampling (V/VIII (8 Importance sampling (VI/VIII 6(8 The Kalman filter straightforwardly allows us to evaluate the importance weights w i = p(y :T θ i, p(y :T θ i = T t= p(y t y :t, θ i = ŷ t t (θ i =. x t t (θ i, S t t (θ i =. P t t (θ i +., T t= ( y t ŷ t t (θ i, S t t (θ i, Algorithm Importance sampler targeting p(θ y :T. Generate i.i.d. samples from the proposal distribution, i.e. θ i ( θ µ θ, σθ for i =,...,.. Compute importance weights w i = p(y :T θ i for i =,...,.. ormalize the importance weights w i = wi for i =,...,. j= wj where x t t (θ i and P t t (θ i are provided by the Kalman filter.

5 Importance sampling (VII/VIII 7(8 Importance sampling (VIII/VIII 8(8 Using T = measurements, y :..6.6 p(θ y:t θ samples, q(θ = ( θ,. p(θ y:t θ samples, q(θ = ( θ,. p(θ y:t θ samples, q(θ = ( θ,. p(θ y:t θ samples, q(θ = ( θ., ote the different proposal distributions used above. The importance of a good proposal density 9(8 Using IS for our purposes (8 A first attempt to solve the nonlinear filtering problem, which amounts to compute p(x t y :t for x t+ x t f (x t+ x t, y t x t h(y t x t, x µ(x. q (x = (, (dashed curve samples used in booth experiments. q (x = (, (dashed curve Lesson learned (again: It is very important to be careful in selecting the importance density. The solution is given by p(x t y :t = h(y t x t p(x t y :t, p(y t y :t p(x t y :t = f (x t x t p(x t y :t dx t. Relevant idea: Try to solve this using importance sampling!!

6 IS algorithm again (8 Sequential sampling from the proposal (8 Algorithm 4 Importance sampler (IS. Generate i.i.d. samples {z i } i= from the proposal distribution q(z.. Compute the importance weights w i = π(zi q(z i, i =,...,.. ormalize the importance weights w i = wi j= wj, i =,...,. We have just motivated the following proposal (which is just one of many possible choices In practice this means: q(x :t = µ(x t k= At time t = we sample x µ(x. f (x k x k At each time k =,..., t we sample x i k f (x k x i k. This completes step one of the importance sampler. What about sequential computation of the importance weights? A sequential importance sampler (SIS (8 Algorithm SIS targeting p(x :t y :t. Generate initial samples x i µ(x, set the importance weights, w i = /, i =,..., and set k =.. for k = to t do (a Compute the importance weights w i k = p(y k x i k wi k. (b ormalize the importance weights w i k = wi k j= wj k and store the new weights { w i :k} i= = { w i :k, k} wi i=. (c Generate i.i.d. samples from the proposal distribution, x i k+ f (x k+ x i k and store the new samples { } x i = { :k+ x i i= :k, } xi k+ i=. SIS example (I/III 4(8 Consider the following LGSS model x t+ =.7x t + v t, v t (,., y t =.x t + e t, e t (,., p(x = (x,., We will now make use of the SIS algorithm to compute an approximation of the filtering distribution p(x t y :t = i= w i tδ x i t (x t. Study Point estimate x t t = x t p(x t y :t dx t = i= wi t xi t. The weights w i t.

7 SIS example (II/III (8 SIS example (III/III 6(8. Use T = samples, realisations of data and =, = and =, respectively x Compare with the true filter distribution (from KF. RMSE( x SIS x KF t t t t Importance weight index 4 Illustration of the problem in the weights importance weight index The importance weights at t = Histograms of the weights for t =,,, and t =, respectively. Very important question: How do we resolve this weight degeneracy problem? Solving the weight degeneracy problem 7(8 Resampling (I/II 8(8 The SIS representation of the target density is Idea: Remove the weights from the representation! This of course leads us to the next question, How? π (z = i= w i δ z i(z. An unweighted representation of the target density can be created by resampling with replacement. This is done by generating a new sample z i for each i =,...,, where ( P z i = z j = w j, j =,...,. The resulting unweighted representation is π (z = i= δ z i(z.

8 Resampling (II/II 9(8 Sampling importance resampling (SIR ( M w i. i=.4... Illustrating how resampling with replacement works (using 7 particles.. Compute the cumulative sum of the weights.. Generate u U[, ]. Algorithm 6 Sampling Importance Resampler (SIR. Generate i.i.d. samples { z i } i= from the proposal q(z.. Compute the importance weights w i = π( zi, i =,...,. q( z i. ormalize the importance weights w i = wi, i =,...,. j= wj 4. For each i =,..., generate a new sample z i, where ( P z i = z j = w j, j =,..., Particle index Three new samples are generated in the figure above, corresponding to sample, 4 and 4. ote that step corresponds to the importance sampler. Revisit the nonlinear filtering problem (8 Algorithm 7 A first particle filter targeting p(x :t y :t. Generate initial samples x i µ(x, set the importance weights, w i = /, i =,..., and set k =.. for k = to t do (a Compute the importance weights w i k = p(y k x i k wi k. (b ormalize the importance weights w i k = wi k j= wj k (c For ( each i =,..., generate a new sample x i t, where P x i t = xj t = w j, j =,...,. (d Generate i.i.d. samples from the proposal distribution, x i k+ f (x k+ x i k. PF example (I/V (8 Consider the same LGSS model used in illustrating the SIS algorithm, x t+ =.7x t + v t, v t (,., y t =.x t + e t, e t (,., p(x = (x,.. We will now make use of SIS + resampling (particle filter to compute an approximation of the filtering distribution p(x t y :t = i= w i tδ xt ( x i t. Study Point estimate x t t = x t p(x t y :t dx t = i= wi t xi t. The weights w i t.

9 PF example (II/V (8 PF example (III/V 4(8 Same setting as before, exactly the same data.. Compare with the true filter distribution (from KF, RMSE(b xpf b xkf t t t t Importance weight index Importance weight index SIS 6 4 PF ote the different scaling! SIS PF PF example (IV/V (8 PF example (V/V 6(8.7 x x.4 x x x x importance weight index 4 importance weight index 4 SIS PF PF ote the different scaling! 6 x SIS

10 An LGSS example (I/II 7(8 Whenever you are working on a nonlinear estimation algorithm, always make sure that it solves the linear special case first. Consider the following LGSS model (simple one dimensional positioning example p t+ T s T s / p t v t+ = T s v t a t+ a t ( t y t = p + e t, v t a t + ( T s /6 T s /T s v t, v t (, Q, e t (, R. The KF provides the true filtering distribution, which implies that we can compare the PF to the truth in this case. An LGSS example (II/II 8(8 ˆp P F ˆp KF (m ˆv KF ˆv KF (m/s (s (s Using particles. ˆp P F ˆp KF (m ˆv KF ˆv KF (m/s (s (s Using particles. The PF estimate converge as the number of particles tends to infinity. Xiao-Li Hu, Thomas B. Schön and Lennart Ljung. A Basic Convergence Result for Particle Filtering. IEEE Transactions on Signal Processing, 6(4:7-48, April 8. [pdf] D. Crisan and A. Doucet, A survey of convergence results on particle filtering methods for practitioners, IEEE Transactions on Signal Processing, vol., no., pp ,. [pdf] A nonlinear example (I/II 9(8 A nonlinear example (II/II 4(8 Consider the following SSM (standard example in PF literature x t+ = x t + x t + x t + 8 cos(.t + v t, v t (,., y t = x t + e t, e t (,.. What it tricky with this model? The best (only? way of really understanding something is to implement it yourself (s True state (gray and PF estimate (black State x 6 PF estimate of the filtering pdf p(x 6 y :6.

11 . Xdin of sensors and a IEEE 8..4 radio with a radio chip and an Ethernet chip. port, serving as a base station for the Beebadges. Figure.. The two main components of the radio network. onlinear example indoor positioning (I/III 4(8. Map onlinear example indoor positioning (I/III Aim: Compute the position of a person moving around indoors using sensors located in an ID badge and a map. 4(8 Decay for different n n=, m= n=, m= n=4, m= Relative probability The sensors (IMU and radio and the Figure.. Beebadge worn by a man. DSP are mounted inside an ID badge. pdf for an office environment, the (a Relative probability density for parts of bright areas are rooms and corridors Xdin s office, the bright areas are rooms and (a A Beebadge, carrying a number (b A coordinator, both (i.e., walkable the bright lines equipped are space. corridors that interconnect The inside of the ID badge.. (a An estimated trajectory at Xdin s of- (b A scenario where the filter hav Position [m]... An estimated trajectory and the converged yet. The spread in hypot fice, particles represented as circles, size section of a circle indicates weight of the is caused by a large coverage for a co (b Cross of the the relative probparticle cloud visualized at a particle. nator. ability function for a line with differparticular instance. ent n Figure 4.. Output from the particle filter. of sensors and a IEEE 8..4 radio with athe radio chip and an Ethernet rooms chip. port, serving as a base station for the Beebadges. Figure.7. Probability interpretation of the map. Figure.. The two main components of the radio network. onlinear example indoor positioning (III/III those would suffice to give a magnitude of the force. The force is intuitively 4(8 directed Sequential (SMCthe target moreand general orthogonallymonte from thecarlo wall towards multiple forces 44(8 can be added together to get a resulting force affecting the momentum of the target. Equation (.9 describes how the force is constructed. The function wallj (p Assume that at time t we have a particle system {xi:t, wit } givencase 4.. Illustration of ai= problematic is a convex function giving the magnitude and Figure direction of the force thewhere a correct trajectory ( an incorrect trajectory approximating the target distribution p(xbeing y:t byaccording to (red, causing the filter to potentially :t starved position of the target, p. fi = Show movies This is work by Johan Kihlberg and Simon Tegelid in their MSc thesis entitled Map Aided Indoor Positioning, which is available for download. [pdf] wi b p(x:tj (p iy,:t where = W is tthel δset x:twalls.. wall xi:t (of i= l= wt jœw ÿ (.9 If positions fromsteps otherintargets are algorithm available, are repellent forces from them can be The three the SMC modeled as well, which is thoroughlyi discussed in []. The {e concept is visualized. Resampling: { x, w } x, / } :t :t t i = i= and another target in Figure.8 where the target Ti is affected by two walls i. in Propose new xi:t for Tm, resulting the force fi.samples: xt+ Qt (xt+ e i =,...,.. Weighting: wit+ = Wt (xit+, e xi:t for i =,...,. SMC is a combination of SIS and resampling. Figure.. Beebadge worn by a man.

12 SMC algorithm 4(8 Algorithm 8 Sequential Monte Carlo (SMC. Initialise by sampling x i setting t =. Q (x, setting w i = W (x i and. for t = to T do (a Resample: {x i :t, wi t } i= { xi :t, /} i=. (b Propose new samples: x i t+ Q t(x t+ x i :t. (c Weighting: w i t+ = W t(x i t+, xi :t. There is a myriad of possible design choices available, some of them carrying their own name. The important thing is to understand the basic principles in action, then the rest is just design choices. SMC reformulated 46(8 otation: w t {w t,..., w t }. Algorithm 9 Sequential Monte Carlo (SMC. Initialise: Sample x i Q (x and set w i = W (x i. Set t =.. For t = : T do: (a Resampling: a i t R(a t w t. (b Sample from the proposal kernel: x i t Q t(x t x ai t :t and set x i :t = {xai t :t, xi t }. (c Weighting: w i t = W t(x i t, xai t :t. Here: a i t to denote the index of the parent/ancestor at time t of particle x i t. SMC as a random number generator 47(8 Just like any algorithm that is used to generate random numbers there is an underlying distribution also for the SMC sampler that encodes the probabilistic properties of the involved stochastic variables. The SMC sampler generates a realisation of the random variables X :T = {X:T,..., X :T } and A :T = {A :T,..., A :T }, where the pdf is { } ψ(x :T, a :T = Q (x i T R(a i t w t Q t (x i t x ai t }{{} i= }{{} t= i= Resampling initialisation and it is defined on the space X T {,..., } (T. :t }{{} Proposing new particles Sampling a state trajectory 48(8 The SMC algorithm produce the following approximation of the target distribution π (x :T = i= W i T δ X i :T(x :T. We can use π (x :T to produce samples of the state trajectory by sampling from [ ] q (x :T = E φ π (x :T = WT i δ X:T(x i :T ψ(x :T, A :T dx :T da :T. i= ote that this integration can never be carried out explicitly. Hence, we cannot evaluate this distribution for a specific sample, but it can be used to generate samples from it.

13 Illustration of the particle degeneracy problem 49(8 Implication of the particle degeneracy problem (8. State... This implies that if we are interested in the smoothing distribution p(x :T y :T or some of its marginals we are forced to use different algorithms, which leads us to particle smoothers.. 4 Particle smoother (8 Algorithm Forward filtering/backward smoothing (FFBSm. Run the PF and store the particles and their weights {(w i t, xi t } i= for t =,..., T.. Initialise the smoothed weights w i T T = wi T, for i =,...,.. for t = T to do (a Compute the smoothed weights w i t T = wi t w i t+ T k= f (x k t+ xi t, v k t = v k t i= w i tf (x k t+ xi t. The resulting approximation p(x t y :T = i= wi t T δ x i t (x t does not suffer from degeneracy. Sampling from the backward kernel (8 Key idea: Generate a backward trajectory by sampling from the following approximation p(x t x j t+, y :t = i= w i t f ( xj t+ xi t k= wk t f ( xj t+ xk t }{{ δ x i (x t t } w i,j t T of the backward kernel. The resulting sample trajectory is a sample from the joint smoothing density p(x :T y :T. Still computationally very costly! x j t p(x t x j t+, y :t, x j t:t = { xj t, xj t+:t }.

14 Sampling from the backward kernel (8 Sampling from the backward kernel - in words 4(8 Algorithm Backward simulator (BSi. Initialise: Set b T = m with probability w m T / l= wl T.. For t = T : : do: (a Given x b t+ t+, compute the smoothing weights, w m t T = wm t f (xb t+ t+ xm t l= wl t f (xb t+ t+ m =,...,. xl t, (b Set b t = m with probability w m t T. The BSi samples a trajectory from the empirical joint smoothing density by first choosing a particle x m T with probability proportional to w m T and then generating the ancestral path b :T according to the backward kernel. The resulting trajectory is x :T = {x b,..., xb T T }. The computational cost is O(, which is too expensive to be practical. Sampling from the backward kernel - in pictures (8 Sampling from the backward kernel - in pictures 6(8 Recall that the main problem with using the PF to sample from p(x :T y :T is the particle degeneracy problem. State In the figures we show two samples from p(x :T y :T generated using the PF. The diversity is completely gone from time and backwards. The backward simulator resolves the particle degeneracy problem. State ote that in the above samples there is no loss of diversity. 4

15 Forward filtering/backward simulation (FFBSi 7(8 Towards a practical particle smoother 8(8 Algorithm Forward filtering/backward simulation (FFBSi. Run the PF and store {(w m t, xm t } m= for t =,..., T.. Initialise: Set the smoothed weights w m T T = wm T and set t = T.. For t = T : : do: 4. For j = : M do: (a Set v j t = k= wk t f ( xj t+ xk t (b For m =,...,, compute w m,j t T = wm t f ( x j t+ xm t. v k t (c Sample from( the empirical backward kernel, I(j Cat { w m,j t T } m=, x j t = xi(j t, and append the sample x j t:t = { xj t, xj t+:t } To obtain a practical algorithm we have to reduce the computational complexity, how?! Key insight: We do not have to compute all the smoothing weights } to be able to sample from the empirical backward kernel. { w m,j t T How: Use rejection sampling to sample the indices! This means that we should not have to compute indices just to draw one sample. For details and MATLAB code, see Algorithm.4 (Fast FFBSi in F. Lindsten, Rao-Blackwellised particle methods for inference and identification. Licentiate thesis, Linköping University, LiU-TEK-LIC-:9,. [pdf] Original reference: R. Douc, A. Garivier, E. Moulines, and J. Olsson. Sequential Monte Carlo smoothing for general state space hidden Markov models. Annals of Applied Probability, (6:9 4,. [pdf]