A Comparison of Particle Filters for Personal Positioning

Transcription

1 VI Hotine-Marussi Symposium of Theoretical and Computational Geodesy May 9-June 6. A Comparison of Particle Filters for Personal Positioning D. Petrovich and R. Piché Institute of Mathematics Tampere University of Technology P.O. Bo Tampere Finland Abstract. Particle filters also nown as sequential Monte Carlo methods are a convenient and popular way to numerically approimate optimal Bayesian filters for nonlinear non-gaussian problems. In the literature the performance of different filters is often determined empirically by comparing the filter s conditional with the true trac in a set of simulations. This is not ideal. Because these filters produce approimations of the optimal Bayesian posterior distribution the comparison should be based on the quality of this approimation rather than on an estimate formed from the distribution. In this wor we apply a multivariate binning technique to compare the performance of different particle filters. In our simulation we find that the conclusions of the distribution comparison are similar to the conclusions of a root square error analysis of the conditional estimate. Keywords. Sequential Monte Carlo Particle Filter Bayesian estimation Introduction Particle filters (PFs implement the recursive Bayesian filter with Monte Carlo (MC simulation and approimate the posterior distribution by a set of samples with appropriate weights. This is most attractive in nonlinear and non-gaussian situations where the integrals of Bayes recursions are not tractable. In the literature many PF simulations focus on the MC variation of the estimates i.e. the randomness introduced by the MC algorithm that can be observed from the empirical estimates. Better PFs vary less. In this wor we develop a method to compare PF performance that uses the distribution approimation itself rather than a single estimate formed from it. A distribution analysis could be more useful than an estimate analysis. As eamples two filters could give similar estimates although one of them has a distribution closer to the true distribution. Also in a bimodal case the could be between the two modes in a region of the state-space where there is little probability of the target being located; in a case such as this the is less interesting to analyze and we are more concerned if our filter appropriately characterizes this bimodal characteristic. We will discuss one proposed method for comparing distributions and then apply this method to the comparison of four PFs (SIR SIR SIR3 SIR4 described in the Appendi. Comparing Distributions with χ -tests An interesting application of distribution comparisons was given by Roederer et al. ( in the field of cytometry. Test samples i.e. sets of multidimensional data are to be raned according to their similarity to a control sample which is a sample of data chosen to represent some nown behavior. A multivariate data-dependent binning technique was proposed that adaptively constructed bins according to the control sample followed by the use of a test statistic to quantify the difference between the test and control sample. Baggerly ( provides a more theoretical discussion of this approach with the recommendation to use the standard two-sample χ - test statistic. The form of the computed test statistic is (Baggerly ( B ( oi ei ψ = i= = ei where B is the number of bins o i is the observed count of the th bin of the i th sample set and e i is the epected count of the th bin of the i th sample set given by ( o + o ei = ni ( n + n where n i is the number of samples in set i. The algorithm of Roederer et al. ( that is used for constructing the bins is called probability binning and is as follows. The variance of the control sample along each of the d dimensions is computed and the dimension with the largest variance is chosen to be divided. The sample median value of the chosen dimension is then chosen as the point at which to partition the state-

2 space in two. This is then repeated for each partitioned subspace continuing until the desired number of bins has been reached. The result is a set of d-dimensional hyper-rectangular bins with sides parallel to the coordinate aes and each bin containing roughly the same number of control samples see Figure. Assuming then that a test sample is from the same distribution as the control sample each bin will roughly have the same epected frequency. The two-sample test statistic is then approimately χ B distributed i.e. approimately distributed according to a χ distribution with B- degrees of freedom. The binning of the PF samples is done as follows. At each time step we simulate M=N= 4 samples using multinomial sampling from the importance distribution evaluate weights and sip the final resampling step. The binning at each time step uses the weighted samples. We draw 4 samples using systematic resampling from this weighted discrete distribution bin the equallyweighted samples compute the test score and discard these resampled points. If the weights are equal then the binning can proceed without resampling. To verify that the test score analysis is ingful we can compare our conclusions to those made from a root square error (RMSE analysis of the estimates. The RMSE is given as RMSE = Ε ˆ μ μ ( Fig. Probability Binning in R with B=3 3 Application to Linear and Gaussian Filtering Scenarios Assume for a moment that we are able to generate IID samples from the true marginal posterior. We can then apply the probability binning procedure at each time step so that the state-space R n is partitioned into bins using a sample from the true marginal posterior. It might be reasonable then to assume that the quality of a PF approimation of the marginal posterior can be assessed using the twosample χ -test where the hypothesis is that the samples from the true marginal posterior and the samples from the PF approimation are from the same distribution. Roughly speaing we might epect better PFs to give better test scores where better test scores refers to realizations of a random variable with distribution closer to the assumed χ distribution. We chec this empirically by repeating the simulation times and comparing the of the realized test scores to that from the assumed χ distribution. For a χ B distribution the is B-. All of our simulations use B=64. where μˆ is the random posterior estimate and μ is the true posterior. The epectation in equation is approimated by repeating the simulation 5 times and using the sample. Due to lac of space we have omitted our linear- Gaussian simulations and proceed directly to the general filtering scenario. The interested reader is referred to Petrovich (6. 4 Application to General Filtering Scenarios We would lie to apply this distribution analysis to PFs in the general filtering scenario where we do not have an analytic form for the posterior. The difficulty is then to determine a control sample that can be used to partition the state-space. Ideally we would have some algorithm that was nown to produce IID samples from the marginal posterior that we could use to produce the control sample. In the absence of such an ideal algorithm we have chosen one PF algorithm as a reference. The partitioning of the state-space using a reference PF is done as follows. We have used the SIR3 importance distribution and at each time step draw M= 7 samples from the importance distribution evaluate weights and then resample N= 4 samples from this weighted distribution. The importance sampling uses deterministic sampling ( 3 samples from each miture component and the resampling uses systematic sampling. The binning at each time step uses the M weighted samples. We draw 6 samples from this weighted discrete distribution perform the probability binning on these equally-weighted samples and discard these resampled points.

3 It should be mentioned that the simulation scenario of this section is the same simulation scenario as that in Heine (5 i.e. the same signal and measurement models and the same realized measurement and signal process. We have reproduced the results (i.e. the relative RMSE plots and review the conclusions of that publication here for convenience. The contribution of this wor is the application of the probability binning method to the comparison of PFs. 4. Simulation Description For this simulation the state is in R 4 with two position coordinates r and two velocity coordinates u i.e. e n e n T = [ r r u u ]. The evolution of the state in continuous-time can be described by a white noise acceleration model see e.g. Bar-Shalom and Li (998. The discretized state equation for each [ r u ] T coordinate is written as Δt = F + v F = where Δt = t -t - and v - Ν( Σ v 3 3 Σ v = γ. Δt Δt Δt Δt with We simulate a second traectory with a constant time step of Δt = second and γ = 3 m /s 3. The state's initial distribution P is Gaussian and nown. Three base stations are used and at each time step one base station produces a range measurement. The range measurement of the form b y = r r + z z ~ Ν( σ where r b is the nown position of a base station. In the simulation of the observation process the probability of a base station producing a measurement is inversely proportional to the squared distance to the target. For each filter four separate simulations were run each using a different variance σ for the noise in the measurement model: 4 5. m. The true posterior μ in the RMSE analysis see equation is approimated using the same reference PF that was used for creating the bins. 4. Results We first consider an analysis of the RMSE of the estimates. The RMSE values are divided by the SIR RMSE values and plotted in Figure. These relative RMSE values show how the MC variation of estimates for the different filters compare to SIR: values lower than one indicate improved performance compared to SIR (less MC variation while values above one indicate worse performance (more MC variation. With the largest noise variance (σ = 4 m all the filters seem to have similar MC variance. With σ =5 m the SIR and SIR4 filters clearly have less MC variation while for cases with smaller measurement noise variance (σ = and. m the SIR3 and SIR4 filters have less MC variance. Also the SIR behaves erratically at the smaller measurement noise variance. Due to weights summing to zero in the PF algorithm on the σ =. m case SIR is averaged over only 998 realizations and SIR is averaged over only 365 realizations for that case Fig. Relative RMSE w.r.t. SIR for nonlinear- Gaussian scenario. From top-to-bottom σ = 4 5. m. In Figure 3 the test score is plotted over the whole simulation. It is interesting to note that the test scores rarely resemble the χ theoretical distribution. However there appears to be different behavior for the different filters. Note that all the results are rather similar with large σ and the differences become more apparent as σ decreases. Also note that for σ = 4 and 5 m there is near time SIR SIR SIR3 SIR4

4 identical results for the SIR and SIR3 filters and for the SIR and SIR4 filter. For σ =. m the SIR score s are quite large and are outside of the plotted region. The intention of these plots was not to display the actual score s but instead to show the relative performance of the different filters. For cases with larger measurement noise variances (σ = 4 and 5 m the two filters using alternative weights in the importance distribution (SIR and SIR4 have smaller s. For cases with smaller measurement noise variances (σ = and. m the two filters using alternative components in the importance distribution (SIR3 and SIR4 show smaller s while the SIR results behave erratically. It might be reasonable then to conclude that the filters having test scores closer to the theoretical distribution χ B i.e. having smaller test score s are woring better Fig. 3 Mean of two-sample χ test statistic for nonlinear-gaussian scenario. From top-to-bottom σ = 4 5. m. In summary similar conclusions about the relative performance of the different PFs can be found using alternative criteria i.e. the distribution analysis and the RMSE analysis of the estimates. The empirical comparison of different PFs using χ techniques seems to be feasible even in scenarios where the state-space partitioning relies on a PF. 5 Conclusions In this wor we applied a multivariate binning technique from Roederer et al. ( to the time SIR SIR SIR3 SIR4 comparison of PFs. This was described for a linear and Gaussian filtering scenario where we have an analytical form of the marginal posterior and also for a nonlinear and Gaussian filtering scenario where we estimated the optimal solution with a PF. The conclusions resulting from the proposed test were similar to the conclusions from an RMSE analysis of the estimates. We have not offered a detailed discussion of the practical implementation aspects of such a test. It should be mentioned that our implementation of the test i.e. the construction of the bins and the actual binning of the samples used data structures similar to Kd-trees see e.g. de Berg et al. ( and was computationally feasible for the cases that we considered. The literature on the χ -test is vast and admittedly our treatment of the test has been brief. In this section we point out some questionable aspects of our use of this test for comparing PFs. First we should question the use of the χ -test itself. We are testing whether the two samples are from the same distribution although it was already noted in Pitt and Shephard (999 that the methods will not produce IID samples from the true posterior due to the finite miture approimation. In spite of this we have still considered the test scores as a way to empirically quantify the difference between distributions. Second we should question how the χ -test was actually performed. The number of bins and the construction of the bins in χ -tests is not always straightforward and is therefore debatable. The resampling that is carried out before binning the PF samples is also questionable; due to the etra resampling our test is then comparing an approimation of the PF approimation which complicates the analysis. Finally our use of a large sample PF to approimate the optimal posterior as well as our choice of importance distribution for this PF is not properly ustified; other methods such as reection sampling or MCMC might result in a better approimation of the posterior. The intention of the distribution comparison was to devise better ways of comparing PFs and possibly other Bayesian filters. The binning procedure that was described is of course limited to sample-based methods. Other possible methods could include integrating the posterior distribution over a finite number of regions and then using some distance e.g. Kullbac Leibler divergence or total variation distance on the resulting finite state-space to quantify the distance between the distributions. However the choice of distance is then quite arbitrary and there is little reason to prefer one distance over another. This seems to be an interesting area for future wor.

5 References Baggerly K.A. (. Probability Binning and testing agreement between multivariate immunofluorescence histograms: Etending the chi-square test. Cytometry Vol. 45 pp De Berg M. M. van Kreveld M. Overmars and O. Schwartzopf (. Computational Geometry: Algorithms and Applications. Springer nd Ed. Heine K. (5. Unified framewor for sampling/importance resampling algorithms. In Proceedings of Fusion 5. Pitt M.K. and N. Shephard (999. Filtering via simulation: Auiliary particle filter. In Journal of the American Statistical Association Vol. 94 No. 446 pp Petrovich D. (6. Sequential Monte Carlo for Personal Positioning. M.Sc. Thesis submitted to: Tampere University of Technology. Roederer M. W. Moore A. Treister R. Hardy and L.A. Herzenberg (. Probability binning comparison: a metric for quantitating multivariate differences. Cytometry Vol. 45 pp Appendi In this section we briefly describe the four different importance distributions used for the PFs tested in this wor. The system equations are typically given as = f ( + v y = h ( + z where and y are the values of the signal and observation stochastic process respectively at time. Furthermore v - denotes the state of the signal noise process at time - and z denotes the state of the observation noise process at time. We follow the auiliary formulation of Pitt and Shephard (999. Assuming that we have N particles approimately distributed according to P the algorithm is as follows: For = For i=:n assign st i -state weights β and normalize For i=:m sample inde i from discrete distribution of st -stage weights i For i=:m sample state conditioned on inde i and all received measurements y : For i=:m evaluate nd -stage weights and normalize Resample N particles The general form of the unnormalized second-stage importance weight can be written as w( q( w y y : : y β q where is the auiliary variable ( β is the th firststage weight and q is the th importance density. Note that for notational convenience we have dropped the conditioning on the measurements for the first-stage weight and importance density. The four different importance distributions result from different choices of β and q. SIR results from using β = w and q =. SIR is an eample from Pitt and Shephard (999 that uses β w y ξ and q = p where is the of the ( ξ distribution of. Note that in the literature this eample is often referred to as the auiliary particle filter. SIR3 again uses β = w but now uses an EKF for each importance distribution i.e. q = ν ( ; μ C where ν is a density of μ a Gaussian distribution and the and th covariance C are given by the posterior of the EKF. SIR4 uses the same importance distribution as SIR3 but has different first-stage weights given by β = w c where c = ν ( y ; m Ξ m = h ( f ( ˆ ( ˆ T Ξ = H Σ H + Σ Ĥ v z where is the Jacobian matri of h evaluated at f ( Σ v is the covariance matri of the signal noise v- Σ z is the covariance of the observation noise z.