RESEARCH ARTICLE. Online quantization in nonlinear filtering

Journal of Statistical Computation & Simulation Vol. 00, No. 00, Month 200x, 3 RESEARCH ARTICLE Online quantization in nonlinear filtering A. Feuer and G. C. Goodwin Received 00 Month 200x; in final form 00 Month 200x We propose a novel approach to nonlinear filtering utilizing on-line quantization. We develop performance bounds for the algorithm. We also present an example which illustrates the advantages of the method relative to related schemes which utilize off-line quantization. Keywords: Nonlinear filter, Vector quantization, Kalman filter AMS Subject Classification 93E, 93E0, 93E25, 93C0. Introduction Sequential Baysian filtering arises in many practical problems. The complexity of this problem depends very much on the underlying mathematical model. When a linear Gaussian model is assumed, the well known Kalman filter provides the desired optimal solution. However, in many situations these assumptions do not hold, even approximately. We consider here a somewhat simplified version of the general set up. Namely, we have the following state space model X k F X k + W k, Y k G X k + V k, where X k R d and Y k R m are the system state and measurements respectively, and W k R d and V k R m are process and measurements noises, typically assumes i.i.d sequences independent of each other, with Gaussian probability density functions pdf p w w N 0, Q and p v v N 0, R. It is also assumed that the initial state is a random vector with pdf p 0 x 0 N 0, Q 0 and that the functions F and G are known. One would like to be able to estimate, at time t > 0, a function f : R d R of the state, f X t, given the measurements Y t {y k : k,..., t}. This estimate, which is the filtering operation, is given by Π y,t f E {f X t Y t } f x t p t x t Y t dx t. 2 Department of EE, Technion, Haifa, Israel. Email: feuer@ee.technion.ac.il School of Electrical Engineering and Computer Science, University of Newcastle, Newcastle, Australia. Email: graham.goodwin@newcastle.edu.au ISSN: 0094-9655 print/issn 563-563 online c 200x Taylor & Francis DOI: 0.080/0094965YYxxxxxxxx http://www.informaworld.com

2 Clearly, to be able to calculate this estimate one needs to know the pdf p t x t Y t. To this end one could use iteratively the Chapman-Kolmogorov equation see e.g. [] or [2]: Suppose that at time k the pdf p k x k Y k is available. Using the model equation we can derive the prior pdf p k x k Y k p k x k x k p k x k Y k dx k, 3 where see [3] p k x k x k p ω x k F x k. 4 At time step k the new measurement y k becomes available so that it can be used to update the pdf where p k x k Y k p k y k x k p k x k Y k, 5 p k y k Y k p k y k x k p v y k G x k 6 and p k y k Y k p k y k x k p k x k Y k dx k. 7 While the above expressions provide a theoretical framework, only a very limited number of special cases can be directly solved this way. A prime example is the linear Gaussian case for which closed form expressions can be derived resulting in the well known Kalman filter. However, for the general non-linear filtering problem no exact solutions can be derived hence the need for numerical approximation methods. There are many such methods in the literature - we refer the reader to comprehensive surveys and discussion in [], [2] and the references therein. Common to all these approximation methods is replacing the integral in 2 with a finite sum of the form Π y,t f E {f X t Y t } N t P t,i f x i t. 8 i These methods differ in the way the values { x i } Nt t i Rd and {P t,i } Nt i are generated. One class is based on using a Monte Carlo random approach, commonly referred to as Particle Filtering, while another class uses deterministic considerations. Let us consider, for a moment, the estimation error E {f X N t t Y t } P t,i f x i t i N t f x t p t x t Y t dx t P t,i f x i t. Since, for every choice {P t,i } Nt i, there exist a tiling {A t,i} Nt i such that P t,i i

A t,i p t x t Y t dx t by tiling we mean that A t,i A t,j for every i j and Nt i A t,i R d, we can write E {f X N t t Y t } P t,i f x i t N t f xt f x i t pt x t Y t dx t A t,i i i N t f x t f x i t p t x t Y t dx t, A t,i i and assuming f to be Lipshitz, namely, f xt f xt i [f]lip xt x i t, we obtain the bound E {f X N t t Y t } P t,i f x i N t t [f] xt Lip x i t pt x t Y t dx t. 9 A t,i i The bound derived above for the estimation error is readily recognized as the distortion in a quantization problem for the random variable X t with the pdf p t x t Y t, encoder {A t,i } N t i and decoder or code book { x i } Nt t. This clearly provides a motivation of applying optimal quantization as a means of obtaining the desired i approximation 8. In light of the above it hardly is surprising that an approach in the deterministic class, using vector quantization, has been suggested in [4]. We will describe this approach in some detail in the next section, quote some of the results and point to a potential weakness. This provides the basis for the novel method we propose here which is also based on vector quantization but, as we argue later, does not suffer from the same potential weakness. In [5] a comparison of the quantization methods described in [4] is presented and compared to particle filtering. The comparison indicates an advantage of the quantization approach over particle filtering with the same size grids especially for small grid size. Given this comparison, we focus our attention on quantization methods and demonstrate the advantage of our approach over the ones presented in [4]. In the sequel we assume that the reader has some familiarity with optimal vector quantization and suggest [7] as a reference on the subject. i 3 2. Quantization in nonlinear filtering Before we describe the quantization approaches it will be helpful to use 3-7 to rewrite 2 as follows: Π y,t f π y,tf π y,t E {f X t L X t, Y t }, 0 E {L X t, Y t } where we recall that Y t {y k : k,..., t} while X t {X k : k,..., t} is the set of random states, and L X t, Y t t p k y k X k. k0

4 We further denote π y,t E {L X t, Y t } φ t Y t. 2 Note that π y,t f is commonly referred to as the unnormalized filter. 2.. Marginal quantization Here we briefly describe the Marginal Quantization filtering algorithm of [4]. Using the sequence of pdfs, p k x k for the Markov chain {X k } t k0 can be calculated off-line and optimally quantized, resulting in the set of grids {Γ k {x k,,..., x k,n }} N k0 and the corresponding Voronoi cells {V k,,...v k,n } N k0 see [7]. One could choose different grid sizes for different times. However, for simplicity, we choose the same size for all times. These grids are now used to define the sequence X k proj Γk X k for k 0,..., t, 3 where we note that p k x k Y k has the form N j P k,jδ x k x k,j. Let us define the matrix H ij k p k y k X [ k x k,j Prob Xk x k,j X ] k x k,i p k y k X k x k,j p k x x k,i dx V k,j p k y k X k x k,j p ω x F x k,i dx, 4 V k,j and denote P k [P k,,..., P k,n ] and f Xk [f x k,,..., f x k,n ] T. Then, using Bayes rule we obtain as the estimate of the unnormalized filter π y,t f P t H t f Xt P 0 H H t f Xt, 5 where P 0 [P 0,,..., P 0,N ] T and P 0,i V 0,i p 0 x 0 dx 0. The estimate of the filter then is given by where { } Π y,t f E f Xt Y t π y,tf π y,t, 6 π y,t T t H k P 0 k φ t Y t. 7

5 Markovian quantization [4]: An alternative approach described in [4] is achieved by defining the Markovian process, with X 0 X 0 X k Proj Γk Xk, 8 X k F Xk + W k, Ŷ k G Xk + V k. 9 The grids Γ k here are generated by optimal quantization of the pdfs p k Xk which are again, calculated off-line. With this setup we can imitate 0 and define L Xt, Y t t k0 p k y k X k 20 { } π y,t f E f Xt L Xt, Y t 2 and { } Π y,t f E f Xt Y t π y,tf π y,t 22 with } π y,t E { L Xt, Y t φ t Y t. 23 2.2. On-line quantization We start with the observation that if our goal is to use optimal quantization to provide an estimate of the form N j P t,jδ x t x t,j of the pdf p t x t Y t or equivalently, an estimate of the filter 8, this is the pdf we need to quantize. Namely, both the grid and the respective weights should be data dependent and not calculated off-line. This, we feel, is the potential weakness inherent to both methods described above. On the other hand, any such attempt needs to be computationally feasible. The method we propose we believe yields an appropriate compromise for certain cases of practical importance.

6 We consider the Markovian process, X 0 X 0 and X k F Xk + Ŵk, 24 Ỹ k G Xk + V k, X k Proj Γk Xk Y k, 25 where Ŵk Proj Γw W k which is done off-line and is an optimal quantization of p w w resulting in a grid of size N w. Hence, N w pŵ w P w,s δ w w s. 26 s At each time k we have p k Xk Y k then and p k Xk Y k N i P k,iδ x k x k,i, N N w P k,i P w,s δ x k F x k.i w s i s N N w r P k,r δ x k x k,r, 27 p k Xk Y k N Nw r p k y k X k x k,r Pk,r δ x k x k,r N Nw r p k y k X k x k,r Pk,r N N w r Then, the approximate filter is P k,r δ x k x k,r. 28 { } Π y,t f E f Xt Y t N N w r P k,r f x k,r. 29 We note that p k Xk Y k which is defined on an N N w grid, can be calculated on-line at each time k, optimally quantized to provide us with the grid Γ k to be used to generate X k. The intuitive benefit in the proposed approximation is clear. We will in the sequel, attempt to support this intuition with analysis. Remark : The optimal quantization of X k, which is a discrete valued random vector, is in fact a clustering problem for which one could apply one of many available fast algorithms see e.g. [6].

7 3. Error analysis For our analysis we make the following assumption. Assumption : The functions F : R d R d, G : R d R d are Lipschitz with ratios [F ] Lip, [G] Lip respectively, and the function f : R d R is both bounded and Lipshitz with bound f and ratio [f] Lip. Remark 2: For example, we note that the Gaussian pdf N 0, Q in R d is Lipschitz with ratio eλ min Q 2 for 2π d 2 Q 2 2. Hence, since x 2 x and p k y k X k x N G x, R, by Assumption we have p k y k X k x p k y k X k x eλ min R 2 2π m 2 R 2 G x G x 2 eλ min R 2 [G] Lip 2π m 2 R 2 x x 2 [p ] Lip x x. 30 Furthermore, we also note that p k y k X k x We can now write the approximate filter 29 as where X { } t Xk : k,..., t, 2π m 2 R 2 K. 3 Π y,t f π y,tf π y,t { } E f Xt L Xt, Y t }, 32 E { L Xt, Y t L Xt, Y t t k0 p k y k X k, 33 and } π y,t E { L Xt, Y t Y t φ t Y t. 34 Our derivation closely imitates the one presented in [4]. We quote first the following Lemma 3. from [4] : Lemma 3.: Let µ y and ν y be two families of finite positive measures on a measurable space E, E. Assume that there exist two symmetric functions R and

8 S defined on µ y and ν y such that for every bounded Lipschitz function f, fdµ y then fdµy µ y E fdνy ν y E max µ y E, ν y E fdν y f R µ y, ν y + [f] Lip S µ y, ν y, 35 We are ready now to state our result on the error bound. 2 f R µ y, ν y + [f] Lip S µ y, ν y. Theorem 3.2 : Let Assumption hold, then, for every bounded Lipschitz continuous function f : R d R with ratio [f] Lip and sequence of measured data Y t {y,..., y t } generated by eqn., the error of the estimator 29 satisfies Π y,t f Π y,t f where K t max φ t Y t, φ t Y t 36 t D t E { w } + Ck t f, Y t E { k }, k0 37 Ck t f, Y t [f] Lip [F ] t k Lip + 2 f K [p ] Lip [F ] [F ] t k Lip Lip [F ] Lip, 38 and [F ] t Lip D t [f] Lip [F ] Lip [F ] Lip + 2 f K w W Ŵ [p ] Lip W Proj Γw W, k X k Proj Γk Xk Y k t k k [F ] t k Lip, 39 X k X k. 40 Note that E { w }, E { k } are the respective quantization distortions. Proof : We begin by applying 0 and 32 to π y,t f π y,t f { } E {f X t L X t, Y t } E f Xt L Xt, Y t { } Xt, Y t E L X t, Y t L f X t { } +E f X t f Xt L Xt, Y t { L f E Xt, Y t L } Xt, Y t + [f] Lip K t E { Xt X t }, 4

where we have used Remark 2 and 33 to conclude that L Xt, Y t K t. Applying Lemma 3. we obtain 9 Π y,t f Π y,t f max φ t Y t, φ t Y t + [f] Lip K t E { L 2 f E Xt, Y t L } Xt, Y t { Xt X t }. 42 Using and 33 we obtain L X k, Y k L Xk, Y k p k y k X k L X k, Y k p k y k X k L Xk, Y k p k y k X k p k y k X k L X k, Y k +p k y k X k L X k, Y k L Xk, Y k, so that L X k, Y k L Xk, Y k K k Xk [p ] Lip X k +K L X k, Y k L Xk, Y k. Since L X 0, Y 0 L X0, Y 0 we obtain from the above that L X k, Y k L k Xk, Y k K k [p ] Lip X j X j. 43 j From, 24, 25 and 40 we have X k X k F X k F Xk + W k Ŵk [F ] Lip Xk X k + w k [F ] Lip Xk X k + k + w k. Noting that X 0 X 0, we conclude that X k X k k j [F ] k j Lip [F ] Lip j + w j. 44

0 Substituting 44 into 43 we obtain L X k, Y k L k Xk, Y k K k [p ] Lip j j r [F ] j r Lip [F ] Lip r + w r 45 and substituting both 44 and 45 in 4a, using the fact that E { w r } E { w } does not depend on time, concludes the proof of the theorem. 3.. Discussion We recall that the purpose in all approaches aimed at the current problem is to generate an approximation of the form p t x t Y t N i P t,iδ x t x t,i. As argued in 9, the optimal quantization of p t x t Y t would guarantee a minimal estimation error bound. Clearly, when the grid is chosen off-line the best one can hope for is to have the weights as close as possible to V i p t x t Y t dx t as for a fixed grid this will give the optimal weights see [7]. The result may however be quite far from a good approximation. On the other hand, since the on-line approach uses the on-line data, it will result in a choice of grid which is closer to the optimal for p t x t Y t. Hence it is likely to result in a better approximation. On the other hand, since it involves more computation at each time step it is bound to be more computationally demanding. Indeed, all grid type optimizations become more and more difficult in high dimensions since they rely, amongst other things, on nearest neighbour searches - see, for example [8]. The only caveat we would place on this is when one uses fast sampling since then the optimal quantizer for one sample acts as a good initial condition for the next sample - see [9] for further discussion. The above intuitive discussion of relative performance can be made somewhat more rigorous when viewing the bounds derived in [4] for the two off-line algorithms. Both have the form Π y,t f Π y,t f K t max φ t Y t, φ t Y t t A t k E { k } k0 and differ in the expressions for the coefficients A t k and in the quantization errors k. However, when applying the bound, since φ t Y t can not be calculated, the denominator will be in fact φ t Y t. This value, by its definition see 4-7 and 20-23, depends on products of the terms p k y k X k x k,j. Clearly, as the grid points x k,j are chosen off-line, the conditional pdf, p k y k x k, evaluated at these points may be very small resulting in a very small value for φ t Y t and a large value for the performance bound. As our experiments indicate this actually results, not only in a poor performance bound, but actually provides poor performance in practice. When compared to the on-line method, as the quantization depends upon the data, the grid points will necessarily results in larger values of p k y k X k x k,j see comment following eqn. 29 and a smaller bound.

0.9 0.8 t Real real offline online 0.7 0.6 0.5 Estimated online quantization 0.4 0.3 0.2 0. Estimated offline marginal quantization 0 4 3 2 0 2 3 4 x Figure. The true conditional distribution and its two approximations at time t 4. Numerical example We present here a very simple numerical example for which we could calculate to a high accuracy via the use of very fine griding the true pdfs and use it to demonstrate the points we made earlier regarding the relative accuracy of off-line and on-line approaches. Consider the process defined by X k 0.9X k + W k, Y k X t + V k. 46 We chose N, N w 25 and calculated the true pdf p t x t Y t, the marginal approximation p t x t Y t and our on-line approximation. Note that, in our method, at each time, we have the pdf p t x t Y t on a N N w size grid and after its on-line quantization we have p t x t Y t on a N size grid. In Figures -3 we present a comparison of the resulting pdfs at different times, the real p t x t Y t and the two estimated ones - one using the marginal off-line quantization and the other the on-line quantization. In fact, in the figures we present the respective distributions as they are easier to compare. We clearly observe how very close the on-line approximation is to the real one while the off-line provides a very poor approximation. Next, to demonstrate the filtering properties, we have chosen the function } Xt Y y - the off- { } f Xt Y y - the f x e x and calculated the errors E {f X t Y y } E line marginal quantization estimates and { f E {f X t Y y } E on-line quantization estimates. The results are presented in Figure 4. We observe that the errors for the off-line are consistently larger by an order of magnitude.

2 0.9 0.8 Real t30 Real Online Offline 0.7 Estimated online quantization 0.6 0.5 0.4 0.3 Estimated offline marginal quantization 0.2 0. 0 4 3 2 0 2 3 4 x Figure 2. The true conditional distribution and its two approximations at time t 30 0.9 0.8 t80 Real Real Offline Online 0.7 0.6 0.5 0.4 0.3 0.2 Estimated online quantization Estimated offline marginal quantization 0. 0 4 3 2 0 2 3 4 5 6 x Figure 3. The true conditional distribution and its two approximations at time t 80 5. Conclusion The idea of using quantization as a deterministic alternative to particle filtering for nonlinear systems has been discussed here. As we have pointed out, previous literature on this topic use off-line quantization. This approach suffers from a potential problem since the quantization grids, a crucial part of the approximation, are determined off-line with no consideration of the measured data. The approach we present here, which we believe to be novel, is based on carrying out the quantization on-line based on discrete versions of the posterior pdfs. This makes the on-line

REFERENCES 3 0.25 0.2 Offline marginal quantization Filtering error 0.5 0. 0.05 online quantization 0 0 20 40 60 80 00 20 time Figure 4. Filtering errors for the function f x e x using the online and offline approximations. quantization, while more on-line computation intensive, still feasible for modest size problems. We wish to point out that our main goal here is the comparison with off-line quantization methods. An extensive comparison of these off-line methods has been presented in [5] with clear advantages to the quantization approach at relatively small grid sizes. Hence, claiming potential advantage over the off-line quantization methods clearly implies advantage over particle filtering approaches in particular, with small grid sizes. For the new approach presented in the paper, we have derived a performance bound by appropriately adapting the methods previously described in [4]. We have also pointed to the potential advantages of our proposal when compared with the earlier off-line approach. Finally, we use a simple example to demonstrate the validity of our claims regarding the advantages of the on-line approach when compared to the off-line one. References [] S. Arulampalam, S. Maskell, N. Gordon and T. Clapp, A Tutorial on Particle Filters for On-line Non-linear/Non-Gaussian Bayesian Tracking, IEEE Trans. On Signal Proces. Vol. 50, No. 2, Feb. 2002, pp. 74-88. [2] Z. Chen, Baysian Filtering: From Kalman Filters to Particle Filters, and Beyond,on the Internet at URL: http://users.isr.ist.utl.pt/ jpg/tfc0607/chen bayesian.pdf [3] N. Gordon, D. Salmond and A.F. M. Smith, Novel Approach to Non-linear and Non-Gaussian Baysian State Estimation, IEE Proceedings-F, Vol. 40, 993, pp. 07-3. [4] G. Pagès and H. Pham, Optimal quantization methods for nonlinear filtering with discrete time observations. Bernoulli, Vol., 2005, pp. 893-932. [5] A. Sellami, Comparative survey on nonlinear filtering methods: the quantization and the particle filtering approaches, Journal of Statistical Computation and Simulation, Vol. 78, No. 2, 2008, pp. 93-3. [6] G. Gan, C. Ma and J. Wu, Data Clustering: Theory, Algorithms and Applications, SIAM, 2007. [7] A. Gersho and R. M. Gray, Vector Quantization and Siganl Compression, Kluwer Academic Publishers, Boston, USA, 99. [8] G. Pagès and J. Printems, Optimal quadratic quantization for numerics: the Gaussian case, Monte Carlo Methods and Applications, 92, pp.35-65, 2003. [9] M.G. Cea, G.C. Goodwin and A. Feuer, A discrete nonlinear filter for fast sampled problems based on vector quantization, Proceedings of the American Control Conference, Baltimore, USA, 200.