Joint Tracking and Classification of Airbourne Objects using Particle Filters and the Continuous Transferable Belief Model

Transcription

1 Joint Tracking and Classification of Airbourne Objects using Particle Filters and the Continuous Transferable Belief Model Gavin Powell & David Marshall The Geometric Computing & Computer Vision Group, School of Computer Science, Cardiff University, Cardiff, UK {G.R.Powell, Philippe Smets IRIDIA, Universite libre de Bruxelles, 5 Av Roosevelt, CP 94-6, 5 Bruxelles, Belgium Branko Ristic Intelligence, Surveillance, and Reconnaissance DSTO Australia, Branko.Ristic@dsto.def ence.gov.au Simon Maskell QinetiQ Malvern Technology Centre, Malvern, Worcestershire, UK srmaskell@qinetiq.com Abstract - This paper describes the integration of a particle filter and a continuous version of the transferable belief model. The output from the particle filter is used as input to the transferable belief model. The transferable belief model's continuous nature allows for the prior knowledge over the classification space to be incorporated within the system. Classification of objects is demonstrated within the paper and compared to the more classical Bayesian classification routine. This is the first time that such an approach has been taken to jointly classify and track targets. We show that there is a great deal of flexibility built into the continuous transferable belief model and in our comparison with a Bayesian classifier, we show that our novel approach offers a more robust classification output that is less influenced by noise. ctbm are limited to land vehicle positioning [] and model based classification [3]. We will also identify some deficiencies of a recursive application of the ctbm, such as unwanted convergence and empty set dominance, and show how these deficiencies can be overcome. Our approach to jointly tracking and classifying using particle filters [4] differs from more classical approaches [5]. Within the ctbm framework each of the particles of the particle filter are able to contribute to the classification stage. Each particle outputs a classification distribution. These distributions are not fused using a Bayesian approach, which can be thought of as taking the mean classification distribution from the particles, as we regard this as a 'throwing away' of valuable data. Instead, the individual classification of each of the particles are combined using the ctbm to produce a fused classification output. This is updated recursively in time as shown in Figure. Keywords: Tracking, particle filter, transferable belief model, classification. Introduction Object observed by a sensor Classification used to update the particle filter Recent advances with the transferable belief model (TBM) have led to the inclusion of belief functions that operate within the continuous domain []. This is a substantial improvement over previous discrete versions of the TBM and made such approaches usable in many more applications. We present in this paper one such application where we combine it with a particle filter for joint tracking with classification of moving objects. The continuous transferable belief model (ctbm) is a powerful tool that can utilise all of the features of the TBM, such as the transfer of beliefs and combinations of belief functions, but also allows for probability density functions to be output and used to articulate prior information. Previous examples of applications of the Particle Filter receives noisy measurement Classification output from each particle All classifications from..n fused for overall classification All particles' classifications fused together in ctbm for time n Figure. Data flow through system

2 Paper Outline In Section 3 we present the basic elements, or the building blocks, of the TBM. In Section 4 we show that the model has been extended for use in the continuous domain through the ctbm. Section 4 also includes discussion of how probability density functions can be used as prior information within the ctbm. Section 5 describes how the output of a particle filter can be used as input to a ctbm. Section 6 presents the results, highlights deficiencies of previous applications of the ctbm and demonstrates that the suggested strategies do indeed address these deficiencies. Our conclusions are drawn in Section 7. 3 The basic elements of the transferable belief model The TBM began life in the 97's and is an extension of work completed by A.P.Dempster [6] and Glenn Shafer [7], which later became known as Dempster Shafer Theory (DST). DST itself is a generalisation of Bayesian theory and is based on two ideas, firstly that you create beliefs from subjective probabilities and secondly that you can fuse such information using Dempster's rule of combination. The TBM works on two levels ) The credal level where beliefs are entertained and quantified by belief functions ) The pignistic level where beliefs can be used to make decisions and are quantified by probability functions. The credal level derives its name from the way that beliefs can be transferred when new information is received. A brief summary of the TBM is now given. We refer readers to other introductions [8] for full details on the TBM. The set of all possible aircraft classes is Ω = {A, B, F 6}. A basic belief assignment (bba) is given by m : Ω [,] with m(a) =, where m(a) is the basic belief mass (bbm) given to A. Every A Ω such that m(a)> is a focal representation. Every subset has some support that the real state is within that subset we will provide a mass (bbm) for, and it is these subsets that are focal sets. This is one of the key points that the TBM can allocate masses to subsets of Ω, rather than only the mutually exclusive hypotheses, the elements, as in probability theory. This allows for ignorance to be accounted for within the model. The greater the number of elements within the focal subset, the greater the ignorance that is being displayed. A focal set with one element is called a singleton. We quantify our belief that the actual world belongs to the subset A by bel(a). This is given by the sum of all bbm's that support A given by A Ω bel(a) = m(b) A Ω, A () B A The Generalised Bayesian Theorem (GBT) [9] allows for us to calculate conditional beliefs, much in the same way that Bayes theorem works with probabilities. We of course can use a vacuous a priori, m(ω) =, if we have no prior knowledge. The GBT is a generalisation of Bayes theorem where all conditional probabilities are replaced by belief functions. And the a priori belief function can be vacuous if necessary, stating that we are completely ignorant a priori. If we are completely ignorant over some space, Θ, and so have no idea, which θ i Θ is the true case. But we do have some alternative knowledge such that, for each θ i Θ we know what our beliefs are on another space, X. So the belief function over the space X given that θ i is the case is written as bel X (. θ i ). The plausibility of x X being the true case given that θ i has already occurred is written as pl X (x θ i ). and bel Θ (θ x) = (bel X (x θ i ) + m( θ i )) θ i θ (bel X (x θ i ) + m( θ i )) () θ i Θ pl Θ (θ x) = ( pl X (x θ i )) (3) θ i θ If some belief is known a priori it can simply be combined through conjunctive combination [8]. 4 The continuous transferable belief model Thus far we have given some of the major elements of the TBM applied to finite and discrete spaces. Lots of applications involve continuous spaces. The difficulties of considering a fine discretisation can be seen by considering the rotation of an object, with look angle discretised into 36 segments, each of which is one degree wide. This results in 36 subsets in the power set. Computing power and memory considerations dictate that - is an upper limit of the size of this power set for real time operation. It is therefore necessary to consider clever methods to exploit the continuous nature of the domain.

3 4. Moving away from the discrete We consider intervals in the range [,], and with a finite amount of focal sets (sets with a bbm > ). We will use a triangle space shown below in Figure to plot and visualize these intervals. [a,b], could be straddling [a,b] and outside [a,b]. It shows belief (mass) that is vague with regard to the interval [a,b]. A singleton value is just represented by [a,b] where a=b, whose point K would lie on the diagonal of the triangle. from a to b K a b 5.7. Figure. Point K = (a,b) inside the triangle This triangle defines an interval [a, b] [,]. A point K in the triangle represents an interval, [a,b] within [,] as shown in Figure []. Such a point has a mass associated with it. The belief of an interval [a,b] is defined to be the sum of the mass of all other intervals [x,y] where x a AND y b. The plausibility is defined to be the sum of the mass of all intervals where a x b OR a y b. The commonality is the sum of all intervals where x a AND y b. The relevant masses that are used to calculate the bel, commonality and pl for the interval [.,.7] are also given in Table []. Table. a bba with six focal sets and their corresponding mass, interval limits and inclusion in the belief for the interval [.,.7] i m A a i b i bel A X X total.. The commonality denotes the belief that is free to flow anywhere. It contains all intervals that could be within Figure 3. Triangle space representation of Table We can calculate the probability density function over this triangle space using Equation 4. Betf (s) = m A (A) a a m A ( ) (4) A:s A [,] where a a represents the interval length. If we relax the fact that the number of focal sets has to be finite, we now have a mass density over our triangle and instead of summing we integrate to find the bel, pl etc. The pignistic density now becomes Betf (a) = lim ε y = f T [, ] (x, y) dx, dy (5) y x x = a x = y = a + ε Extension of this approach to the whole of the real axis is described in []. 4. The least committed basic belief density For the continuous case a bba will become a basic belief density (bbd) []. There exists multiple belief functions that can project to the same probability density function. When defining a belief that projects to a given probability density function, there are therefore multiple candidate beliefs. We choose the least committed belief. We define the least committed belief to be the belief that maximises the

4 commonality function. This commonality articulates the mass that is free to flow. The associated mass could support the proposition being considered, partially support it or not support it at all. So by maximising the commonality, we are hedging our bets as much as is possible; we are creating the underlying belief function that is least committed. The least committed basic belief density is in fact a line on the triangle space which starts at (µ, µ) (the mean of the pignistic density function) is denoted as ϕ(u), where u is the distance along the line from (µ, µ). For a unimodal symmetrical density this is a straight line. This is shown in Figure 4 []. The line of focal intervals we base ours on is taken from a paper by Smetts []. If we want to classify a target on its speed then we can construct some pdf s of target speed conditional of target class, Betf(x c i ). Currently we use Gaussian distributions for simplicity, but any distribution can be used. Using the least committed bbd theory we can construct the relevant underlying conditional bbd s for each of these pdf s which will give us m(x c i ), pl(x c i ), bel(x c i ) etc. It is in fact the likelihoods that are required l(c i x) = pl(x c i ). The bbd is given by ϕ(y) = y π e y (8) and the plausibility by An interval Normal pignistic density As can be seen in Figure 4, there is a relationship between the probability density function and the focal sets of the least committed bbd. The relationship between Betf(s) and ϕ(u) is Betf (s) = ϕ(u) u u du (6) u = s Probability given to the interval Figure 4. The focal sets of the least committed basic belief density induced by a normal pignistic density, using the same triangle space as Figure. µ, µ is denoted by a donut. where Betf (u) = Betf (u ) and through differentiation of Equation 6 we can obtain dbetf (s) ϕ(s) = (s s ) ds 5 Joint tracking and classification using a particle filter and the ctbm We show a scenario where we are tracking and identifying aircraft based on their air speeds. We use the integrated particle filter and ctbm which uses prior information in the form of PDF's. The integration of the ctbm with a particle filter is novel. The classification technique that (6) pl(y) = y π e y + erfc(y ) (9) where y = (x µ) / σ and erfc(s) = e t dt π s Using the GBT mentioned previously and described in Smets' paper [,,7] we can get m(a x), which can be used, for making decisions on the class when given the speed of the target. m(a x) = pl(x c i ) [ pl(x c i )] () c i A 5. Integration of the continuous transferable belief model and a particle filter We assume that every particle in the particle filter outputs a value of a variable that is being tracked, upon which a classification can be made within the ctbm. These classification outputs will be weighted according to the particle weights. These classification outputs will be combined to create an overall classification. The resulting classification is combined with the classification output from the previous time step to get a classification decision based on all the data. c i A The particle filter at time t = k will have a state of x k = [ posx, posy, velx, vely] T which is approximated by N samples or particles which are given by S j k k k = [x j, w j ] where j = N and the particle weight is w j k. With each of the particles we calculate a conditional belief function, using equation, for that particular particle. We use the weight associated with each particle as a measure of how reliable that particle is. We weight the belief function through 'discounting' [9]; discounting is a process that allows for an uncertainty over a belief to be taken into account in the ctbm framework. The discounting factor is given as ( α) = w j k. To discount our belief function we use:

5 m α (A x) = ( α ) m(a) A Ω, A Ω () [ ] +α A = Ω m α (A x) = ( α ) m(a) The bba given by application of Equation will give rise to values being assigned to the empty set. In an open world example values given to the empty set indicate that the true object is not one that we have covered with the prior pdf's. Since we use the particles' states to provide input to the ctbm, this occurs on a frequent basis with the outlying particles. When using the ctbm in a recursive manner these regular empty set values give rise to an unrecoverable convergence on the empty set. For this reason after discounting we remove the mass assigned to the empty set and redistribute it to Ω. We combine all of the bba's from each of the particles by recursively using a closed world normalised combination. We use Dempster's combination rule given by where m (A) = K = m (B)m (C) B C K () m (B)m (C) (3) B C = {c i x} = m(a x) A [ m( x)] (4) A :c i A The pignistic transform will provide us with class probabilities and is a proper probability function. 6 Results We have chosen to highlight the features of our approach through classifying a moving aircraft. We show the tracking and classification results of three aircraft, an A tankbuster, a B- stealth bomber and an F-6 fighter jet. We also graphically demonstrate the outcome of convergence and empty set domination problems. A track is created and then PDF models chosen so as to highlight some of the features of our approach. The observed track will produce positional coordinates for the object at each time frame, from these we can deduce a velocity. The changes in position are presented to the particle filter for it to track with. The velocity output of the particle filter is used for classification. One example of the tracks is given in Figure 6. We are using a normalised version of the combination rule for the same reasons as was previously mentioned with regard to empty set values. When this combination rule is used recursively the mass associated with the empty set will quickly dominate. To prevent this we simply put an upper threshold on the mass associated to the singleton sets of.99. If at any point they exceed this we remove that additional mass and place it in Ω. This is a simple way to prevent us getting masses of. on the singleton sets. This means that the ctbm can now recover or change its mind about its classification. Once we have combined the bba's from all of the particles we are presented with a fused belief function for that particular time k. To gain a fused belief function for all time steps k we must use Equation again to fuse these together, see Figure 5. At each time step k begin For each particle j=..n Begin bba_time_k = combine bba part J WITH bba_time_k end bba_time_..k = combine bba_time_k WITH bba_time_..k- end Figure 5. Pseudo code for the fusion of bba's P Figure 6. A- motion Prior PDF's Figure 7. PDF's for track shown in Figure 6 6. Scenario B- A- F-6 To obtain our classifications we use the pignistic transform on the fused conditional masses. The observed velocity for the track shown in Figure 6 is shown in Figure 8 along with the estimated velocity by the particle filter.

6 Observed and estimated object velocities Prior PDF's Estimated Observed P.5..5 B- A- F Figure 8. Observed and estimated velocities of track shown in Figure Figure. Prior PDF's for target model's velocities Classification of object 3 Observed and estimated target velocities B- A- F-6 5 Estimated Observed Figure 9. Classification of aircraft shown in Figure 6 using prior PDF's shown in Figure 7 It should be noted that the particle filter was initiated with samples from a broad prior; hence the poor initial estimates of the object state. This appears to be a very simple classification task, that the ctbm is shown to succeed with in Figure 9, but it should be noted that within the first frames there is confusion to the aircraft type. This is solely due to the particle filter settling down from its poor initialisation. 6. Scenario Figure. Observed and estimated velocities for aircraft shown in Figure Figure 3 shows the classification output from the ctbm when we do not prevent the singleton sets from converging to TBM Classification B- A- F-6 An object motion of an A- is shown in Figure Figure 3. Classification of object shown in Figure using prior PDF's shown in Figure Figure. B- motion We used the prior PDF's as shown in Figure and have the observed and estimated velocities shown in Figure By frame we can see that aircraft type A- has been classified and that particular singleton set converged to. Note that the system is unable to change its mind in any way. It can be argued that if the system changes its mind then either: our models are incorrect; an open world should be assumed. However, the use of the particle filter can result in errors which necessitate the need for a recovery mechanism. A closed world where mass is prevented from converging in singleton sets is a more robust approach. The simplest method of performing this is to set an upper limit close to for the bbm's of the underlying belief function, we will call this convergence protection. Later in the time series, the velocity suggests that the target is in fact an F-6, but due to the previous convergence the ctbm is unable to recover in any way.

7 6.3 Scenario 3 TBM Classification A more ambiguous example is now presented and the TBM results compared to a more classical Bayesian classification, which classifies on the estimated target velocity from the particle filter. The aircraft's motion is shown in Figure B- A- F Figure 7. Classification of object shown in Figure 4 using prior PDF's shown in Figure and with convergence protection Figure 6 and Figure 7 show how the ctbm is affected when one of the singleton's mass converges to. The output appears to latch and to be unable to recover from erroneous initial conclusion. What if the initial classification is incorrect, then the true classification will never be reached how ever obvious it may become. Figure 4. F-6 object motion.4.35 Prior PDF's.3 Estimated and observed target velocities.5 B- 8 P..5 A- F-6 6. velocity Estimated Observed Figure 8. Prior PDF's for target model's velocities Figure 5. Estimated and observed velocities for target shown in Figure 4 TBM Classification We now try to classify the same object motion given in Figure 4 but using altered PDF models as shown in Figure 8 and using both the ctbm approach described in this paper and a simple Bayesian classifier using the same PDF's and the mean estimate of velocity, derived from the particles within the particle filter.. TBM Classification B- A- F B- A- F Figure 6. Classification of object shown in Figure 4 using prior PDF's shown in Figure and with no convergence protection Figure 9. Classification of object shown in Figure 4 using prior PDF's shown in Figure 8 and with convergence protection

8 P Bayesian classification Figure. Classification of object shown in Figure 4 using prior PDF's shown in Figure 8 and using Bayesian classification Comparison between Figure 9 and Figure highlights how a frame wise classification technique can be very susceptible to noisy inputs and are over sensitive to outlying measurements. Both techniques give a similar classification output until the time is approximately 6. From this point we can see a little more how the two techniques differ. The ctbm is concerned with how close the measurement is to the mean of the PDF for each class, as well as its probability, which is different from the Bayesian approach, which just looks at the probability for that speed. 6.4 Discussion If we used an open world example with this then we would have a massive mass given to the empty set due to the presence of conflicting information. The target is one of our modelled targets, but the ctbm is not sure which it is. Intuitively, it should pick F-6 as this is the only one whose model completely covers the velocity measurements. The problem is that such intuitive performance is not currently output by the ctbm. Within the ctbm there will also be masses assigned to non singleton sets, which can be analysed to give us Type A or Type B classifications. Even though we are normalising by the empty set its values prior to normalisation can be used to show conflict within measurements, which also denotes the possibility of the target not being a member of the target set. 7 Conclusions We have shown how a particle filter and the continuous transferable belief model have novelly being integrated within our system. We have noted the weak points of the ctbm in such a heavily recursive application and proposed methods to address these. When compared to a simple Bayesian classifier we show to have a more robust method for classifying. By using the ctbm we have truly flexible approach to classification. Its ability to transfer beliefs when the world we are working within changes is a very useful feature for object classification, and one that B- A- F-6 will be looked at in the future. We found three avenues for future research. Firstly, the lack of intelligent memory in the ctbm. No knowledge of a target's behaviour is stored. It is possible for a target to behave unlike a particular class yet still be classified as a member of this class Secondly, the empty set has a very important role to play in the ctbm in articulating conflict and the presence of a model that is not a member of the model set. Thirdly, the recursive updating of beliefs in an open world remains a challenge. Acknowledgements Thanks go to the Ministry of Defence, Data Information Fusion defence Technology Centre for funding project 4. Statistical Fusion of Battlefield Data for which this paper is a part. Also a special thanks to the late Philippe Smets for his assistance, and his family for their hospitality. References [] P. Smets, Belief functions on real numbers, International Journal on Approximate Reasoning, Vol 4:3, pp8-3, 5 [] F.Caron, P.Smets, E.Duflos and P.Vanheeghe, Multisensor data fusion in the frame of the TBM on reals application to land vehicle positioning, Information Fusion, Philadelphia, USA, 5 [3] P.Smets and B.Ristic, Belief function theory on the continuous space with an application to model based classification, Information Processing and Management of Uncertainty in Knowledge-Based Systems, Italy, pp 9-6, July 4 [4] S.Arulampalam, S.Maskell, N.Gordon and T.Clapp, A tutorial on particle filters for on-line non-linear/nongaussian bayesian tracking, IEEE Transactions on Signal Processing, Vol 5, pp , February [5] N.Gordon, S.Maskell and T.Kirubarajan, Efficient particle filters for joint tracking and classification, In Proceedings of SPIE: Signal and Data Processing of Small Targets, pp , [6] A.P.Dempster, A generalisation of Bayesian inference, Journal of the Royal Statistical Society, Series B3. pp [7] G.Shafer, A mathematical theory of evidence, Princeton University Press, Princeton, NJ 976. [8] P.Smets and R.Kennes, The transferable belief model, Artificial Intelligence, 66, pp9-34, 994 [9] P.Smets, Belief functions: the disjunctive rule of combination and the generalised Bayesian theorem, International Journal of Approximate Reasoning, 9, pp- 35, 993.