Rao-Blackwellized Particle Filter for Multiple Target Tracking

Rao-Blackwellized Particle Filter for Multiple Target Tracking Simo Särkkä, Aki Vehtari, Jouko Lampinen Helsinki University of Technology, Finland Abstract In this article we propose a new Rao-Blackwellized particle filtering based algorithm for tracking an unknown number of targets. The algorithm is based on formulating probabilistic stochastic process models for target states, data associations, and birth and death processes. The tracking of these stochastic processes is implemented using sequential Monte Carlo sampling or particle filtering, and the efficiency of the Monte Carlo sampling is improved by using Rao-Blackwellization. Key words: multiple target tracking, data association, unknown number of targets, Rao-Blackwellization, sequential Monte Carlo, particle filtering 1 Simo Särkkä (the corresponding author) is with Helsinki University of Technology, P.O. Box 9203. FIN-02015 HUT, Finland. E-mail: simo.sarkka@hut.fi, Tel: +358 40 757 0730, Fax: +358 9 451 4830. 2 Aki Vehtari is with Helsinki University of Technology, P.O. Box 9203. FIN-02015 HUT, Finland. E-mail: aki.vehtari@hut.fi, Tel: +358 9 451 4849, Fax: +358 9 451 4830. 3 Jouko Lampinen is with Helsinki University of Technology, P.O. Box 9203. FIN- 02015 HUT, Finland. E-mail: jouko.lampinen@hut.fi, Tel: +358 9 451 4827, Fax: +358 9 451 4830. Preprint submitted to Elsevier Science 22 September 2005

1 Introduction This article is an extended version 4 of the article [1], in which we proposed a Rao-Blackwellized particle filtering based multiple target tracking algorithm called Rao-Blackwellized Monte Carlo data association (RBMCDA). In this article we extend the RBMCDA algorithm to tracking an unknown and time varying number of targets. In the proposed algorithm, not only the target states and the data associations, but also the births and deaths of the targets are modeled as hidden stochastic processes, which are observed through the measurements. The states of the stochastic processes are inferred from the measurements using Rao-Blackwellized particle filtering. In principle, it is always possible to approximate the optimal Bayesian filtering solution [2] to the estimation problem with a particle filter [3 5], which generates samples from the joint posterior distribution of the hidden stochastic processes. The idea behind Rao-Blackwellization is that by conditioning on the data associations and the birth and death processes, the posterior distributions of the states of the targets can be approximated with Gaussian distributions. For this reason, the target states can be integrated out analytically and the particle filter only needs to be applied to the data associations and the birth and death processes. This significantly reduces the computational requirements and increases the efficiency of the particle filter. The problem of tracking an unknown number of targets can be divided into the subproblems of tracking single targets, data association and estimating the number of targets. Of course, these subproblems are coupled such that one cannot be solved without the other, but this conceptual division is often done in literature. In the article [1] we presented a solution to first two of these problems using Rao-Blackwellized particle filtering together with classical filtering theory [2,6]. The main contribution of this article is to solve the problem of estimating the number of targets. In the next two sections we shall present a short review of the existing methods for data association in the case of a known number of targets (Section 1.1) and for data association and number of targets estimation in the case of an unknown number of targets (Section 1.2). Relationships with the method described in this article are also discussed. 4 A short version of this study was presented at the Seventh International Conference on Information Fusion (FUSION 2004) in Stockholm, Sweden [1] 2

1.1 Approaches to Data Association The classical data association methods for multiple target tracking can be divided into two main classes [7]. Unique-neighbor data association methods, such as multiple hypothesis tracking (MHT), associate each measurement with one of the previously established tracks. All-neighbors data association methods, such as joint probabilistic data association (JPDA), use all measurements for updating all the track estimates. The idea of MHT [8,7,9] is to associate each measurement with one of the existing tracks, or to form a new track from the measurement. Because this association is not necessarily unique, several hypotheses are continuously formed and maintained. The MHT algorithm calculates the likelihoods of the measurements and the posterior probabilities of the hypotheses, storing only the most probable hypotheses. To enhance the computational efficiency, heuristic methods such as gating, hypothesis merging, clustering and several other strategies can be employed. Probabilistic multiple hypothesis tracking (PMHT) [10] is a modification of the MHT, where the data associations are assumed to be independent over the target tracks. This way the computational complexity of the method is substantially reduced, but it is also impossible to model certain practical constraints, for example, to restrict the number of contacts per target to one on each scan. Our RBMCDA method [1] in its original form had this same restriction, but it can be easily overcome by allowing dependencies in the data association priors, as shown in Section 2.3. JPDA [8,7] approximates the posterior distributions of the targets as separate Gaussian distributions for each target. If the number of targets is T, then T separate Gaussian distributions are maintained. The number of Gaussian distributions is kept constant by integrating over the distribution of data associations of the previous step. This results in an algorithm where each of the target estimates gets updated by every measurement with weights that depend on the predicted probabilities of the associations. Gating is used for limiting the number of measurements for each track. If the predicted probabilities are too low (i.e., below a predefined threshold) for certain targets, those targets are not updated at all. Clutter measurements can be modeled similarly. Sequential Monte Carlo (SMC) based multiple target tracking methods [11 13] typically belong to the class of unique-neighbor data association methods, as they are based on representing the data association and state posteriors as discrete sets of hypotheses. These kind of SMC methods can be considered generalizations of MHT. Instead of maintaining the N most probable data association hypotheses, the joint tracking and data association problem 3

is modeled as a Bayesian estimation problem and the posterior distribution is estimated with SMC methods. This particle filtering approach has the advantage that there are no restrictions on the analytic form of model, although the required number of particles for a given accuracy can be high. In article [1] we proposed how SMC based tracking and data association algorithms can be made more accurate and efficient using Rao-Blackwellization. In the Rao-Blackwellized Monte Carlo data association (RBMCDA) algorithm the states are integrated out in closed form and SMC is only used for the data association indicators. Instead of a pure particle representation, this leads to a mixture of Gaussians representation of the joint posterior distribution, which reduces variance and requires less particles for the same accuracy. 1.2 Approaches to Tracking Unknown Number of Targets The JPDA method was originally formulated for a known number of targets, but it is possible to include track formation and termination logic in cascade with the algorithm [8]. There is no explicit probability model for target appearance and disappearance, but instead there can be a Markov chain model for the number of data associations before track initiation is confirmed. Tracks are terminated when the probability of target existence goes below a predefined threshold. MHT based methods [8,7,9] as well as PMHT [10] form hypotheses of associations with new targets for every measurement. In practice, to reduce the computational complexity, new target hypotheses (or new track hypotheses) are formed only when the measurement falls into an area where the likelihood of the association with the existing targets is too low. A track is deleted when its likelihood becomes too low compared to the other tracks. Random sets and finite set statistics (FISST) [14] provide a very general framework for Bayesian modeling of multiple target tracking in the case of an unknown number of targets. A tractable implementation of the framework is to use the first order moment of the multi-target posterior, the probability hypothesis density (PHD) [15] as an approximation. SMC based implementations of the PHD have been reported, for example, in the articles [16] [17]. In the SMC based method presented in the article [13] the extension to an unknown number of targets is based on hypothesis testing. Because the algorithm generates estimates of data association probabilities, these estimates can be used for approximating the probability of the hypothesis that the target has disappeared from the surveillance area. The detection of the appearance of a new target is based on testing the hypothesis between association with the old targets and with the new target. 4

The article [18] presents a SMC based method, which is similar to our method except that a plain particle presentation of the joint posterior distribution is used. In the method, birth and death moves in particle proposals are used, and the moves resemble the birth and death models used in this article. Due to the plain particle presentation, the method in the article [18] is also applicable to the more general case of target tracking without explicit thresholding of measurements. The method in [19] also resembles our method, except that the article does not suggest any particular form for the birth and death models. The approximation based on limiting the number of births and deaths on each time step is also discussed in [19]. The particle filtering based method in [20] uses exponential (Poisson) models for target appearance and disappearance a bit similarly to our method. The branching particle based solution [21] also models target appearance as a stochastic (Markov) process. The tracking of an unknown number of targets is also closely related to model selection. An application of SMC methods to estimating the number of RBF network weights from data is presented in [22]. In this article, we extend the SMC based RBMCDA method [1] to tracking an unknown number of targets. The extension is based on modeling the birth and death stochastic processes, such that track formation and termination (or initiation and deletion) are not based on heuristic rules, but on the rules determined by the estimation algorithm designed for the probability model. However, this probability model of extension to an unknown number of targets is closer to the approach presented in [9] than to the approach in [15] despite the more strict Bayesian nature of the latter. 1.3 Particle Filtering Sequential Importance Resampling (SIR) (see, e.g. [3 5,23]), is a generalization of the particle filtering framework for the estimation of state space models of the form x k p(x k x k 1 ) y k p(y k x k ), (1) where x k R n is the state at time step t k and y k R d is the measurement. The algorithm uses a weighted Monte Carlo representation of the posterior state distribution. This set of particles is updated and reweighted using a recursive version of importance sampling. An additional resampling procedure 5

is used for removing particles with very small weights and duplicating particles with large weights. The variance introduced by the resampling procedure can be reduced by proper choice of the resampling method [4]. In our version of SIR resampling is not performed on every step, but only when it is actually needed. One way of implementing this is to do resampling on every kth step, where k is some predefined constant. This method has the advantage that it is unbiased. Another way, which is also used in our simulation system, is adaptive resampling, in which the effective number of particles, which is estimated from the variance of the particle weights [24] is used for monitoring the need for resampling. The performance of the SIR algorithm is also dependent on the importance distribution π( ), which is an approximation of the posterior distribution of states given the values at the previous step. The importance distribution should be of such functional form that we can easily draw samples from it and evaluate the probability densities of the sample points. The optimal importance distribution (see, e.g., [5,23]) is π(x k x k 1, y 1:k ) = p(x k x k 1, y 1:k ). This importance distribution is optimal in the sense that it minimizes the variance of the importance weights. One way of improving the efficiency of SMC is to use Rao-Blackwellization. The idea of the Rao-Blackwellized particle filter (RBPF) [25,5,23] is that sometimes it is possible to evaluate some of the filtering equations analytically and the others with Monte Carlo sampling instead of computing everything with pure sampling. According to the Rao-Blackwell theorem this leads to estimators with less variance than what could be obtained with pure Monte Carlo sampling [26]. An intuitive way of understanding this is that the marginalization replaces the finite Monte Carlo particle set representation with an infinite closed form particle set, which is always more accurate than any finite set. The sampling and resampling approach that we used is not necessarily the most efficient in all conditions, but it turned out to work well in our applications. By tuning the resampling algorithm and possibly changing the order of weight computation and sampling, accuracy and computational efficiency of the algorithm could possibly be improved [27]. An important issue is that sampling could be more efficient without replacement, such that duplicate samples are not stored. There is also evidence that in some situations it is more efficient to use a simple deterministic algorithm for preserving the N most likely particles. In the article [28] it is shown that in digital demodulation, where the sampled space is discrete and the optimization criterion is the minimum error, the deterministic algorithm performs better. 6

2 RBMCDA with a Known Number of Targets This section reviews the Rao-Blackwellized Monte Carlo data association (RBM- CDA) method that we proposed in [1] and relates it to the Rao-Blackwellized particle filtering framework. The method described here is basically the same as in the original article except that the requirement of IID prior data association probabilities is replaced with a Markov chain assumption. 2.1 Filtering Model The filtering model of the RBMCDA algorithm is the following: Clutter or false alarm measurements can be modeled using any probability density, which is independent of the target states x k = (x k,1 x k,t ) T p(y k c k = 0). (2) Here c k is the data association indicator, which has the value c k = 0 for clutter and c k = j for the targets j = 1,..., T. The clutter measurements can be, for example, uniformly distributed in the measurement space of volume V p(y k c k = 0) = 1/V. (3) Target measurements are linear Gaussian p(y k x k,j, c k = j) = N(y k H k,j x k,j, R k,j ). (4) where the measurement matrices H k,j and covariance matrices R k,j can be different for each target. Non-linear measurement models can be used by replacing the non-linear model with a locally linearized model as in the extended Kalman filter (EKF) [2,6] or by using the unscented transformation as in the unscented Kalman filter (UKF) [29]. Target dynamics are linear Gaussian p(x k,j x k 1,j ) = N(x k,j A k 1,j x k 1,j, Q k 1,j ), (5) where the transition matrix A k 1,j and process noise covariance matrix Q k 1,j may be different for different targets. The motions of individual targets are a priori independent. Because for any continuous time linear Gaussian dynamic model there exists an equivalent discrete time linear Gaussian model [2,6], this directly generalizes to continuous time linear models. Nonlinear discrete or continuous time dynamic models can be also used as in EKF [2,6] or UKF [29]. With slight modifications to the algorithm, it would be possible to use the interacting multiple model (IMM) filter [6], which allows the modeling of target maneuvers. 7

Target and clutter association priors are known and can be modeled as an mth order Markov chain p(c k c k 1,..., c k m ). (6) This is kind of higher order models are needed for implementing constraints to the data associations on a single measurement scan (see Section 2.3). Of course, the model can be also a first order Markov chain (m = 1) or the associations can be completely independent (m = 0). The number of targets T is known and constant. Target state priors can be represented as a weighted importance sample set p(x 0,j ) = i w (i) N(x 0,j m (i) 0,j, P (i) 0,j). (7) 2.2 Relationship with the Rao-Blackwellized Particle Filter The RBMCDA model described in the previous section fits exactly to the Rao-Blackwellized particle filtering framework (see, e.g., [5]) when the sampled latent variable λ k is defined to contain the data association event indicators, c k at time step k λ k = c k. (8) (1) The state x k on time step k consists of the stacked vector of target states x k,1 x k =.. (9) x k,t (2) The prior distribution of the joint state p(x 0 ) is Gaussian, because the individual target prior distributions are Gaussian. (3) The joint dynamic model of targets is linear Gaussian p(x k x k 1 ) = N(x k A k 1 x k 1, Q k 1 ), (10) where A k 1 is a block diagonal matrix consisting of the dynamic models of the targets and Q k 1 is the block diagonal process noise covariance. (4) The joint measurement model of the targets can be written as p(y k x k, c k ) = N(y k H k (c k )x k, R k (c k )), (11) where the measurement model matrix H k (c k ) is formed conditional on the data association c k such that the only nonzero entries are the ones 8

corresponding to the measurement model of the target c k. The measurement noise covariance R k (c k ) is the measurement noise covariance of the target c k. (5) The clutter measurements are simply state independent measurements with a measurement model of the form p(y k c k = 0). (6) The data associations are modeled as a Markov chain of latent variables as in the Rao-Blackwellized particle filter model. 2.3 Sequential Measurement Update Issues In Section 2.1 we formulated the model such that exactly one measurement is obtained at one time step t k. This way, the data association procedure can be reduced to processing one measurement at a time. However, this does not mean that we are restricted to one measurement per time instance (i.e., scan), because the successive time steps may occur on the same time instance. scan 1 (3 measurements): scan 2 (2 measurements): scan 3 (3 measurements): scan 4 (1 measurement): t 1 = t 2 = t 3 = 1 s t 4 = t 5 = 2 s t 6 = t 7 = t 8 = 2.5 s t 9 = 3.5 s Table 1 Several measurement steps can occur on the same time instance or scan. In the example presented in Table 1 the time steps 1,2 and 3 all occur on time instance 1 s, time steps 4 and 5 occur on time instance 2 s and so on. Now, for example, time steps 1,2 and 3 can be processed sequentially by setting t = 0 in the dynamic model and in the prediction step, which is equivalent to not performing the prediction step at all between the measurements. The original RBMCDA that was presented in [1] had the restriction that the data associations were required to be conditionally independent as in PMHT [10]. This makes it impossible to model certain joint effects in sets of measurements obtained on one time instance (i.e., scan). For example, it is impossible to restrict the maximum number of data associations with each target to one per time instance. However, by allowing the data association priors to depend on previous data associations (at least on the same time instance) this restriction can be included in the model. As discussed in Section 2.1, the data association indicators may also depend on previous data associations, that is, they may form an mth order Markov chain p(c k c k 1,..., c k m ). (12) 9

This kind of model can be used for restricting the data associations to at most one data association to each target as follows: The joint prior model can be written in the general form p(c k+m 1,..., c k ). (13) We can encode the desired restriction into the prior by assigning zero probability to any joint event, which would have two associations to the same target. The joint model can be also expanded as follows: p(c k+m 1,..., c k ) m = p(c k+j c k,..., c k+j 1 ). j=1 (14) This means that exactly the same model can be realized by using using the following priors for the data associations: c k has the prior p(c k ). c k+1 has the prior p(c k+1 c k ).... c k+m 1 has the prior p(c k+m 1 c k+m 2,..., c k ). Using the above idea it is possible to reduce any joint data association prior of form (13) to an equivalent Markov prior model, which is suitable for sequential processing. Assume, for example, that we are tracking two targets and on each time instance we may obtain zero or one detections from each of the targets. The rest of the detections are false alarms, that is, clutter. The target detection probabilities of both targets are the same and given as p(detection) = p d. (15) Assume that on time instance t scan we obtain m measurements y k,..., y k+m 1 (i.e., the times of the steps are t k,..., t k+m 1 = t scan ). The prior for the data associations can be now defined sequentially as follows: Define detection indicators as follows 1, if there is target 1 detection in c k+j 1...c k δ 1 (j) = 0, otherwise 1, if there is target 2 detection in c k+j 1...c k δ 2 (j) = 0, otherwise. (16) 10

Compute the data association priors given each of the possible detection indicator combinations: p(c k+j δ 1 (j) = 0 and δ 2 (j) = 0) p(c k+j δ 1 (j) = 0 and δ 2 (j) = 1) p(c k+j δ 1 (j) = 1 and δ 2 (j) = 0) p(c k+j δ 1 (j) = 1 and δ 2 (j) = 1). (17) Prior models having the restriction of one data association to each target per time instance can be defined in similar manner for any number of targets. The resulting prior is of the recursive (Markov) form, which is a special case of the model described in Section 2.1. 2.4 Data Representation The algorithm state consists of a set of N particles, where each particle i at time step k contains the following: where {c (i) k m+1:k, m(i) k,1,..., m(i) k,j,..., m(i) k,t, P(i) k,1,..., P(i) k,j,..., P(i) k,t, w(i) k }, (18) c (i) k m+1:k are the data association indicators of time steps k m + 1,..., k with integer values 0,..., T, where T is the number of targets. If the data association prior model is an mth order Markov model, then m previous data associations should be stored. If the data association prior is time independent, the data association indicators do not need to be stored at all. m (i) k,j,p(i) k,j are the mean and covariance of the target j, and they are conditional on the data association history c (i) 1:k. w (i) k is the importance weight of the particle. 2.5 Evaluating and Sampling from the Optimal Importance Distribution For each particle i, the optimal importance distribution is given by p(c k y 1:k, c (i) 1:k 1 ). (19) 11

The marginal measurement likelihood is given by [1] p(y k c k, y 1:k 1, c (i) 1:k 1 ) 1/V if c k = 0 = KF lh (y k, m (i) j,k, P (i) j,k, H j,k, R j,k ) if c k = j (20) where j = 1,..., T and KF lh ( ) denotes the Kalman filter measurement likelihood evaluation. H j,k and R j,k are the measurement model matrix and the measurement covariance matrix of the target j, respectively. For j = 1,..., T we have [m (i) j,k, P (i) j,k ] = KF p(m (i) j,k 1, P(i) j,k 1, A j,k 1, Q j,k 1 ), (21) where KF p ( ) denotes the Kalman filter prediction step, and m (i) j,k 1, P(i) j,k 1 are the mean and the covariance of target j in particle i, which is conditioned on the state history c (i) 1:k 1. A j,k 1 and Q j,k 1 are the transition matrix of dynamic model and the process noise covariance matrix of the target j, respectively. The posterior distribution of c k can be calculated using Bayes rule p(c k y 1:k, c (i) 1:k 1 ) p(y k c k, y 1:k 1, c (i) 1:k 1 ) (22) p(c k c (i) k m:k 1 ), where we have used the fact that an association c k does not depend on the previous measurements y 1:k 1, and depends only on the m previous associations c k m:k 1 if the order of the Markov model is m. We can sample from the optimal importance distribution as follows: (1) Compute the unnormalized clutter association probability ˆπ (i) 0 = p(y k c (i) k = 0, y 1:k 1, c (i) 1:k 1 ) p(c (i) k = 0 c (i) k m:k 1 ). (23) (2) Compute the unnormalized target association probabilities for each target j = 1,..., T ˆπ (i) j = p(y k c (i) k p(c (i) k (3) Normalize the importance distribution: π (i) j = = j, y 1:k 1, c (i) 1:k 1 ) = j c (i) k m:k 1 ). (24) ˆπ (i) j Tj =0 ˆπ (i) j, j = 0,..., T. (25) 12

(4) Sample a new association c (i) k Draw c (i) k Draw c (i) k = 0 with probability π (i) 0 = 1 with probability π (i) with the following probabilities: 1. 2. Draw c (i) k = 2 with probability π (i)... Draw c (i) k = T with probability π (i) T. Now it is easy to evaluate also the probabilities of data association hypotheses in the optimal importance distribution, because the probabilities are given by the terms π (i) j, j = 1,..., T conditional on each particle. 2.6 Algorithm Implementation As shown in Section 2.2 the RBMCDA algorithm is a special case of the generic RBPF algorithm. However, due to the conditional independences between the targets, the full Kalman filter prediction and update steps for all targets can be reduced to single target predictions and updates. Actually, because the targets are a priori independent, conditional on the data associations c k the targets will remain independent during tracking. This leads to the following simplifications to the RBPF computations: (1) The Kalman filter prediction steps can be done for each target in each particle separately. That is, we do not need to do Kalman filter prediction to the joint mean and covariance of all targets, but only to each target separately. (2) We can always use the optimal importance distribution, which was described in the previous section, as the importance distribution for the latent variables. (3) The marginalized measurement likelihoods can be computed for each target separately. Note that these likelihoods have already been computed for each target during the evaluation of the optimal importance distribution. (4) The measurement updates can be also performed for each target separately. This means that the Kalman filter update is actually performed only to one target in each particle. 3 RBMCDA with an Unknown Number of Targets In the next sections we extend the RBMCDA algorithm to an unknown and time varying number of targets. The probabilistic construction of the model for an unknown number of targets, that is, the probability model for varying 13

state space dimension follows roughly the approach described in [9], which extends MHT to tracking an unknown number of targets. The idea is to assume that there is always a (very large) constant number of targets T. But an unknown, varying number of them are visible (or alive), and they are the ones we are tracking. The visibility of targets is represented with a vector of discrete indicator variables. The number T ensures that the joint distribution of the target states is a Gaussian distribution with constant dimensionality and thus a well defined probability distribution. The model is formulated such that we do not explicitly need to know the actual number of targets T as long as it is theoretically large enough. As an extension to the MHT method, we construct a probabilistic stochastic process model for the births and deaths of the targets and show how Rao- Blackwellized particle filtering can be applied to this problem. 3.1 Filtering Model (1) Clutter or false alarm measurements have the same kind of model as in the RBMCDA model in Section 2.1. (2) Target measurements are linear Gaussian or non-linear EKF/UKF based as in the RBMCDA model in Section 2.1. (3) Target dynamics are linear Gaussian or EKF/UKF/IMM based as in the RBMCDA model in Section 2.1. (4) The target (c k = j) and clutter (c k = 0) association priors, in the case that births and deaths do not occur at the current time step, are known and can be modeled as an mth order Markov chain p(c k c k m:k 1, T k m:k 1 ), where T k m:k 1 contains the number of targets at time steps k m,..., k 1. For example, we might have a uniform prior over targets and clutter: p(c k c k m:k 1, T k m:k 1 ) = 1 1 + T k 1. (26) (5) Target births may happen only when a measurement is obtained, and in that case a birth happens with probability p b. For simplicity, the model is defined such that a birth may happen only jointly with an association event, so that if there is no association to a newborn target, there is no birth. This is equivalent to stating that the target state prior remains constant until the first measurement is associated, that is, the dynamic model does not affect the target state before the first measurement has been associated to the target. This indicates that it is sufficient to consider the time of the first associated measurement as the actual birth moment. 14

(6) After associating a measurement with a target, the life time t d (or time to death) of the target has probability density t d p(t d ), (27) which can be, for example, an exponential or gamma distribution. (7) At the time of birth each target has a known Gaussian prior distribution (which can be non-informative) p(x k0,j) = N(x 0,j m 0,j, P k0,j). (28) 3.2 Probabilities of Birth and Death If a birth has occurred, it is assumed to be certain that the current measurement is associated to the newborn target: 1, if c k = T k 1 + 1 p(c k birth) = 0, otherwise. In the case of no birth, the Markov model for data the associations applies: (29) p(c k no birth) = p(c k c k m:k 1 ). (30) The data association and birth events can be divided into the following cases with different probabilities: (1) A target is born and the measurement is associated with the newborn target: b k = birth c k = T k 1 + 1. (31) (2) A target is not born and the measurement is associated with one of the existing targets or with clutter: b k = no birth c k = j, j = 0,..., T k. (32) (3) Other events have zero probability. Thus, given the associations c k m:k 1 on the m previous steps, the joint distribution of the event b k {no birth, birth} and the association c k is given 15

as p(b k, c k c k m:k 1 ) = p b, in case (1) (1 p b ) p(c k c k m:k 1 ), in case (2) 0, in case (3), (33) where p b (see Section 3.1) is the prior probability of birth. The restriction of one data association per target on single time instance (see Section 2.3) in the case of unknown number of targets can be handled in the same manner as in the case of known number of targets. We simply assume that there is positive probability of detecting a newborn target on each step. That is, in addition to existing targets we model the possibility detection of a new target which has the detection probability p b. This probability of detecting a new target is equivalent to the probability of birth, because we have defined the birth to be the event of detecting the target for the first time. However, the difference to the restriction of one association per target is that births may occur as many times as there are measurements on scan, not only once per scan. The filtering model represented in Section 3.1, states that after associating a measurement with a target, the life time t d of the target has the known probability density (27). Thus if the last association with target j was at time τ k,j, and on the previous time step t k 1 we sampled a hypothesis that the target is alive, then the probability that the target is dead at current time step t k is p(death of j t k, t k 1, τ k,j ) = P (t d [t k 1 τ k,j, t k τ k,j ] t d t k 1 τ k,j ). (34) 3.3 Relationship to RBPF The RBMCDA algorithm with an unknown number of targets fits to the RBPF framework, if the latent variable λ k contains the visibility indicator e k and the data association indicator c k at the current time step λ k = {e k, c k }. (35) The visibility indicators and the data associations implicitly define the number of (visible) targets T k at each time step. Given that the targets are a priori unordered, there is a high permutation sym- 16

metry in the posterior distributions of the target states, visibility indicators and data association indicators. We can change the indices of any two targets, including the visibility indicators and data associations, and the probability of the configuration will remain the same. For this reason, we shall select one of the permutations arbitrarily and use it for representing all the permutations. This permutation is based on the times of the first associations with the targets. This does not change the model, because this is not a priori ordering, but is merely a way of selecting a compact representation for a very high number of redundant permutations. (1) The joint state x k contains the states of the T targets x k,1 x k =.... (36) x k,t (2) At the initial time step the targets have Gaussian prior distributions N(x k,j m 0, P 0 ). The model is constructed such that the invisible targets at any time step k (indicated by e k ) do not have a dynamic model. This means that the targets which have not yet become visible (have not been born yet) at any time step k have independent Gaussian prior distributions N(x k,j m 0, P 0 ). If we denote the sets of not visible and visible target indices with J 0 and J 1, respectively, the joint prior distribution of all targets is of the form p(x k y 1:k ) = j J 1 N(x k,j m k,j, P k,j ) j J 0 N(x k,j m 0, P 0 ). (37) That is, the distribution of the visible targets is completely independent of the distributions of the invisible targets and thus it suffices to store only the states of the visible targets instead of all T targets. The joint distribution of all targets is still always theoretically a Gaussian distribution of dimension T. (3) When a target birth occurs, that is, a new target becomes visible (i.e., produces the first measurement) a new item in the indicator vector e k is set and the corresponding target prior distribution is updated (initialized) by the measurement. Because we only need to store one possible permutation from a high number of equivalent target permutations, we can add the new target to the first empty place in the indicator vector e k. (4) When a target dies, that is, becomes invisible again, the target distribution again becomes the prior and the target state is moved to the end of the joint state vector x k and indicator vector e k. The targets in the 17

vectors can be shifted such that the visible targets always remain in the beginning of the vectors. (5) The target dynamics, target measurements and clutter measurements are modeled in the same way as in the RBMCDA model of Section 2.1 and thus they fit into the RBPF framework easily. (6) By constructing a prior model for to births and deaths, we get the distribution p(e k e k 1 ), (38) which defines the dynamics of births and deaths. The data association model is of the form p(c k c k m:k 1, e k ), (39) and thus these two models together give a joint Markov chain model for the indicators: p(e k, c k c k m:k 1, e k m:k 1 ) = p(c k c k m:k 1, e k ) p(e k e k 1 ), (40) which is the form required by the RBPF model. 3.4 Evaluating and Sampling from the Optimal Importance Distribution The possible events between two measurements y k 1 and y k and at the association of measurement y k are: (1) Targets may die (indicated by elements of e k ): (a) none of the targets dies (b) one or more targets die (2) y k is associated with (indicated by c k ): (a) clutter (b) one of the existing targets (c) a newborn target Death events are independent of the measurements. However, the two event families are related such that a new measurement y k can be associated only to the targets that have not died between the measurements y k 1 and y k. The model (40) assigns unique prior probabilities to each of the finite number of different events, but the problem is that the number of possible events grows exponentially with the number of targets. The combinatorial problem in the number of events is solely due to the exponential number of possible combinations of target deaths. There is no combinatorial problem in target births, because we can always use the sequential update scheme as discussed in Section 2.3. However, the purpose of the death model is only to remove the targets with which no measurements have been associated for a long time. 18

Because the death model is built only for serving this purpose without any physical meaning, it is not desirable to spend most of the computing power on it. For the above reason we shall use an approximation where we restrict the possible number of deaths at each time step to one. This means that at each time step (measurement time) either none or exactly one of the targets dies. The probability of two or more death events between measurements is assumed to be zero. Note that the varying time step size is accounted already in the death model (34) and thus the survival rate will change only a bit, because we assume that there cannot be more deaths than there are measurements. We could overcome this restriction by performing redundant measurement update steps, for example, by generating artificial clutter measurements, which have prior probability one of being clutter. Because due to restricting the number of deaths on each time step the number of events grows only linearly with the number of targets, we can construct an approximate prior distribution of births, deaths and associations as follows: (1) Enumerate all possible combinations of joint birth, (zero or one) deaths, and association events and compute probabilities for each of the combinations. (2) Normalize the list of events such that their probabilities sum to one. For each combination of birth, death and association events there is a transition pair (e 1:k 1, c 1:k 1 ) (e k, c k ) with a probability given by the above procedure. That is, we have an approximate representation of the distribution p(e k, c k e 1:k 1, c 1:k 1 ). The likelihood term p(y k e k, c k ) can be computed similarly as in the case of a known number of targets (see Equation (20)). By multiplying each of the birth, death, and association combinations with the measurement likelihood and normalizing, we can form the optimal importance distribution similarly as in Section 2.5. 3.5 Data Representation The algorithm state consists of a set of N particles, where each particle i at time step k contains the following: {c (i) k m+1:k, e(i) k, m(i) k,1,..., m(i) k,j,..., m(i) k,t, P(i) k,1,..., P(i) k,j,..., P(i) where k,t, w(i) k }, (41) 19

c (i) k m+1:k are the data association indicators of the time steps k m+1,..., k. e (i) k is the life-indicator, which is a binary vector of length T indicating which of the targets are alive at current time step. m (i) k,j,p(i) k,j are the mean and covariance of the target j, and they are conditional on the data association history c (i) 1:k. w (i) k is the importance weight of the particle. The following information is also implicitly or explicitly stored for each particle: where T (i) k τ (i) k,j {T (i) k, τ (i) k,j, id(i) k,j, }, (42) is the number of targets. is the time of the last measurement associated with target j. id (i) k,j is a unique integer valued identifier, unique over all targets in all particles, which is assigned at the birth of the target. 3.6 Algorithm Implementation Similarly to the case of RBMCDA (see, Section 2.6), because the targets are a priori independent, conditional on data associations c k and indicators e k, the targets will also remain independent during tracking. This means that exactly the same simplifications to RBPF apply to the case of an unknown number of targets as to a known number of targets. 4 Simulations 4.1 Bearings Only Tracking of a Known Number of Targets First we shall consider a classical bearings only multiple target tracking problem, which frequently arises in the context of passive sensor tracking. The simulation scenario is similar to that was presented in [1], but now the simulation includes clutter measurements and the restriction of one data association per target on single time instance is also modeled. The dynamics of target j with the state vector x j,k = (x j,k y j,k ẋ j,k ẏ j,k ) T can 20

be modeled with a discretized Wiener velocity model [6] x j,k 1 0 t 0 x j,k 1 y j,k 0 1 0 t y = j,k 1 + q k 1, (43) ẋ j,k 0 0 1 0 ẋ j,k 1 ẏ j,k 0 0 0 1 ẏ j,k 1 where q k 1 is the Gaussian process noise with moments E[q k 1 ] = 0 1 3 t3 1 0 2 t2 0 1 E[q k 1 q T 0 k 1] = 3 t3 1 0 2 t2 q, 1 2 t2 0 t 0 1 0 2 t2 0 t where q is the spectral density of the noise. In the simulation, the value q = 0.1 was used for both targets. The noise in an angular measurement from target j by sensor i can be modeled as Gaussian ( yj,k s ˆθ i ) y k = arctan + r x j,k s i k, (44) x where (s i x, s i y) is the position of sensor i and r k N(0, σ 2 ) with σ = 0.02 radians. Because the measurement model is non-linear we replace the Kalman filter in the data association algorithm with EKF. The uncertainty in data associations can be modeled by defining a variable c k, which has the value c k = j if the measurement at time step k is associated with target j. The target detection probability is set to p d = 80% and the number of clutter measurements (uniform on range [ π, π]) is Poisson-distributed with mean 5. The measurement data obtained from simulated sensors is shown in Figure 1. The initial distribution was on purpose selected such that all the four crossings of measurements from the two sensors contain some probability mass, and the distributions of the targets are two-modal as shown in Figure 2. The particles in the figure are a random sample drawn from the posterior distribution estimate, used for visualizing the distribution. The actual posterior distribution estimate is a mixture of Gaussians which is hard to visualize directly. The number of Monte Carlo samples used in the estimation method was 100. Figure 3 shows the final tracking result, and it can be seen that in the beginning of the trajectory the posterior distribution is multimodal. Also the 21

4 3 2 Measurement [rad] 1 0 1 2 3 4 0 0.5 1 1.5 2 Time [s] Fig. 1. Measurement data obtained from angular sensors. True Target 1 True Target 2 Fig. 2. The prior distributions of the targets. Half of the prior probability mass is located in the wrong sensor measurement crossings. posterior distributions of the trajectories are a bit wider in the area between the sensors, because in that area the position uncertainty is higher in the direction of the line connecting the two angular sensors. Again, particles are used for visualizing the distribution, although the true posterior distribution estimate is a mixture of Gaussians. Figure 4 shows the smoothed tracking result, which is an estimate where the distributions of all time steps are conditioned on all the measurements. This kind of an estimate can be easily calculated with (fixed interval) Kalman smoothers [2,6] and particle smoothers [4] also in the Rao-Blackwellized particle filtering case. Conditional on all the measurements the trajectory no longer contains multimodalities. Also the position uncertainty on the line connecting the sensors is lower. 22

Estimated Target 1 Estimated Target 2 Fig. 3. Filter estimates for each time step. In the beginning of the trajectory the posterior distribution is multimodal. The multimodality can be seen from the two extra clouds of particles, which are located quite far away from the actual target trajectories. The higher position uncertainty on the line connecting the two sensors can be also seen. Smoothed Target 1 Smoothed Target 2 Fig. 4. Smoothed estimates do not have the multimodality in the beginning of the trajectory, because later measurements have resolved it. The position uncertainty on the line connecting the two sensors is also lower than in the filter estimates. 4.2 Unknown Number of 1D Signals In this section the algorithm extension to an unknown number of signals is simulated with the appearing and disappearing signals as described in Table 2. The true signals and simulated data are shown in the Figure 5. All the signals are modeled with discretized white noise acceleration models [6] x k = 1 t x k 1 + q k 1, (45) ẋ k 0 1 ẋ k 1 where x k = x(t k ), ẋ k = ẋ(t k ), the sampling period is t = 1/100, and the process noise spectral density is q = 1/10. The signal conditioned measurements are modeled (and simulated) as measurements of the signal plus a white 23

Signal Appears Disappears x (1) (t) t = 0 t = 8 x (2) (t) t = 0 t = 15 x (3) (t) t = 1 t = 4 x (4) (t) t = 2 t = 5 x (5) (t) t = 5.5 t = 10 x (6) (t) t = 6 t = 15 Table 2 Signal appearance and disappearance schedule in the 1D scenario with an unknown number of signals. 5 4 3 2 1 0 1 2 3 4 0 5 10 15 Fig. 5. Simulated data of the 1D scenario with an unknown number of signals. Gaussian noise component y k,j = x (j) (t k ) + r k, (46) where r k N(0, 1/5 2 ), given that the measurement is from signal j. Every measurement has an equal chance of originating from each of the visible signals and 1% change of being a corrupted measurement uniformly distributed on the area [ 5, 5]. The number of signals is unknown and the following model for the births and deaths is used: The prior probability of birth p b = 1/100. A priori time to death t d from the last data association has the gamma distribution t d Gamma(t d α, β), (47) with constant parameters α and β. 24

The number of Monte Carlo samples used in estimation was N = 10. Figure 6 shows the result of filtering with parameters α = 2 and β = 1. The plotted result is the hypothesis contained in the particle with largest weight. As can be seen from Figure 7 there is slight delay after the disappearance of the signals before they disappear from estimation. The longest delay is in the signal that ends very near the other signal. Also the 1 time step gap in the lowest signal is not detected. Figure 8 shows the result of applying a Kalman smoother to the filtering result in the particle with the largest weight. This corresponds to the maximum a posteriori signal estimate. It can be seen that the estimation result follows the actual signal paths quite well except for the slight delays in signal disappearance. Figures 9, 10 and 11 show the results when the disappearance model parameters were set to α = 2 and β = 1/10, which means that the signals disappear almost ten times faster than in the previous figures. It can be seen that in this case the signals are estimated to disappear and reappear also when there are random gaps in the signals due to uneven measurement times. This result is quite natural, because the model states that even quite small gaps in the measurement sequence (or actually in the signal) should be interpreted as disappearances and reappearances of the signal. 5 4 3 2 1 0 1 2 3 4 0 5 10 15 Time Fig. 6. Filtering result of the 1D scenario with an unknown number of signals and parameters α = 2 and β = 1. The circles represent the estimated starting points of the signals. 4.3 Tracking an Unknown Number of Targets in 2D In this section we demonstrate the algorithm in case of an unknown number of targets moving in 2D space. The prior model for the data associations is 25

5 4 True Number of Signals Estimated Number of Signals 3 2 1 0 2 4 6 8 10 12 14 Time Fig. 7. Estimated number of signals in the 1D scenario with an unknown number of signals and parameters α = 2 and β = 1. 6 4 2 0 2 4 0 5 10 15 Time Fig. 8. Smoothing result of the 1D scenario with an unknown number of signals and parameters α = 2 and β = 1. The circles represent the estimated starting points of the signals. defined such that only zero or one associations with each target on single scan is allowed. 26

5 4 3 2 1 0 1 2 3 4 0 5 10 15 Time Fig. 9. Filtering result of the 1D scenario with an unknown number of signals and parameters α = 2 and β = 1/10. The circles represent the estimated starting points of the signals. 5 4 True Number of Signals Estimated Number of Signals 3 2 1 0 2 4 6 8 10 12 14 Time Fig. 10. Estimated number of signals in the 1D scenario with an unknown number of signals and parameters α = 2 and β = 1/10. The dynamic model for the targets is x j,k 1 0 t 0 x j,k 1 y j,k 0 1 0 t y = j,k 1 + q k 1, (48) ẋ j,k 0 0 1 0 ẋ j,k 1 ẏ j,k 0 0 0 1 ẏ j,k 1 27

6 4 2 0 2 4 0 5 10 15 Time Fig. 11. Smoothing result of the 1D scenario with an unknown number of signals and parameters α = 2 and β = 1/10. The circles represent the estimated starting points of the signals. where q k 1 is process noise. The model for the measurements z k is z 1,k = x k + r x,k z 2,k = y k + r y,k (49) where r x,k, r y,k, N(0, σ 2 ). The sampling period t = 1/100, process noise q = 1/10 in x and y directions, and measurement variance σ 2 = 1/20 2. The detection probability of each target p d = 95% and at each time step a random number of clutter measurements on area [ 2, 2] [ 2.2] is drawn from a Poisson distribution with mean 1. The clutter measurement prior is chosen to restrict maximum the number of data associations per target to one. The birth/death model parameters are set to p b = 1/100, α = 2 and β = 1/2. The number of Monte Carlo samples was N = 100. Figures 12, 13 and 14 show the filtering results, estimated number of targets and smoothed results, respectively. Again, a slight delay in the estimated target disappearance can be seen after the actual disappearance, but yet the estimated trajectories follow the true trajectories quite well. Also the estimated number of targets matches the actual number of targets well. The restriction of maximum number of data associations per target is very important for tracking performance, especially when the amount of clutter is significant. As we have seen 100 particles is enough for in this particular scenario, when the restriction is used. A quick test shows that if the restriction is not used, even with 1000 particles the result is much worse. 28