Statistics 102 for Multisource-Multitarget Detection and Tracking

Transcription

1 100 Statistics 10 for Multisource-Multitarget Detection and Tracking Ronald Mahler Abstract This tutorial paper summarizes the motivations, concepts and techniques of finite-set statistics (FISST), a systemlevel, top-down, direct generalization of ordinary single-sensor, single-target engineering statistics to the realm of multisensor, multitarget detection and tracking. Finite-set statistics provides powerful new conceptual and computational methods for dealing with multisensor-multitarget detection and tracking problems. The paper describes how multitarget integro-differential calculus is used to extend conventional single-sensor, single-target formal Bayesian motion and measurement modeling to general tracking problems. Given such models, the paper describes the Bayes-optimal approach to multisensor-multitarget detection and tracking: the multisensor-multitarget recursive Bayes filter. Finally, it describes how multitarget calculus is used to derive principled statistical approximations of this optimal filter, such as PHD filters, CPHD filters, and multi-bernoulli filters. Index Terms multitarget tracking, multitarget detection, data fusion, finite-set statistics, FISST, random sets. I. INTRODUCTION This paper is a sequel to, and update of, a tutorial published in 004 for the IEEE Aerospace and Electronic Systems Magazine [1]. That tutorial described some central ideas of finite-set statistics (FISST). Finite-set statistics is a systematic, unified approach to multisensor-multitarget detection, tracking, and information fusion. It has been the subject of considerable worldwide research interest during the last decade, including more than 600 research publications by researchers in more than a dozen nations. I attribute this interest to the fact that finite-set statistics: is based on explicit, comprehensive, unified statistical models of multisensor-multitarget systems; unifies two disparate goals of multitarget tracking target detection and state-estimation into a single, seamless, Bayes-optimal procedure; results in new multitarget tracking algorithms PHD filters, CPHD filters, multi-bernoulli filters, etc. that do not require measurement-to-track association, while still achieving tracking performance (localization accuracy, speed) comparable to or better than conventional multitarget tracking algorithms; results in promising generalized CPHD and multi- Bernoulli filters that can operate in unknown, dynamically changing clutter and detection backgrounds; and Copyright (c) 013 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org. R. Mahler is with Lockheed Martin Advanced Techology Laboratories, Eagan, MN. ronald.p.mahler@lmco.com. more generally, has been a fertile source of fundamentally new approaches in multisource-multitarget tracking and information fusion. The emphasis of the earlier tutorial was on the answers to three questions: How does one Bayes-optimally 1 detect and track multiple noncooperative targets using multiple, imperfect sensors? How does one correctly model multisensor-multitarget systems so that Bayes-optimality is possible? How does one accomplish this using a Statistics like formalism that is specifically designed for solving multitarget tracking and data fusion problems? The answer to the first question the multisourcemultitarget Bayes recursive filter is computationally intractable in all but the simplest problems. The answers to the second and third questions multitarget formal Bayes modeling and multitarget integro-differential calculus, respectively were addressed only at a very high level. Thus this paper begins where the previous one left off, with emphasis on answers to the following, consequent questions: How does one actually construct faithful Bayesian models of multisensor-multitarget systems? How does one approximate the optimal multisourcemultitarget Bayes recursive filter in a principled statistical manner meaning that the underlying models and their relationships are preserved as faithfully as possible? What mathematical machinery what specific multitarget Statistics 101 methodology makes this possible? The earlier tutorial paper was written at a very elementary level (it was presumed that even Bayes rule might be an unfamiliar concept). This paper continues along the same path, but does presume some basic knowledge. This includes undergraduate probability and calculus, motion and measurement models, probability density functions, likelihood functions, Markov transition densities, and so on. It should also be emphasized that the paper is a tutorial introduction to, not a survey of, finite-set statistics. It includes pointers to the some of the most significant developments, but these are by no means exhaustive. The paper is organized as follows. Section II describes the engineering philosophy that motivates finite set statistics. Section III presents a review of Statistics 101 for single-sensor, single-target systems, focusing on the single-sensor, singletarget recursive Bayes filter. Section IV summarizes its generalization to multisensor-multitarget Statistics 101, focusing 1 By Bayes-optimal, I mean that target state(s) are determined by a state estimator, applied to the posterior distribution of a Bayes filter, that minimizes the Bayes risk corresponding to some cost function (see [0], p. 63).

2 101 on motion and measurement modeling and the multisensormultitarget recursive Bayes filter. Section V provides an overview of the primary approximations of this filter, the PHD, CPHD, and multi-bernoulli filters. Section VI summarizes the principled statistical approximation methodology that leads to these filters. Conclusions can be found in Section VII. II. THE PHILOSOPHY OF FINITE-SET STATISTICS Multisensor, multitarget systems introduce a major complication that is absent from single-sensor, single-target problems: they are comprised of randomly varying numbers of randomly varying objects of various kinds. These include varying numbers of targets; varying numbers of sensors with varying number of sensor measurements collected by each sensor; and varying numbers of sensor-carrying platforms. A rigorous mathematical foundation for stochastic multiobject problems point process theory [4], [37] has been in existence for a half-century. However, this theory has traditionally been formulated with the requirements of mathematicians rather than tracking and data fusion engineers in mind. The formulation usually preferred by mathematicians random counting-measures is inherently abstract and complex (especially in regard to probabilistic foundations) and not easily assimilable with engineering physical intuition. A. Motivations and Objectives The fundamental motivation for finite-set statistics is: tracking and information fusion R&D engineers should not have to be virtuoso experts in point process theory in order to produce meaningful engineering innovations. As was emphasized in [1], engineering statistics is a tool and not an end in itself. It must have two qualities: Trustworthiness: Constructed upon a systematic, reliable mathematical foundation, to which we can appeal when the going gets rough. Fire and forget: This foundation can be safely neglected in most situations, leaving a serviceable mathematical machinery in its place. These two qualities are inherently in tension. If foundations are so mathematically complex that they cannot be taken for granted in most engineering situations, then they are shackles and not foundations. But if they are so simple that they repeatedly result in engineering blunders, then they are simplistic rather than simple. This inherent gap between trustworthiness and engineering pragmatism is what finite-set statistics attempts to bridge. Four objectives are paramount: Directly generalize familiar single-sensor, single-target Bayesian Statistics 101 concepts to the multisourcemultitarget realm. Avoid all avoidable abstractions. As much as possible, replace theorem-proving with mechanical, turn-the-crank, purely algebraic procedures. Nevertheless retain all mathematical power necessary for effective engineering problem-solving. B. Overview It is worthwhile to begin by first comparing the FISST random finite set (RFS) paradigm with the ubiquitous conventional paradigm: report-to-track association (MTA). 1) The Standard Measurement Model : The most familiar tracking algorithms presume the following measurement model, one that has its origins in radar tracking. A radar amplitude-signature for a given range-bin, azimuth, and elevation is subjected to a thresholding procedure such as CFAR (constant false alarm rate). If the amplitude exceeds the threshold, there are two possible reasons for the existence of this blip. First, it was caused by an actual target in which case a target detection has occurred at z =( ). Second, it was caused by a momentary surge of background noise in which case a false detection or false alarm has occurred at z. A third possibility that a target was present but was not detected is referred to as a missed detection. For target-generated detections, the small target model is presumed. Targets are distant enough (relative to the radar s resolution capability) that a single target generates a single detection. But they are also near enough that only a single target is responsible for any detection. ) Measurement-to-Track Association (MTA): Because of the small-target assumption, a bottom-up, divide and conquer strategy can be applied to the multitarget detection and tracking problem ([0]. pp ). Suppose that, at time, we are in posession of tracks ( 1 x 1 1 )( x ) i.e., possible targets where, for the track, x is its state (position, velocity), its error covariance matrix, and its track label. The Gaussian distribution (x) = (x x ) is the track density of the track. Next, suppose that at time +1 we collect detections = {z 1 z }. Typically, because of false alarms. The prediction step of an extended Kalman filter (EKF) is used to construct predicted tracks ( 1 x ) ( x ). We can then construct the following hypothesis : for each, the target ( x ) generated the detection z () ; or, alternatively, generated no detection. The excess measurements {z 1 z } {z (1) z () } are interpreted as false alarms or as having been generated by previously undetected targets. The hypothesis is a MTA. Taking all possibilities into account, we end up with a list of MTAs. For each +1 +1, we can apply the update step of an EKF to use z () to construct a revised track ( () x () () ). Multi-hypothesis trackers (MHTs) are currently the dominant tracking algorithms based on MTA. 3) Association-Free Multitarget Detection and Tracking : In contrast to MTA, FISST employs a top-down paradigm grounded in point process theory specifically, in the theory of random finite sets (RFS s). In place of the hypothesis-list , one has a probability distribution ( () ) on the finite-set variable = {x 1 x } with 0, where () : 1 is the time-history of measurement-sets at time. Instead of the standard

3 measurement model just described, one constructs from it a multitarget likelihood function () = +1 ( ). The value () is the likelihood that a measurement-set will be generated, if targets with state-set are present. Given this, a multitarget version of the recursive Bayes filter (Section IV-C) is applied instead of the MTA procedure. Since this Bayes filter will in general be computationally intractable, it must be approximated, resulting in the PHD, CPHD, multi- Bernoulli and other filters (Section V). The following point should be emphasized: RFS algorithms are capable of true tracking. It is often, to the contrary, asserted that RFS algorithms are inherently incapable of constucting time-sequences of labeled tracks, because finite sets are order-independent. This is not the case. As was explained in [0], pp target states have, in general, the form x =( x), where is a identifying label unique to each track. Given this, the multitarget Bayes filter as well as any RFS approximation of it, including PHD and CPHD filters can maintain temporally-connected tracks. In particular, Vo and Vo have used this approach to devise an exact, closed-form, computationally tractable solution to the multitarget recursive Bayes filter [4], [43]. Because the solution is exact, this filter s track-management scheme is provably Bayes-optimal. 4) Nonstandard Measurement Models: However ubiquitous the standard model may be, it is actually an approximation the result of applying a detection process to a sensor signature. RFS models and filters are being developed for nonstandard sensor sources that supply raw signature information. Perhaps the two most interesting instances are: RFS models and multi-bernoulli track-before-detect (TBD) filters for pixelized image data [11], [1]. These filters have been shown to outperform the previously-best TBD filter, the histogram-pmht filter. RFS models and CPHD filters for superpositional sensors [4], [35], [47]. These filters have been shown to significantly outperform a conventional MCMC (Markov chain Monte Carlo) approach, while also being much faster. III. SINGLE-SENSOR, -TARGET SYSTEMS The purpose of this section is to summarize the basic elements of the conventional Statistics 101 toolbox. A. Single-Sensor, Single-Target Recursive Bayes Filter The primary tool is the recursive Bayes filter the foundation for optimal single-sensor, single-target tracking. At various times 1, a single sensor with unity probability of detection and no clutter, interrogates a single noncooperative target. The time-sequence of collected measurements is : z 1 z and the state of the target the information about it that we wish to know (position, velocity, type, etc.) is x. The Bayes filter propagates a Bayes posterior distribution through time: (x ) predictor +1 (x ) corrector (x +1 ) estimator ˆx Here, (x ) is the probability (density) that the target has state x, given the accumulated information. The predictor (time-update) step accounts for the increase in uncertainty in the target state between measurement collections. The corrector (measurement-update) step permits fusion of the newest measurement z +1 with previous measurements. These steps are definedbythetime-predictionintegral +1 (x )= +1 (x x 0 ) +1 (x 0 )x 0 (1) and Bayes rule (x +1 )= +1(z +1 x) (x ) +1 (z +1 () ) where +1 (z +1 )= +1 (z +1 x) (x )x (3) is the Bayes normalization factor. The estimator step consists of a Bayes-optimal state estimator, such as the maximum a posteriori (MAP) estimator: x MAP =argsup (x +1 ) (4) x ( Bayes-optimal means that the estimator minimizes the Bayes risk corresponding to some cost function [0], p. 63.) The Bayes filter formulas require knowledge of two a priori density functions: the target Markov transition density +1 (x x 0 ) and the sensor likelihood function +1 (z x). The former, +1 (x x 0 ), is the probability (density) that the target will have state x at time +1 if it had state x 0 at time. The latter, +1 (z x), is the probability (density) that the sensor will collect measurement z at time +1 if a target with state x is present. By true formulas for +1 (x x 0 ) and +1 (z x) is meant: +1 (x x 0 ) and +1 (z x) faithfully incorporate the motion and measurement models; and no extraneous information has inadvertently been introduced into them. B. Moment Approximations of the Bayes Filter Historically, the Bayes filter has typically been implemented using moment approximations. Let (x) = 0 (x x 0 ) denote a Gaussian distribution with mean x 0 (first-order moment of (x)) and covariance matrix 0 (a second-order moment of (x)). Assume that signal-to-noise ratio (SNR) is large enough that the track distributions can be approximately characterized by their first- and second-order moments: (x ) = (x x ) (5) +1 (x ) = +1 (x x +1 ) (6)

4 Then the Bayes filter can be replaced by a filter the extended Kalman filter (EKF) that propagates the first-and secondorder moments: x x +1 x Similarly, assume that SNR is large enough that the track distributions can be approximately characterized by their firstorder moments: (x ) = (x x ) (7) +1 (x ) = (x x +1 ) (8) for some fixed covariance. Then the Bayes filter can be replaced by a filter for example, an - filter that propagates only the first-order moment: x x +1 x C. Single-Target Motion Modeling Target modeling is schematically summarized in Figure 1. At the top, interim target motion is mathematized as a statistical motion model. The function x = (x 0 ) states that the target will have state x at time +1 if it had state x 0 at time. Since this equation is typically just a guess, it is randomly perturbed by the motion noise ( plant noise ) W with probability distribution W (x). The information contained in this model is equivalent to that contained in the next line, the probability mass function (p.m.f.) +1 ( x 0 )=Pr(X +1 x 0 ) (9) The p.m.f. is equivalent to the probability density function (p.d.f.) +1 (x x 0 ): +1 (x x 0 )= W (x (x 0 )) (10) This formula is a standard result easily found in standard textbooks. It is a consequence of the following equation: +1 ( x 0 )= +1 (x x 0 )x (11) The validity of Eq. (11) ensures that Eq. (10) is true because it means that +1 ( x 0 ) and +1 (x x 0 ) are entirely equivalent statistical descriptors of X +1. motion model probabilitymass function true Markov density Bayes filter predictor X +1 {z } = (x 0 ) + W {z } {z} predicted target deterministic plant noise +1 ( x 0 ) =Pr(X +1 x 0 ) R x +1 x single-object calculus x (x 0 )= +1 (x x 0 ) single-target prediction integral (x ) +1 (x ) Figure 1: Single-Sensor, Single-Target Motion Modeling D. Single-Sensor, Single-Target Measurement Modeling Sensor modeling is schematically summarized in Figure. We begin with a statistical measurement model. The function z = +1 (x) states that the sensor will collect measurement z at time +1 if a target with state x is present. Because of sensor noise, this formula must be randomly perturbed by a noise-vector V with distribution V+1 (z). The information in this model is equivalent to that in the p.m.f. +1 ( x) =Pr( +1 x) (1) This p.m.f. and thus the original measurement model is equivalent to the p.d.f. +1 (z x): +1 (z x) = V+1 (z +1 (x)) (13) The fact that this formula is true is assured by the equation +1 ( x) = +1 (z x)z (14) measurement model probabilitymass function true likelihood function Bayes filter corrector {z +1 } = +1 (x) + V {z } {z +1 } measurement deterministic sensor noise +1 z +1 ( x) =Pr( +1 x) R z single-object calculus z (x) = +1 (z x) single-target Bayes rule +1 (x ) (x +1 ) Figure : Single-Sensor, Single-Target Measurement Modeling E. Single-Target, Multisensor Data Fusion Suppose that we have two sensors, with as in the singlesensor case unity probability of detection and no clutter. Measurement-collection times are identical (synchronous), so 1 that the sensors collect measurement-streams and with z 1 and z collected simultaneously at time for any =1. Let the respective likelihood functions be 1 1z (x) abbr. = 1 ( 1 z x), z (x) abbr. = ( z x) (15) If the sensors are independent, then their joint likelihood function has the form 1 1z z (x) = 1 1z (x) z (x) (16) Measurements are optimally fused using Bayes rule: = ) (17) (x 1 1 1z (x) z 1 (x 1 1 ( 1 z z ) 1 )

5 where = ( z 1 z z (x) z 1 (x ) (18) 1 )x This can be accomplished as follows. Let x be an arbitrarily small region around x of size x.then +1 ( x x 0 ) = +1 (x x 0 )x x (4) = +1 (x x 0 ) x (5) F. Data Fusion Via Averaging? The Bayes-optimal fusion approach, Eq. (16), employs a product likelihood. Consider the counterproposal in [38] that one should, instead, use an average: 1 1z z (x) = 1 ( 1 1z (x) + z (x)) (19) One reviewer of this paper objected that this method hardly merits mention, because data fusion averaging is not a sensible approach. Readers patience is nevertheless requested, because it is being explicitly or implicitly promoted by rather powerful individuals. Eq. (19) is problematic because whereas product-likelihoods inherently improve target localization, average-likelihoods inherently worsen it. Consider the following simple example: two bearing-only sensors in the plane with respective Gaussian likelihood functions 1 ( ) = ( ), ( ) = ( ) (0) That is, the sensors are oriented so as to triangulate the position of a target located at ( ). For conceptual clarity, let the prior distribution be +1 ( ) = 0 ( 0 ) 0 ( 0 ) (1) where 0 is arbitrarily large so that +1 ( ) is effectively uniform. Let 1 be the measurements collected by the sensors. Then Bayes rule yields Bayes ( ) = ( 1 ) ( ) () This results in a triangulated localization at ( 1 ) with variance =. But with the average likelihood, av ( ) = 1 ( 1 ) ( 0) ( 0 ) ( ) (3) This distribution has four tails whose lengths increase with the size of its variance, which is = 0. Now apply additional bearing-only sensors, all with orientations different from the first two and each other. The variance increases with the number of averaged sensors whereas it greatly decreases if Bayes rule is used instead. As we shall see in Section V-C, a generalization of Eq. (19) is what leads to the very poor performance of the averagebased multisensor PHD filter proposed in [38]. G. Constructing Markov Densities and Likelihoods Eqs. (10,13) do not tell us how to construct explicit formulas for +1 (x x 0 ) and +1 (z x) from explicit formulas for +1 ( x 0 ) and +1 ( x). and therefore +1 (x x 0 )= lim x &0 +1 ( x x 0 ) (6) x This, the Lebesgue differentiation theorem, provides one (but not the only) way of deriving +1 (x x 0 ) from +1 ( x 0 ) using a constructive Radon-Nikodým derivative ([9], pp ). For the model in Figure 1 we have +1 ( x x 0 ) = Pr( (x 0 )+W x ) (7) = W (x (x 0 )) x (8) from which Eq. (10) follows. IV. MULTISENSOR-MULTITARGET SYSTEMS The purpose of this section is to summarize the basic elements of multisensor-multitarget Statistics 101. A. Random Finite Sets (RFS s) This section introduces the concept of an RFS as the multitarget analog of a random vector. 1) Random Single-Target States : The state x of a singletarget system may (as an example) have the form x = ( ) where are position variables, are velocity variables, and is a discrete identity variable (which could be, for instance, a track label). In a Bayesian approach, the state at time must be a random state X. The precise mathematical definition of a random state X requires that it actually be a measurable mapping from a probability space to the state space. In turn, the state space must be equipped with a topology, typically (but not always) a Euclidean topology. While such details are mathematically necessary, for engineering purposes they can usually be taken for granted. ) Random Multitarget States : The state of a multitarget system, on the other hand, is most accurately represented as a finite set of the form = {x 1 x }. Here, not only the individual target states x 1 x are random but also their number (cardinality). This includes the possibility =0 (no targets are present), in which case we write = (the null set). The finite-set representation is most natural because given that each target already has its own unique identity, as indicated by a variable such as from a physical point of view the targets have no inherent order. Thus in a Bayesian formulation, a state-set is actually a random state-set Ξ it is a random finite set (RFS). Similar comments apply to the measurements collected from the targets by a sensor. These also usually have no inherent physical ordering. They thus have the form = {z 1 z }, where not only the individual measurements z 1 z are random, but also their number 0. Thus

6 in a Bayesian development a measurement-set is actually a random measurement-set Σ +1 an RFS. The RFS representation is more engineering friendly than the random-measure representation of standard point process theory. A finite set {x 1 x } is easily visualizable as a point pattern for example, in the plane or in three dimensions. Similarly, an RFS is easily visualizable as a random point pattern. An everyday example of an RFS: the stars in a night sky, with many stars winking in and out and/or slightly varying in their apparent position ([0], pp ). 3) Fire-and-Forget Foundations of RFS s : If we are to have a trustworthy mathematical foundation for multisensormultitarget systems, we must precisely define RFS s. This forces us to define topologies on so-called hyperspaces that is, spaces X whose points are subsets (in our case finite subsets) of some other space X. Two hyperspaces are of engineering interest: the hyperspace of finite subsets of the state space, and the hyperspace of finite subsets of the measurement space. If we employed the random-measure formulation of point process theory, we would be forced to work with abstract probability measures defined on measurable subsets of an abstract space whose points are counting measures. In an arbitrary RFS formulation, this would be replaced by equally abstract probability measures () =Pr(Ξ ) defined on measurable subsets of X, with any point of being a finite subset of X. Luckily, a simpler stochastic geometry formulation [37] is available. Its hyperspace topology the Fell-Matheron topology allows () to be equivalently replaced by the multitarget analog of a conventional probability-mass function () =Pr(X ). This is the belief-mass function (b.m.f.) () =Pr(Ξ ) (9) which is defined on (closed) subsets of X. 4) Fire-and-Forget Foundations of Multitarget Tracking : The upshot of Eq. (9) is that, in finite-set statistics, it is usually possible to entirely avoid abstractions such as topologies, measurable mappings, and the randomness of finite sets in the formal sense. More generally, finite-set statistics is intentionally formulated as a stripped-down version of point process theory one in which we attempt to avoid all avoidable abstractions. As an illustration, concepts such as thinning and marking are basic to purely mathematical treatments of the subject. But in multitarget detection and tracking, these concepts appear only in a few, concrete contexts that can be adequately addressed at a purely engineering level. Missed detections and disappearing targets can both be described as forms of thinning; and target identity as a form of marking. But from an engineering perspective, does the imposition of such concepts represent an increase of content or of pedantry? The reason is as follows. Consider the set function () =1 ( ) = Pr(Ξ 6= ). Then the Choquet-Matheron capacity theorem states that the additive measure () is equivalent to the nonadditive measure (), in the sense that both completely characterize the probabilistic behavior of Ξ (see [9], p. 6 or [0], p. 713). As a second illustration, in FISST density functions are systematically used in place of measures, except when this is not possible. Thus the Dirac delta function is employed even though it produces engineering-heuristic abbreviations of rigorous expressions (as in Eq. (81), for example). B. Probability Distributions of RFS s Just as a random state-vector X has a probability density (x) = X (x), so an RFS has a multitarget probability density function (m.p.d.f.) () = Ξ () (30) Its form varies with the number of targets: ( ) if = ({x 1 }) if = {x 1 } () = ({x 1 x }) if =, = {x 1 x }.. (31) where denotes the number of elements in. Also, its units of measurement vary with target number: if are the units of x, then the units of () are. In general, a function () that satisfies the same property with respect to units is a multitarget density function. Its set integral in the region is definedtobe () (3) = ( ) X 1 + ({x 1 x })x 1 x! 1 {z } times where, as a convention, define ({x 1 x })=0 whenever {x 1 x } 6=. The probability that there are elements in Ξ is () = () (33) = = 1 ({x 1 x })x 1 x (34)! Thus () for 0 is a probability distribution on the number of targets the cardinality distribution of Ξ. C. Multisensor-Multitarget Recursive Bayes Filter This filter is the theoretical foundation for optimal singlesensor, single-target tracking. At times 1, one or more of sensors interrogate an unknown number of unknown, noncooperative target. The time-sequence of collected measurement-sets is () : 1. The multisensormultitarget Bayes filter propagates a multitarget Bayes posterior distribution through time: ( () ) predictor +1 ( () ) corrector ( (+1) ) multitarget estimator ˆ +1 +1

7 Thesestepsaredefined by multitarget analogs of the timeprediction integral +1 ( () )= +1 ( 0 ) +1 ( 0 () ) 0 and of Bayes rule ( (+1) )= +1( +1 ) ( () ) +1 ( +1 () ) (35) (36) where +1 ( +1 () )= +1 ( +1 ) ( () ) (37) The estimator step consists of a multitarget Bayes-optimal state estimator. As was explained in [1], multitarget versions of the maximum a posteriori (MAP) and expected a posteriori (EAP) estimators do not exist. Rather, one must use alternative estimators ([0], pp ). As in the single-sensor, single-target case, the multisensormultitarget Bayes filter requires two aprioridensity functions: the multitarget Markov transition density +1 ( 0 ) and the multisensor-multitarget likelihood function +1 ( ). Here, +1 ( 0 ) is the probability (density) that the targets will have state-set at time +1 if they had stateset 0 at time. Also, +1 ( ) is the probability (density) that the sensors will jointly collect measurement-set at time +1 if targets with state-set are present. In the single-sensor, single-target case, the formulas for the Markov density and likelihood function Eqs. (10,13) are never derived but, rather, simply looked up. In multisensormultitarget problems, this is not possible because no standard references exist. Thus one must ask: How does one construct statistical multitarget motion models for any given application? In particular, how does one model phenomena such as target disappearance and target appearance? How does one construct statistical multisensormultitarget measurement models for any particular set of sensors? In particular, how does one model phenomena such as sensor fields of view and clutter? Given such models, how does one construct formulas for the true multitarget Markov density and the true multisensor-multitarget likelihood function? That is, how does one know that they are not heuristic contrivances, or that no extraneous information has inadvertently been introduced? D. Multitarget Motion Modeling This is summarized in Figure 3. At the top, interim target motions are represented as a RFSmotionmodel. As an example, consider the most commonly assumed multitarget motion model the standard such model. At time suppose that the target state-set is 0 = {x 0 1 x 0 0} with 0 = 0. At time +1, either each of the targets persists or disappears. Let (x 0 ) be the probability that x 0 will persist into time +1 and transition to some other (random) state X. Then x 0 will transition to ½ (x 0 if disappears (prob. 1 (x )= 0 )) {X } if persists (38) Suppose that is the set of targets that newly appear at time +1. Then the RFS motion model has the form Ξ +1 = (x 0 1) (x 0 0) (39) The set-theoretic union symbol indicates that at time +1, targets will be either persisting targets or new targets. It is assumed that (x 0 1) (x 0 0) are independent. The information contained in this model is equivalent to that contained in the next line of Figure 3, the b.m.f. +1 ( 0 ). Because of independence, it is +1 ( 0 )= (x 0 1 ) () (x 0 0 )() () (40) where the b.m.f. of (x 0 ) is (x 0 ) () (41) = Pr( (x 0 ) ) = Pr( (x 0 )= )+Pr( (x 0 ) 6= (x 0 ) )(4) = 1 (x 0 )+ (x 0 ) +1 (x x 0 )x (43) Also, the b.m.f. of is normally chosen to be Poisson: µ () =exp (x)x (44) Here +1 is the expected number of appearing targets; and +1 (x), the spatial distribution, is the probabilty (density) that an appearing target will have state x. A central aspect of finite-set statistics is a set of procedures for deriving the formula for the multitarget Markov density +1 ( 0 ) from the formula for +1 ( 0 ). These two statistical descriptors are related by the equation +1 ( 0 )= +1 ( 0 ) (45) for all. The validity of this equation ensures that the formula for +1 ( 0 ) is true. This is because Eq. (45) states that +1 ( 0 ) and +1 ( 0 ) are equivalent. The explicit formula for +1 ( 0 ) is too complicated to state here see [0], pp

8 multitarget motion model belief-mass function true multitarget Markov density multitarget Bayes filter predictor Ξ +1 = ( 0 ) {z } {z } {z} predicted targets surviving appearing +1 ( 0 ) =Pr(Ξ +1 0 ) R +1 multiobject calculus ( 0 )= +1 ( 0 ) multitarget prediction integral ( () ) +1 ( () ) Figure 3: Multisensor-Multitarget Motion Modeling E. Multisensor-Multitarget Measurement Modeling This is summarized in Figure 4. We begin with a RFS multisensor-multitarget measurement model. Consider the standard such model. Suppose that the state-set for the targets at time +1 is = {x 1 x } with =. It is assumed that each target generates at most a single measurement, and that any measurement is generated by at most a single target. Let (x ) be the probability that the target x generates a measurement. Then the set of measurements Υ +1 (x ) generated by the target with state x will have at most a single element: ½ if undetected (prob. 1 (x Υ (x )= )) { } if detected (46) Suppose that +1 is the set of measurements that are generated by no target i.e., the clutter measurements. Then the RFS measurement model has the form Σ +1 = Υ (x 1 ) Υ (x ) +1 (47) The symbol indicates that measurements consist of target-generated measurements or clutter measurements. It is assumed that Υ (x 1 ) Υ (x ) +1 are statistically independent. Because of independence, the b.m.f. of the RFS model is +1 ( ) = Υ+1 (x 1 )( ) Υ+1 (x )( ) +1 ( ) (48) where the b.m.f. of Υ +1 (x 0 ) is Υ+1 (x )( )=1 (x )+ (x ) +1 (z x )z (49) and the b.m.f. of +1 is usually chosen to be Poisson: +1 ( )=exp µ (z)z (50) Here +1 (z) is the spatial distribution of clutter measurements, and +1 is the expected number of clutter measurements at time +1 (the clutter rate ). The multisensor-multitarget likelihood function +1 ( ) is related to the belief-mass function by the equation +1 ( ) = +1 ( ) (51) The validity of this equation ensures that the formula for +1 ( ) is true, because it shows that +1 ( ) and +1 ( ) are equivalent. The explicit formula for +1 ( ) is too complicated to reproduce here see [0], pp multitarget meas t model belief-mass function true multitarget likelihood function multitarget Bayes filter corrector Σ +1 {z } = +1 (x) +1 {z } {z } measurements targets clutter +1 ( x) =Pr(Σ +1 x) +1 multiobject calculus R () = +1 ( ) multitarget Bayes rule +1 ( () ) ( (+1) ) Figure 8: Multisensor-Multitarget Measurement Modeling F. Multitarget Calculus for Modeling Eqs. (45,51) do not tell us how to construct formulas for +1 ( 0 ) and +1 ( ) from formulas for +1 ( 0 ) and +1 ( ). This is the purpose of multitarget calculus, which generalizes the reasoning used in Section III-G. Let x and x 0 be arbitrarily small regions surrounding def. x and, for any closed subset, let 0 x = x,where 0 indicates set-theoretic difference. Then 0 x and x are disjoint and so +1 ( 0 x x x 0 ) +1 ( 0 x x 0 )= +1 ( x x 0 ) (5) andsofromeq.(6), (x x 0 ) (53) = lim 0 x &0 lim x &0 +1 ( 0 x x x 0 ) +1 ( 0 x x 0 ) x 1) Set Derivatives : For any real-valued set function () define the generalized Radon-Nikodým derivative ([9], pp ): x () = lim 0 x &0 lim x &0 ( 0 x x ) ( 0 x ) (54) x Extend this definition as follows. For any = {x 1 x } with = 1, define () = () (55) x 1 x

9 and, if =, define () =() (56) Then Eqs. (54-56) define the set derivative of () with respect to ([9], p ). The set derivative is the inverse operation of the set integral: () = ( ) (57) µ ( ) = () (58) ) Construction of True Markov Densities and Likelihood Functions : It is possible to derive turn-the-crank rules of differentiation for set derivatives: sum rules, power rules, product rules, chain rules, etc. ([0], pp ). These rules permit the explicit construction of formulas for multitarget Markov densities and multitarget likelihood functions. This is because of the following two formulas, which are direct consequences of Eqs. (57,58): +1 ( 0 ) = (59) +1 ( ) = +1 ( 0 ) +1( ) = (60) = V. PRINCIPLED APPROXIMATE MULTITARGET FILTERS The multisensor-multitarget recursive Bayes filter of Section IV-C is usually computationally intractable. How can we approximate it in a manner that preserves, as faithfully as possible, the underlying models and their interrelationships? This question is answered by assuming that the m.p.d.f. s ( () ) and/or +1 ( () ) have a particular simplified form one that permits approximate closed-form solution of the multitarget Bayes filter. Three types of approximation have been extensively investigated in the literature thus far: Poisson, independent identically distributed cluster (i.i.d.c.), and multi-bernoulli. They result in, respectively, PHD filters, CPHD filters, and multi-bernoulli filters. The purpose of this section is to summarize these filters, as well as their extensions to unknown clutter and detection profiles. A. Probability Hypothesis Density (PHD) Filters Recall from Section III-B that - filters are based on first-moment approximation of the single-sensor, single-target Bayes filter as in Eqs. (7,8). By analogy, we assume that SNR is large enough that the multitarget Bayes filter can be approximated by its first-order statistical moments. Butfirst one must ask: What is the first-order moment of a multitarget probability distribution? 1) Probability Hypothesis Density Functions : The naïve definition of the first-order moment (expected value) of a multitarget distribution would be = ( () ) (61) However, it is mathematically undefined for the simple reason that addition and subtraction ± of finite sets is undefined. Thus instead we must employ an alternative strategy: replace by some doppelgänger for which addition and subtraction is definable. In point process theory, one (intuitively speaking) chooses to be (x) = X y y (x) (6) where y (x) is the Dirac delta function concentrated at y. In this case Eq. (61) is replaced by ([0], pp ) (x () ) = (x) ( () ) (63) = ({x} () ) (64) This function is called a probability hypothesis density (PHD) or first-moment density function. 3 It is completely characterized by the following property: (x () )x = expected no. of targets in (65) Thus the number (x () ) can be understood as the track density at x. A target with state x is more likely to be present in the scene when (x () ) is large than when it is small. Consequently, it is possible to use (x () ) to estimate the number and states of the targets. Let = (x () )x (66) be the total expected number of targets in the scene. Round off to the nearest integer, and then determine those values x 1 x of x that correspond to the highest peaks of (x () ). Then ˆ = {x 1 x } is an estimate of the number of targets and their states. ) Example of a PHD : Suppose that, in a one-dimensional scenario, the multitarget distribution (m.p.d.f.) corresponds to two targets located at = 1 and = : ({ }) = 1) ) (67) + ) 1) Then the corresponding PHD can be shown to be: () = ( 1)+ ( ) (68) 3) PHD Filters in the General Sense : The m.p.d.f. of a Poisson process has the form () = Y (x) (69) x Rwhere (x) is a density function with integral = (x)x. In analogy with constant-gain Kalman filters Eqs. (7,8) assume that the multitarget densities in the multitarget Bayes filter are all approximately Poisson. Then this 3 PHDs are also known as intensity functions. I avoid this terminology because of the potential for confusion with the many alternative meanings of intensity in tracking and information fusion applications. PHD is a historical usage [18].

10 filter can be approximately replaced by a first-order moment filter of the form (x () ) predictor +1 (x () ) corrector (x (+1) ) Such a filter is called a PHD filter in the general sense. 4) The Classical PHD Filter : The classical PHD filter is a PHD filter with these specific modeling assumptions [18], [0]: (1) a single sensor; () all target motions are independent; (3) measurements are conditionally independent of the target states; (4) the clutter process is Poisson in the sense of Eq. (50) and is independent of other measurements; (5) target-generated measurements are Bernoulli in the sense of Eq. (46); (6) the surviving-target process is Bernoulli in the sense of Eq. (38), and independent of persisting targets; and (7) the appearingtarget process is Poisson in the sense of Eq. (44). Using the methodology in Section VI, it can be shown that the exact 4 time-update equation for the classical PHD filter is ([18], [0], Chapter 16): 5 +1 (x () ) (70) = +1 (x) + (x 0 ) x (x 0 ) +1 (x () )x 0 and, if +1 is the currently-collected measurement-set, that the approximate measurement-update equation is (x (+1) ) = +1 (x) +1 (x () ) (71) Here the PHD pseudolikelihood is +1 (x) =1 (x)+ X (x) z (x) (z)+ +1 (z) z +1 (7) and +1 (z) is the clutter spatial distribution and +1 is the clutter rate. Also, +1 (z) = (x) z (x) +1 (x () )x (73) The classical PHD filter has attractive computational characteristics: its order of complexity is () where is the current number of measurements and is the current number of tracks. The major limitation of the PHD filter is that is not a stable instantaneous estimate of target number its variance is typically large. Thus in practice, must be averaged over a time-window in order to get stable targetnumber estimates. B. Cardinalized PHD (CPHD) Filters The CPHD filter is a generalization of the PHD filter. It has low-variance instantaneous estimates of target number and better tracking performance, but at the cost of greater computational complexity. Its computational order is ( 3 ), although this can be reduced to ( ) using numerical balancing techniques [10]. 4 There seems to be a misconception in some quarters that Eq. (70) is approximate i.e., requires the assumption that the prior multitarget distribution ( () ) be Poisson. This is not true. 5 For clarity, the target-spawning model is neglected. 1) CPHD Filters in the General Sense: The cardinality distribution ( () ) of a multitarget track density ( () ) was defined in Eq. (33). For each, ( () ) is the probability that there are targets in the scene. The multitarget density of an independently identically distributed cluster (i.i.d.c.) process has the form () =! ( ) Y (x) (74) x where () is a cardinality distribution and (x) is a probability density (a spatial distribution ). Assume that the multitarget distribuions in the multitarget Bayes filter are approximately i.i.d.c. Then this filter can be approximately replaced by a higher-order moment filter of the form ½ ½ (x () ) predictor +1 (x ( () () ) ) +1 ( () ) corrector ½ (x (+1) ) ( (+1) ) Any such filter is a CPHD filter in the general sense. ) The Classical CPHD Filter : The classical CPHD filter is a CPHD filter with these specific modeling assumptions [19], [0]: (1) single sensor; () independent target motions; (3) conditionally independent measurements; (4) the clutter process is i.i.d.c. and independent of other measurements; (5) target-generated measurements are Bernoulli in the sense of Eq. (46); (6) surviving targets are Bernoulli in the sense of Eq. (38), and independent of persisting targets; and (7) the appearing-target process is i.i.d.c. Given these assumptions and the methodology in Section VI, one can derive the time- and measurement-update equations for the classical CPHD filter. These equations are beyond the scope of this paper (see [0], Chapter 16). The classical PHD and CPHD filters are most commonly implemented using either Gaussian mixture techniques (assuming moderate motion and/or measurement nonlinearities) or particle methods (for stronger nonlinearities). See [0], Chapter 16, for more details. The classical PHD and CPHD filters have also been employed in hundreds of research papers addressing dozens of applications, far too many to address here. A few diverse examples: ground-target tracking using GMTI (ground moving target indicator) radar [41]; passive-rf air-target racking [40]; satellite-borne optical satellite tracking [6]; audio speaker tracking [13]; underwater monostatic-active sonar [1], []; and underwater multistatic active sonar [8]. C. Multisensor Classical PHD/CPHD Filters The classical PHD and CPHD filters can be extended to the multisensor case. However, these extensions are nontrivial and require special theoretical analysis. The exact approach [3], [17] is combinatorial, and therefore is appropriate only for a small number of sensors. Moratuwage, Vo, and Danwei Wang have successfully employed the exact multisensor PHD filter in a robotics SLAM (simultaneous localization and mapping) application [30].

11 The most common approach, the iterated-corrector method, is heuristic. The PHD filter or CPHD filter corrector equation is applied successively for each sensor. This approach depends on sensor order, but performs well when the sensors probabilities of detection are not too dissimilar. Otherwise, larger- sensors should be applied before smaller- sensors [34]. The parallel combination approximate multisensor (PCAM) approach [14] does not depend on sensor order, is computationally tractable, and results in good tracking performance. Another approach has been proposed [38], in which the PHD pseudolikelihoods of Eq. (7) are averaged. Giventhe discussion in Section III-F, this is an obviously problematic approach that should result in poor target localization which indeed it does. Nagappa and Clark have conducted simulations comparing the approaches just summarized. In a first set of three-sensor simulations, two sensors had = 095 and the third = 09. In decreasing order of performance: PCAM-CPHD, PCAM-PHD, iterated-corrector CPHD, iterated-corrector PHD, averaged-pseudolikelihood PHD. The performance of the averaged-pseudolikelihood PHD filter was particularly bad, with the iterated-corrector PHD filter having intermediate performance. Similar results were observed when the probability of detection of the third sensor was decreased to =085 and again to =07. D. Multi-Bernoulli Filters Suppose that we have target tracks, and that each track has a track distribution (x) and a probability of existence, for =1. Then the multitarget density of a multi- Bernoulli process has the form, for = {x 1 x } with =, () = X Y (x ) 1 (75) where 1 1 6=6= =1 = Y (1 ) (76) =1 Assume that the multitarget distributions in the multitarget Bayes filter are approximately multi-bernoulli. Then it can be approximately replaced by a multi-bernoulli filter in the general sense: { (x)} { (x)} { (x)} The first such filter was proposed in [0], Chapter 17 but, because of an ill-conceived linearization step, exhibited a pronounced bias in the target-number estimate. Vo, Vo, and Cantoni corrected this bias with their cardinality-balanced multitarget multi-bernoulli (CBMeMBer) filter [44]. The CBMeMBer filter appears to be well-suited for applications in which motion and/or measurement nonlinearities are strong and therefore sequential Monte Carlo (a.k.a. particle) implementation techniques must be used [44]. Dunne and Kirubarajan have devised a jump-markov version of this filter [5], and Shanhung Wong, Vo, Vo, and Hoseinnezhad have applied it to road-constrained ground-target tracking [36]. Vo, Vo, Pham, and Suter subsequently devised a multi- Bernoulli track-before-detect (TBD) filter for tracking in pixelized image data, assuming that targets have physical extent and thus cannot overlap [47]. As already noted, this filter outperforms the previously best-known TBD filter, the histogram- PMHT filter. It has been successfully applied to challenging real videos, e.g., hockey and soccer matches [1], [11]. E. Background-Agnostic PHD and CPHD Filters The PHD, CPHD, and multi-bernoulli filters all require a priori models of both the clutter (in the form of a clutter spatial distribution +1 (z) and clutter rate +1 )andthe detection profile (in the form of a state-dependent probability of detection (x)). A series of second-generation PHD and CPHD filters do not require apriorimodels. Any PHD, CPHD, or multi-bernoulli filter can be transformed into a filter that can operate when the probability of detection is unknown and/or dynamic [5]. The basic idea is simple: replace the target state x by an augmented target state x = ( x), where 0 1 is the unknown probability of detection associated with a target with state x. The resulting probability of detection agnostic (PDAG) PHD, CPHD, and multi-bernoulli filters have the same form as before, except that the PHDs and spatial distributions have the form ( x) and ( x). The PDAG-CPHD filter has been successfully implemented in [8]. Any PHD, CPHD, or multi-bernoulli filter can be transformed into one that can operate when the clutter background is unknown and/or dynamic [6]. One simply replaces the target state space by a state space that includes both targets and clutter generators. The clutter generators are assumed to be target-like in that their measurement-generation process is Bernoulli in the sense of Eq. (46). In this case, any state x of the joint target-clutter system can have two forms: x = x or x = c where c is the state of a clutter generator. One defines suitable extensions z ( x) and +1 ( x x 0 ) of the likelihood function and Markov density. Targets must not transition to target generators, and vice-versa: +1 (x c 0 )=0, +1 (c x 0 )=0 (77) Otherwise, target statistics and clutter statistics would be inherently intermixed, thus making it more difficult to distinguish targets from clutter. The filtering equations for clutter agnostic (CAG) versions of the PHD, CPHD, and multi- Bernoulli filters can then be derived using simple algebra. A version of the CAG-CPHD filter was successfully implemented in [8]. The PDAG and CAG approaches can be combined, thus allowing any PHD, CPHD, or multi-bernoulli filter to operate under general background-agnostic (GBAG) conditions. Various versions of the GBAG-CPHD filter have been successfully implemented in [6], and likewise for a GBAG-CMeMBer filter in [45]. A related development is the multitarget intensity filter (MIF) or ifilter [39]. It is actually a CAG-PHD filter, except that Eqs. (77) are violated: clutter generators are (problematically) allowed to become targets, and vice-versa.