Fundamentals of Multiple-Hypothesis Tracking

Transcription

1 Fundamentals of Multiple-Hypothesis Tracing Stefano Coraluppi NATO Lecture Series STO IST-55 Advanced Algorithms for Effectively Fusing Hard and Soft Information ITA NDL SWE GBR, September-October 6

2 Outline Multi-target tracing preliminaries Multiple-hypothesis tracing Hypothesis aggregation

3 The Multi-Target Tracing Challenge Difficulties Unnown and changing number of targets Uncertain target evolution Poor detection statistics (missed detections, fading effects, false alarms, redundant measurements, merged measurements) Poor measurement statistics (random errors, bias errors) Unnown data association Multi-sensor data (disparate data, active and passive sensors, multi-int data) The goal Generate a set of tracs that are close to targets in inematic, feature, and identity space Oh yes, another difficulty Choice of appropriate metric or set of metrics 3

4 Examples in Active and Passive Sonar 4

5 Target Existence Continuous-time birth-death process Birth rate λ b, death rate λ χ We consider a discrete-time sequence t, t,, t, with no targets at t Discrete-time Poisson birth process with mean μ b t over time interval t = t + t μ b t = λ b e λ χ t + τ dτ t t + Discrete-time death probability p χ t = λ χ e λ χτ dτ t t + = λ b λ χ e λ χ t = e λ χ t Notes Birth-discretization approximation: note that μ b t < λ b t Stationarity: t μ b t = λ b λ χ Target independence: the number of births in temporally non-overlapping intervals are independent random variables Poisson birth rate can also be understood as the limit of the Binomial distribution (many potential target cells, small birth probability in each) The independence assumption is generally used for simplicity, though it does not always hold (e.g. group-target existence, uninformative large-variance priors) 5

6 Target Evolution Nearly-constant position model w x Nearly-constant velocity model w x x Ornstein-Uhlenbec process w + x γ Integrated Ornstein-Uhlenbec process Singer model (nearly-constant acceleration) w x w + x γ Notes Multiple-model generalizations are common in the tracing community Direct discrete-time modeling is also possible, with piecewise-constant process noise As with target existence, we generally assume independent target motion; some generalizations exploit hierarchical OU processes γ + x x x 6

7 Time-Discretization of Stochastic Dynamics Nearly constant velocity (NCV) model X t FXt wt X x x F E E wt wt w q t Discrete-time NCV model X A X w X t A X exp t Ft t wt dt A exp t Ft Ew E w w Q qt 3 qt 3 qt qt 7

8 Linear Gaussian Filtering z v The models shown on the previous slide are all linear Gaussian processes If, additionally, we have linear measurements with additive Gaussian noise, the optimal linear estimator, with respect to the minimum mean squared error (MMSE) criterion, is the optimal estimator overall Notes x y ~N x y x y, Σ x Σ xy Σ yx Σ y = x + Σ xy Σ y y y E x,y x x y x x y T = Σ x Σ xy Σ y Σ yx This linear estimator may be written recursively: the well-nown Kalman filter This remains the optimal linear estimator even with non-gaussian noise We often employ the Kalman filter even if linearity or Gaussianity are violated z Example x v ~ U,,... n, x n n z n z v n n... zn n n x z,..., z minz,..., z * n x max n z n n n * z z feasibility 8

9 9 Stationary Target Evolution Modified Ornstein-Uhlenbec process The discrete-time form Notes Stability (& behavior) depend on closed-loop eigenvalues Stationarity: x + w x γ γ X w X X A X x X w E t t t t t t t t A exp exp exp exp exp exp exp exp Q Q Q Q Q w w E exp exp exp t t t q Q t t t q Q exp exp exp t t t q Q exp exp exp 4 ˆ ˆ ˆ, Re Im ˆ path followed by eigenvalues for increasing ˆ t A v p Q Q ˆ ˆ q p ˆ q v

10 The Multiple-Model MOU Process for Evasive Target Filtering Use of variable-structure IMM (VS-IMM) for move-stopmove targets has been studied (Kirubarajan 3) We may consider an OU-IMM approach Motion-dependent detection model (Koch 4): P D r = p d e ln r MDV MOU process for the move mode, second MOU process for the stop mode with small σ v Mode-dependent detection probabilities: P D i = p d erf MDV σ v i (D case), P D i = p d e MDV σ v i (D) Use P D i Λ i in standard IMM mode-matched lielihood computation, and use P D i for missed-detection scoring Advantages Monte Carlo simulation results are encouraging Allows for slow-moving targets below MDV Captures acceleration and deceleration naturally Avoids problematic stop-move transition Filter KF VS-IMM OU-IMM RMS 65.7 m 35.6 m 34.6 m position error (m) RMS velocity error (m) 8.5 m/sec 5.6 m/sec.8 m/sec

11 Kalman Smoothing Smoothing improves estimation performance and provides statistically-consistent interpolation The forward-bacward smoother requires a time-reversed motion model for: x + = A x + w x ~N, Q, w ~N, Q The reverse-time inematic model with uncorrelated disturbances w B ~N, Q B and terminal condition x n ~N, Q (Verghese 979): x = A B x + + w B A B = A I Q Q Q B = A Q Q Q Q A T A form of the forward-bacward smoother avoids the Information Filter on the bacward pass (Wall 98); slightly-simpler form uses inflated priors (Coraluppi 6), similar to recent distributed filtering wor (Koch 4) The equation below is not a valid time reversal: noise increments and prior at final time are not orthogonal x + = A x + w x = A x + A w (Note that there is no problem with the matrix inverse, as A is non-singular)

12 Bayesian Estimation Bayes ris Estimator Cost Xˆ Y Y C X ˆ, X J E E X E C Xˆ Y Y, X X C Xˆ Y, X E E C Xˆ Y, X, Y Y X X Y Minimum mean squared error (MMSE) estimation x, y x y Xˆ y EX y C (conditional mean) Minimum mean absolute error (MMAE) estimation x, y x y Xˆ y M X y C (conditional median) Minimum probability of error, or maximum a posteriori (MAP) estimation C x, y x y Xˆ y arg max f X y arg max f y X f X (conditional mode)

13 A Bayesian Estimation Example Prior distribution Conditional distribution Estimators Prior Posterior Points to note ˆ X MMSE f y x exp x, x y y x x exp xy, y MMSE, MMAE, MAP estimators differ & are nonlinear f x y f f ˆ Computation of the MAP estimator does not require (the unconditional density) f y f X MMAE y x y x y f f x f x dx y exp. 68 y x f x x exp x y f y xf x x yexp x y x ˆ X MMSE y xexp x y log Xˆ MMAE Xˆ MAP ˆ X MAP y x y y 3

14 The Multi-Target Tracing Problem Compute p X Z ; almost nobody attempts this (Kreucher 5) Extract from p X Z a set of tracs that is close to truth but note that MAP estimation is not meaningful here (Mahler 7) Consider the following posterior distribution: with probability p there is one target, distributed in state space according to N μ, Σ ; with probability p, there are no targets. How to determine the MAP estimate? Same difficulty exists when comparing lie-cardinality solutions with different target temporal support Some solution paradigms Detection-based vs. unified detection & tracing Centralized vs. distributed Hard vs. soft Labelled vs. unlabeled Association-based vs. association-free Sequential vs. batch Deterministic vs. stochastic (Multiple-Hypothesis Tracing) 4

15 A Labeled-Tracing, Association-Free Method Symmetric Measurement Equations (SME) approach Transform data association problem to one of nonlinear filtering (Kamen 99) Initial formulation assumes nown number of targets, no false alarms, no missed detections; subsequent wor relaxes these assumptions. Non-trivial filter convergence challenges; it appears most useful in conjunction with established methods (e.g. MHT), for group-target trac maintenance (Blacman 999) Useful as an approach to establishing tracing performance bounds (Daum 997) Example Consider a two-target trac-maintenance problem with measurements z i = x i + v i, E v i = σ, with i =, ; data association is unnown. Symmetric measurements: (i) association-independent zero-mean noise, (ii) invertible (non-linear) observation function h; (iii) full-ran Jacobean matrix H. y = c z + z y = c z + z σ 5

16 Outline Multi-target tracing preliminaries Multiple-hypothesis tracing Hypothesis aggregation 6

17 Association-Based MTT The MTT challenge: an intractable posterior probability distribution p X Z All measured data Hybrid-state decomposition p X Z = p X Z, q p q Z q MHT approach uses maximum a posteriori (MAP) estimation q = arg max q p q Z X arg max X p X Z, q Recursive formulation p q Z = p Z Z,q p q q p q Z p Z Z Unfortunately, this is a very large sum MHT finds lieliest explanation of data, and uses this to determine (approximately) the lieliest set of tracs Recursive processing enables real-time and computational feasibility 7

18 Is MAP Estimation a Good Idea? Single-target tracing: MAP doesn t minimize MSE (Braca ) This is true more generally: MMSE, MMAE, and MAP criteria lead to the same estimator under very narrow assumptions (linearity and Gaussianity) MAP estimation is useful in practice No need for computation of p Z Z MMSE and MMAE are difficult to define in a multi-target setting 8

19 Objections to Association-Based MTT Bayes rule prescribes the following: p q Z = p Z q p q Some issues that have been raised (Vo 8): p Z Since q depends on Z as it is prescribes how to explain the data, is p q a valid prior? Is p Z q a valid lielihood function? It is not clear whether p Z q p q is the joint density p Z, q : p q = p Z q p q dz must not depend on the data; This contradicts Z the fact that q does depend on Z! Does the normalizing constant even exist? We can address these concerns with a conditioning argument: p q Z = p q Z, Z = p Z q, Z p q Z p Z Z q is conditionally independent of Z given Z, hence p q Z is now a valid prior and p Z q, Z is now a valid lielihood function p q Z = p Z q, Z p q Z dz Z, Z = Z p Z Z = p Z q, Z p q Z q consistent with Z 9

20 Hypothesis-Oriented MHT (HO-MHT) The starting point p q Z = p q Z First numerator factor p Z Z = p q Z, Z = p Z Z,q, Z p q Z, Z p Z Z, Z p Z Z, q, Z = p Z Z, q Second numerator factor p q Z, Z = p q Z, Z, q p q Z, Z p q Z, Z = p q Z, Z,q p Z q,z p q Z p Z Z p q Z, Z = p q Z,q p q Z Denominator p Z Z, Z = p Z Z The final form p Z Z p Z Z p q Z = p Z Z,q p q q p q Z p Z Z

21 Assumptions Enabling Trac-Oriented MHT (TO-MHT) Target Poisson assumption Exponentially-distributed target inter-arrival (birth) times with parameter λ b Exponentially distributed target lifetime with parameter λ χ μ b t = λ b λ χ e λ χt p χ t = e λ χt Sensor Bernoulli assumption Each target is detected with probability p d Poisson false alarm assumption Large number of detection cells N and vanishingly small false detection probability p F, with p F N Λ

22 Trac-Oriented MHT Recursion Recall that we have: p q Z = p Z Z,q p q q p q Z p Z Z Hypothesis contribution, with r = Z, τ tracs, d detections, χ deaths, and b births: p q q = exp p dμ b Λ Λ r Filtering contribution: r! p χ χ p Z Z, q = f z j Z, q j J d The MHT recursion: c = exp p d μ b Λ Λ r r! p Z Z j Z f fa z j p q Z = c p q Z p χ χ p χ p d τ χ d p χ p d Λ p χ p d τ χ d p χ p d f z j Z,q j J d d pd μ b Λ j J f z j Z, q b j Jfa f fa z j Λf fa z j j J b b p d μ b f z j Z,q Λf fa z j death missed detection update birth

23 Trac-Oriented MHT Z, Z, Z3, Z,, Z, Z, Z3, Z,, Z, Z, Z3, Z,, Z, Z, Z3, Z,, Z, Z,, Z,, Z3, Z, Z,, Z,, Z3, Z, Z,, Z,, Z3, Z, Z,, Z,, Z3, Z, Z,, Z,, Z3, Z, Z,, Z,, Z3, Z, Z,, Z,, Z3, Z, Z,, Z,, Z3, Z,, Z3, Z,, Z,, Z,, Z3, Z,, Z,, Z,, Z3, Z,, Z,, Z,, Z3, Z,, Z, Z,,, Z, Z3, Z,, Z,,, Z, Z3, Z,, Z,, Z, Z3, Z,, Z,,, Z, Z3, Z,, Z,,, Z, Z3, Z,, Z,, Z, Z3, Z,, Z,,, Z, Z3, Z,, Z,,, Z, Z3, Z,, Z,, Z, Z3, Z,, Z,,, Z, Z3, Z,, Z,,, Z, Z3, Z,, Z,, Z, Z3, Z,, Z,,, Z,, Z,, Z3, Z,,, Z,, Z,, Z3, Z,, Z,, Z,, Z3, Z,,,,,,,,,,,,,,,,,,,,,,, Z Z Z3 Z Z Z Z3 Z Z Z Z3 Z,,, Z,, Z,, Z3, Z,,, Z,, Z,, Z3, Z,, Z,, Z,, Z3, Z,,, Z,, Z,, Z, Z,,, Z,, Z,, Z, Z,, Z,, Z,, Z, 3 Hypothesis-oriented view 3 3 Z, Z, Z3 Trac-oriented view Z Z Z Z Z Z 3 Z 3 Z 3 Z 3 Z 3 Z 3 3

24 Correcting Some Statements About MHT Both TO-MHT and HO-MHT are solving the same problem MAP global-hypothesis estimation problem MMSE estimation problem conditioned on the MAP global hypothesis In the literature, it is occasionally (and, in my view, erroneously) stated that HO-MHT is MAP-MMSE, while TO-MHT is ML-MMSE (e.g. Bar-Shalom 9) The global hypothesis q is well-posed as a state variable Mahler (7) claims that, since it is an observable, the use of Z in the MHT recursion is suspicious Note that we may express recursion in prediction-update form: p q Z = p q q p q Z (prediction step) p q Z = p Z Z,q p q Z p Z Z (update step) In practice, we are not concerned with predictions that ultimately lead to null posterior probability Mahler (7) raises the concern that measurement labelling introduces an a priori order on the data that may introduce a statistical bias in the MHT solution In fact, measurement labels are arbitrary, and have no impact on p q Z, hence there is no impact on the resulting MHT solution X, q 4

25 Practical Considerations Hypothesis generation Limit to a single global hypothesis for a single global association hypothesis Measurement gating Limit the number of association hypotheses Hypothesis pruning Reduce the number of association hypotheses Online output even for forensic problems Trac extraction Decoupled and sequential data association and trac extraction The sequential mode enables feedbac and proves beneficial Data association Trac extraction 5

26 An Example: Two Closely-Spaced Targets Newly-acquired detections Confirmed tracs T R3(t ) R(t ) R(t ) T How to perform trac maintenance for T & T? 6

27 High-Performance Hypothesis Resolution: Illustration with n-scan = (Tree Depth) trac coast T A T R R R R R3 trac termination Trac hypotheses resolved trac R R3 optimal global hypothesis new trac hypothesis R3 trac update objective subject to Account for tracs T and T Account for reports R, R, and R3 max cx Ax b x i, (for each vector element) maxcx Ax b x, N x~ b max ~ xx Ax b x, A simple, alternative scheme: trac-score normalization and greedy trac selection N 7

28 Performance Metrics Trac fragmentation (this lowers target purity) Non-traced target (this lowers target completeness) Legend Target Trac Localization error (this is averaged over all target-trac associations) Trac swap (this lowers trac purity) False tracs (these lower trac completeness) 8

29 Tracer Performance Calibration Model-based optimization Blac-box optimization Deterministic e.g. gradient-free generalized bisection algorithm Stochastic e.g. Marov Chain Monte Carlo (MCMC) with the Metropolis- Hastings algorithm Scalar objective Weighted combination of tracer metrics, including label-free optimal sub-pattern assignment (OSPA) Global lielihood Why global lielihood? Matches parameter optimization objective to MHT optimization objective No need for ground truth f y x f x p xy α xy = f y p yx α yx α xy = min, f y p yx f x p xy 9

30 Some Key Points MAP estimation based on p X Z is not meaningful MAP estimation based on p q Z is well posed MAP estimation may not achieve optimality for metrics of operational interest Recursive formulation of p q Z is the ey enabler for MHT Trac-oriented MHT and further simplifying approximations are needed as well for practical implementations 3

31 Outline Multi-target tracing preliminaries Multiple-hypothesis tracing Hypothesis aggregation 3

32 Case I: Exact Aggregation, Indistinguishable Global Hypotheses Richer set of global hypotheses, without increasing data-association hypotheses (Coraluppi 4) Consider multiple target birth and death times, for the same data-association sequence Consider never-detected targets The enhanced MHT recursion Aggregation over indistinguishable birth events on undetected targets Solution structure: decoupled observed-target and ghost-target solutions, data-independent ghost solution, common structure for visible targets (assuming stationary dynamics) Further aggregation is possible (with stationarity) p Q p Z exp b r! p p d d fa fa jj d JbJ fa p Q Z z Z, Q p pd fd z j Z, Q jj f z Z, Q jj d r fa fa fa f j j pdb fb z fa f fa z j b c j Z Z, Q, Q Enhanced MHT identifies optimal birth time interval (in this illustration, birth is followed by a misseddetection event) p d b u p d i Classical MHT assumes birth occurs in this time interval n! i b g i n i time measurement Further aggregation considers multiple intervals g deaths missed detections detections births unnoticed births ghost births 3

33 Benefits of Enhanced MHT [m] Improved trac extraction in temporally-staggered sensor settings Improved estimate of target cardinality in low p d settings The time-invariant solution No undetected births of visible targets Fixed ghost-target structure, with decreasing numbers at longer lifetime Example: 7 p.. 5 b p d t state t t 3 t 4 t time [sec] 33

34 34 34 Cardinality estimation for speed, privacy (Durand 3, Kodialam 7) Case II: Exact Aggregation, Indistinguishable Measurements!!! g i i g b d b d b b d b d d d Z Q p n p b d r r p p p p Z Q p.! i n i i n i n i n b i n i i n i n p p n p i n n i n e p p n p i n n i n p n p b n p Z L n Z L n p truth data CMHT CPHD Kalman filter b x b b x p x b p p p L d d p L I d x p Z L x x d Benefit of MHT aggregation and direct MHT-PHD comparison Cardinality MHT (CMHT) recursion (Coraluppi 4) Counting-targets limit of CPHD (Mahler 7) Kalman filter (suboptimal, problem is neither linear nor Gaussian)

35 Case III: Approximate Aggregation Trac coalescence (Fitzgerald 985), trac repulsion (Willett 7), and trac repulsion mitigation via trac-brea-trac (Coraluppi 9) Further mitigation via hypothesis aggregation (Coraluppi ) position [m] truth MHT ideal PDAF - - m distance between targets unique hypothesis hypothesis class - average log probability number of swaps optimal swap rate as a function of distance between targets time [sec].5 truth MHT ideal PDAF ECMHT equivalent measurement RMS error [m] m distance between targets swap rate displacement position [m] time [sec] number of swaps 35

36 Conclusions Multi-target tracing is a challenging problem that requires statistical estimation and combinatorial optimization Multiple-hypothesis tracing is a mathematically rigorous and well-performing solution paradigm Practical MHTs must use judicious hypothesis management, trac extraction, and hypothesis aggregation where possible MHT performance can be optimized via principled tracer modeling (this is hard) or through grey-box optimization 36

37 Recent Advances in Multiple-Hypothesis and Graph-Based Tracing Stefano Coraluppi NATO Lecture Series STO IST-55 Advanced Algorithms for Effectively Fusing Hard and Soft Information ITA NDL SWE GBR, September-October 6 37

38 Outline Distributed MHT Asynchronous MHT Graph-Based Tracing 38

39 Centralized MHT Single-sensor MHT wors well in many applications Multi-sensor MHT is effective in some applications fragmentation Without MHT processing With MHT processing ADULTS 3 Bistatic # Bistatic # Fused Theory Practice 39

40 Multi-Sensor Centralized MHT Small sensor networs Crazy Ivan Run SEABAR 7 Large sensor networs Full-networ processing Adaptive processing (based on context information) No active trac P N U P D Adaptive processing (based on tracer modeling) sojourn N t P D sojourn t N P U K P M Active trac P M sojourn t P M sojourn K t P M K detection probability sensor sensors sensors Fusion Performance Curves 3 sensors PERFORMANCE CURVE false trac rate FTR (per hour) 4

41 The Case for Distributed MHT Why distributed multi-sensor systems? Why distributed (i.e. multi-stage) single-sensor systems? Bandwidth and legacy constraints Robustness against target fading and registration errors Ability to handle disparate sensors, including active and passive sensors Improved performance with high-confidence same-sensor association An Example: Airborne Sense and Avoid Successful trac maintenance (yellow) Effectiveness in high-ambiguity settings (e.g. passive bistatic radar, HF radar) Processing flexibility with limited-quality sensor data (e.g. redundant measurements, hardware-induced artifacts, dim targets) An Example: GMTI Radar Tracing Significant processing flexibility, beyond examples discussed here Upfront static fusion via ML processing: Active sonobuoy field Residual processing: HF Radar Tracing fuse-before-trac processing BREST 9 trac-extract-trac processing 4

42 MHT with Active and Passive Sensors The use of virtual measurements Yes: trac initialization, hypothesis gating No: trac update, hypothesis scoring The use of equivalent measurements Equivalent-measurement (traclet) formation: sequence of measurements may replaced by an equivalent measurement: X + = X + L Z CX, P + = I LC P, L = P C T CP C T + R Columns of C T are fixed as the orthonormal eigenvectors of P + P corresponding to positive eigenvalues L = I P + P C T, Z = CX + L T L L T X + X, R = Λ Reduction in spurious-hypothesis formation, filter computations at fusion center, and bandwidth requirements Note: may not be full-dimensional; suboptimal filtering (common target process noise); sub-optimal trac scoring (even in single-sensor setting); processing latency The use of composite fusion logic Reduction in impact of airborne clutter while minimizing latency to single-sensor trac initiation 4

43 MHT for Dim Targets in Clutter Redundant returns: MHT-based measurement clustering Persistent sidelobes: multiple-mode filtering Hardware-induced artifact: constant-range tracing Dim target: two-stage dynamic tracing 43

44 Results The overall processing architecture Visualization of end-to-end result Low-power UAV-based GMTI radar data 44

45 Independent Performance Evaluation Significant in-house testing AFRL-provided CTESS STR Measures of Performance software that includes additional metrics, e.g. the optimal subpattern assignment (OSPA) metric AFRL performance evaluation against leading GMTI tracing technology providers Full set of metrics confirms STR Multi-Stage MHT (MS-MHT) is best performer by a sizeable margin 45

46 Handling Redundant Measurements Most tracing paradigms adopt a point-target assumption (i.e. at most one measurement per target per sensor per scan) These algorithms do not directly address extended-object tracing and limited-performance detectors that lead to multiple detections per target Many researchers have addressed this challenge Some researchers explicitly consider multiple-model phenomena, e.g. for over-the-horizon-radar (OTHR) sensors (MHT Sathyan 3) Other researchers assume a more challenging formulation, whereby the same detection model applies to all measurements (PDAF Kirubarajan, PHD Clar, Degen 4) The MHT recursion can be generalized (Coraluppi 6) Poisson case (p i = λi i! e λ ): p q q = Λr e Λ e μ b r! p χ χ λ i i! Λ i e λ p χ Global hypothesis factorization is achieved. i d i λ i i! Λ i e λ μ b The number of measurements in a cluster impacts explicitly the global hypothesis score; the benefit of the principled derivation is to establish the exact dependence. Substituting p d for e λ and limiting clusters to unity cardinality yields the familiar, classical MHT recursion. i b i D illustration of improved clustering (red) with respect to classical processing (blue) 46

47 Practical Considerations Single-stage redundant-measurement MHT is computationally infeasible The number of ways to cluster N measurements is given by the Bell number B N, which is (roughly) O(N N ) MHT Feasibility relies on distributed MHT processing Multi-stage processing solutions include (i) redundant-measurement MHT for static clustering, followed by classical MHT; or (ii) classical MHT, followed by redundant-measurement MHT for trac fusion MHT MHT 47

48 More on Trac Repulsion Not surprisingly, more measurements lead to improved localization performance Interestingly, analysis of bias errors shows that the impact of multiple measurements increases the tracrepulsion effect An explanation: 48

49 Outline Distributed MHT Asynchronous MHT Graph-Based Tracing 49

50 The Multi-INT Trac Fusion Challenge No viable multi-int technology solutions are available Distributed MHT performs correct multi-sensor fusion, but cannot scale to large problems Graph-based tracing (GBT) is fast, but cannot incorporate multi-int data (Castañon ) Paradigms are of interest that provide improved scalability while addressing the problem Asynchronous MHT (A-MHT) exploits forensic identity information to reduce spurious hypotheses and improve data association decisions L y n = L y L y i y i Kinematicbased tracs 3 i=,,n 4 L y L y i y i i=,,n Sparse identity information Significant trac overlap / confusion 5 5

51 Asynchronous Global Nearest Neighbor From an estimation perspective, MTT leads to surprises Distributed processing may outperform centralized processing in robustness (simplify registration, handle target fading) and performance (exploit single-sensor association) Delayed information (e.g. traclets) reduces spurious association hypotheses Asynchronous data association may perform better than insequence sequential processing position truth clairvoyant sequential 5asynchronous (approx) asynchronous (exact) average positional error clairvoyant sequential asynchronous (approx) asynchronous (exact) time number of sensor scans 5

52 Target purity Asynchronous MHT (A-MHT) MHT with trac-breaage logic S,W S t S S W A-MHT: batch-level (forensic) processing S W W t W terminated W S t S W W time Asynchronous MHT W S Processing steps (not time) W Classical MHT W W Hypothesis tree depth 5

53 Improved Graphical Exploitation Basic idea Forward-bacward coarse gating A-MHT on reduced hypothesis space Same object (SIGINT trac) Two MHT paths (but not A-MHT paths) N levels of data time N- levels of inematic tracs An A-MHT path 53

54 Improved Graphical Exploitation Basic idea Forward-bacward coarse gating A-MHT on reduced hypothesis space Forward light cone Same object (SIGINT trac) N levels of data No path enumeration in coarse gating time position m N- levels of inematic tracs 54

55 Improved Graphical Exploitation Basic idea Forward-bacward coarse gating A-MHT on reduced hypothesis space Bacward light cone Same object (SIGINT trac) N levels of data No path enumeration in coarse gating time position m N- levels of inematic tracs 55

56 Improved Graphical Exploitation Basic idea Forward-bacward coarse gating A-MHT on reduced hypothesis space Reduced hypothesis space Same object (SIGINT trac) N levels of data No path enumeration in coarse gating time position m N- levels of inematic tracs 56

57 A-MHT with Enhanced Association Processing Algorithm Clairvoyant MHT A-MHT Improved A-MHT (light cone) Average localization error (m) Fraction of correct associations Results based on Monte Carlo realizations 57

58 Outline Distributed MHT Asynchronous MHT Graph-Based Tracing 58

59 An Illustration of the Complexity of Some Tracing Paradigms The data Fusion inference with no approximations: hypothesis-oriented MHT Fusion inference with some approximations (Poisson targets and clutter): trac-oriented MHT Fusion inference with additional approximations (path independence): GBT 59

60 Multi-INT Graph-Based Tracing (MI-GBT) e Simple scenario: e v v e is location of ID measurement v i are inematic tracs v 3 t Generate separate graphs that define feasible set of flows for each ID; crucially, the emitter tracs do not show up as nodes. Constrain each inematic trac to be used exactly once. Source Nodes Kinematic Tracs Graph G (no ID) Graph G (ID e ) x v x v v v 3 x x x 3 x 3 x x v x v v x x Solve for consistent set of flows that maximize solution lielihood. Details in Coraluppi (6) Sin Nodes x v x v 6

61 MI-GBT Mathematical Formulation Minimize: J = e E i,j A c ij x ij. Subject to: Costs encode target existence and evolution statistics as well as sensor detection and localization statistics x ij,, i, j A, s. t. e E, Linear objective function: the unnowns are the edges in the multi-int graph. Each edge must be selected or not: no fractional values. :e E i: i,j A x ij =, j s. t. v j V, Each inematic trac must be used exactly once. x i i:v i V \ =, s. t. e E, Each emitter must be used exactly once: unity flow in each emitter sub-graph. x ji j: j,i A x ij j: i,j A =, i s. t. v i V, s. t. e E. Flow must be preserved within each sub-graph 6

62 Complexity Estimates Variables m sets of update tracs V inematic tracs in each set E emitter tracs Exact solution to the multi-int problem Integer linear program (ILP) size: A-MHT: M~O V m E MI-GBT: M~O m V E GBT: M~O m V ILP solution complexity: A-MHT & MI-GBT: assume O M 4 via LP relaxation GBT: O M 3 via min-cost networ flow (MCNF) or bipartite matching solution Approximate solution to multi-int problem Does not solve the multi-int problem 6

63 Multi-INT MCMC (MI-MCMC) Efficient sampling in global hypothesis space (Oh 9) Metropolis-Hastings sampling Moves follow a proposal distribution Moves that improve the solution lielihood are always accepted, otherwise an acceptance probability is applied Design includes merge, split, and swap moves A single swap move if executed here will correct some measurement assignment errors in the MCMC solution 63

64 Empirical Analysis on Small Scenarios Confirms Analytical Expectations GBT often violates identity constraints MHT (& A-MHT) have difficulty when emissions are temporally distant GBT and MI-GBT commit some errors when reasoning over inematic-only data, due to path-independence approximation Two realizations of the two-target scenario GBT & MI-GBT errors GBT error MHT & A-MHT error MHT & A-MHT recovery from error MI-MCMC converges to highlielihood solutions 64

65 MI-GBT, MCMC and MI-MCMC Exhibit Excellent Tracing Performance Metrics Localization error: reflects target & trac completeness Probability of correct association: reflects target & trac purity Benefit of fast MI-MCMC convergence will be important for large-scale problems (not needed here) (small is good here) (large is good here) Results on 5 Monte Carlo realizations for each scenario duration 65

66 More Challenging Problems Demonstrate Need for MI-MCMC (Hot Start) One realization of MCMC & MI-MCMC lielihood convergence MI-MCMC performance is best Results on 5 Monte Carlo realizations for each scenario duration 66

67 Multi-INT Challenge Problem Simulated inematic tracs for + targets High purity, high fragmentation Simulated identity emissions only on three high value targets (HVTs) Infrequent 5min scenario HVT # HVT #3 HVT # HVT # & #3 time-space confusion HVTs in multi-int challenge problem 67

68 Experimental Framewor data simulation multi-target tracing evaluation GBT HVT DD scenario Poisson point process Kinematic tracer model Identity sensor model MHT MI- GBT Metrics evaluator Algorithms: Graph-Based Tracer (GBT) Multiple-Hypothesis Tracer (MHT) Multi-INT Graph-Based Tracer (MI-GBT) 68

69 Results for HVT Scenario Non-ideal performance due to high target density, move-stop-move, limited identity detections MI-GBT outperforms MHT (on all metrics) and GBT (on HVT metrics) HVT purity lower than target purity as confusion is significant, by design MI-GBT HVT purity remains below unity, due to many closely-spaced inematic tracs 8% improvement Metric MHT GBT MI-GBT Target completeness Target purity Trac completeness Trac purity HVT completeness HVT purity

70 The Way Forward on Multi-INT Fusion Exploit complementary strengths of MHT, GBT, and MCMC solution paradigms MHT Kinematic sensor data... A-MHT MI- GBT MI- MCMC MHT Intelligence sensor data Consider more complex target phenomena that include emitters, persons, vehicles Emitters may be indistinguishable or may provide imprecise identification Multiple emitters may move between people Multiple people may move between vehicles Include other types of data including human-derived information Text-based reports Human intervention in fusion processing 7

71 Conclusions MHT is a powerful and mature approach to MTT MAP global hypothesis estimation is well posed, though it does not guarantee optimality with respect to metrics of interest There are surprising opportunities for performance gains in MHT Decoupling of data association and trac extraction Hypothesis aggregation over indistinguishable or similar hypotheses Distributed tracing in both multi-sensor and single-sensor settings Asynchronous data association The multi-int fusion problem goes beyond what advanced MHT alone can address Graph-based tracing must be exploited Stochastic-sampling methods may provide valuable solution refinement 7