TARGET TRACKING AND DATA FUSION: How to Get the Most Out of Your Sensors and make a living out of it FUSION 2017

Transcription

1 TARGET TRACKING AND DATA FUSION: How to Get the Most Out of Your Sensors and make a living out of it AN OVERVIEW OF TRACKING ALGORITHMS FOR CLUTTERED AND MULTITARGET-MULTISENSOR ENVIRONMENTS Yaakov Bar-Shalom, Distinguished IEEE AESS Lecturer University of Connecticut, ECE Dept. Box U-4157, Storrs, CT ybs@ee.uconn.edu Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 1/ 31

2 INFORMATION EXTRACTION AND FUSION Extract the maximum possible amount of information from each sensor by using appropriate sensor and target models. Quantify the corresponding uncertainties. Fuse the information from the various sources accounting for their uncertainties. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 2/ 31

3 INFORMATION EXTRACTION AND FUSION Extract the maximum possible amount of information from each sensor by using appropriate sensor and target models. Quantify the corresponding uncertainties. Fuse the information from the various sources accounting for their uncertainties. Method of approach Make things as simple as possible, but not simpler. A. Einstein Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 2/ 31

4 OUTLINE The evolution of the technology of tracking targets (objects of interest) in a cluttered environment starting from the Kalman filter (recursive LMMSE estimator for Markovian dynamic systems), the backbone of most current systems. Approaches for handling target maneuvers (unpredictable motion, including thrusting/ballistic targets) and false measurements (clutter). Advanced robust techniques with moderate complexity. Tracking of multiple targets. Tracking with multiple sensors: Fusion architectures. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 3/ 31

5 TRACKING WITH UNCERTAIN MOTION MODELS AND UNCERTAIN MEASUREMENTS TRACKING consists of: Estimation of the current state of a target (i.e., filtering) based on uncertain measurements to reduce the effect of the various noises. Calculation of the accuracy/credibility associated with the state estimate. TARGET MODEL UNCERTAINTIES motion is subject to: Random perturbations and/or Unknown maneuvers or motion model changes. Multiple models are needed to describe different target behavior modes. MEASUREMENT UNCERTAINTIES: Measured values from the target are inaccurate (noisy) Origin of the measurements is not perfectly certain the measurement(s) can be from the target of interest, false alarms, clutter or other targets data association is necessary. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 6/ 31

6 TYPES OF DATA ASSOCIATION MEASUREMENT-TO-MEASUREMENT association (Start-up). MEASUREMENT-TO-TRACK association (Continuation). Gating is done in the measurement space consisting of kinematic variables (position, Doppler, etc.) as well as feature components (signal strength, frequency, etc.). TRACK-TO-TRACK association (in the decentralized multisensor case) Given two tracks, each based on the data from a different sensor, are they from the same target? Common origin hypothesis test Combination (fusion) of the estimates if common origin hypothesis is accepted for improved accuracy. Gating is done in the state space with a weighted Cartesian norm and the dependence of the state estimation errors (across independent sensors!) has to be accounted for. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 8/ 31

7 MANEUVERING TARGETS The true measurement of a kinematic variable can be far from the predicted location this can cause problems in data association. Modeling of maneuvers: PROCESS NOISE (assumed by the filter pseudo noise" white or from a subsystem driven by white noise) [Q: why white?] with a single high level (conservative) with several discrete levels with heuristic hard switching based on the norm of the innovations (not practical in clutter) MULTIPLE MODELS use various models that differ in state equation and/or process noise levels, state dimension (e.g., add turn rate or thrust for thrusting/ballistic targets) with hard switching (based on some logic not practical in clutter) with soft (probabilistic) switching Interacting Multiple Model (IMM) estimator works in clutter. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 9/ 31

8 ALGORITHMS FOR TRACKING AND DATA ASSOCIATION The α-β Filter Uses fixed gains and fixed association gates (with possible simple logic of switching between several sets gain scheduling) It does not yield state estimation accuracies (covariances) This filter is actually the steady-state Kalman filter for a kinematic model (2nd order with acceleration as white process noise) with a given set of parameters. A similar filter (α-β-γ) is available for a 3rd order model Handling of measurement ambiguities Measurement selection nearest neighbor" (following thresholding of the signal) strongest neighbor" (following gating). This is then used in the state update as if it was the correct one. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 11/ 31

9 ALGORITHMS FOR TRACKING AND DATA ASSOCIATION The α-β Filter Uses fixed gains and fixed association gates (with possible simple logic of switching between several sets gain scheduling) It does not yield state estimation accuracies (covariances) This filter is actually the steady-state Kalman filter for a kinematic model (2nd order with acceleration as white process noise) with a given set of parameters. A similar filter (α-β-γ) is available for a 3rd order model Handling of measurement ambiguities Measurement selection nearest neighbor" (following thresholding of the signal) strongest neighbor" (following gating). This is then used in the state update as if it was the correct one. Q: How can one improve on the α-β filter in clutter? (outlw) Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 11/ 31

10 THE STANDARD KALMAN FILTER Selection of the measurement (from the gate) for state update is done according to A minimum distance rule" Nearest-Neighbor (NNSKF), or A feature, e.g., the signal strength Strongest Neighbor (SNSKF). The update is done with a time-varying gain (as opposed to the α-β filter), which is optimal if the assumed motion model parameters are correct and the selected measurement is the correct one. No accounting is made of the possibility that a clutter measurement might have been selected it is a standard filter. A logic can be used to effect a switching between several process noise levels ( spaghetti logic unless the SNR is very high). For nonlinear state or measurement models: Extended KF uses linearization. (KF workhrs; α β mule) Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 13/ 31

11 THE PROBABILISTIC DATA ASSOCIATION FILTER (PDAF) This filter calculates for all the current measurements from the gate the association probability of having originated from the target in track based on their locations/features time depth 1. The state is then updated with a weighted combination of these measurements with the weights being the above association probabilities soft association decision. The covariance associated with the resulting state estimate includes a term due to the measurement origin uncertainty. This algorithm is suboptimal since it lumps" all the measurements in a single state estimate it replaces a Gaussian mixture by a single Gaussian using moment matching It is simple (1.3 the NNSKF) and yields significantly improved tracking performance. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 14/ 31

12 THE PROBABILISTIC DATA ASSOCIATION FILTER (PDAF) This filter calculates for all the current measurements from the gate the association probability of having originated from the target in track based on their locations/features time depth 1. The state is then updated with a weighted combination of these measurements with the weights being the above association probabilities soft association decision. The covariance associated with the resulting state estimate includes a term due to the measurement origin uncertainty. This algorithm is suboptimal since it lumps" all the measurements in a single state estimate it replaces a Gaussian mixture by a single Gaussian using moment matching It is simple (1.3 the NNSKF) and yields significantly improved tracking performance. Some implementations of the PDAF Jindalee over-the-horizon radar in Australia the only algorithm that was capable of working in very heavy clutter At Raytheon: Hawk SAM, ROTHR, THAAD, ASDE, GBR At EUROCONTROL (combined with the IMM). Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 14/ 31

13 THE MULTIPLE HYPOTHESIS TRACKER (MHT) This algorithm, with time depth > 1 Splits the existing track (within a sliding window) whenever there is an association ambiguity and follows each branch (sequence of measurements) with a probability calculation Updates the tracks for each hypothesis with a KF/IMM Has built-in track initiation capability. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 15/ 31

14 THE MULTIPLE HYPOTHESIS TRACKER (MHT) This algorithm, with time depth > 1 Splits the existing track (within a sliding window) whenever there is an association ambiguity and follows each branch (sequence of measurements) with a probability calculation Updates the tracks for each hypothesis with a KF/IMM Has built-in track initiation capability. Disadvantages Computational and memory requirements (NP-hard) Very complex data management and debugging Multitude of the output all the hypotheses are put out and it is very complicated to present an overall picture: one can display the most likely hypothesis which can jump" It does not provide target existence probabilities. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 15/ 31

15 THE MULTI-BERNOULLI FILTER (MBF) A RFS (random finite set) approach It provides, for each track estimate and covariance, the corresponding target existence probability Multi-Object Particles (MOP) represent a hypothesized set of tracks with a joint probability The association of new measurements to MOPs is done via 2-D assignment (auction) The posterior joint probabilities are marginalized to obtain the existence probabilities of a target behind a track (which appears in several MOPs). Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 16/ 31

16 THE MULTI-BERNOULLI FILTER (MBF) cont d Applied to a real world motivated scenario of a salvo of objects observed from a single stationary optical sensor which has initially unresolved measurements The MBF was used to associate the 2D measurements which were then used to predict the full 3D trajectories ch Modeling of unresolved measurements: variance is (unbeknownst to the tracker) equal to a multiple of the single target measurement variance A physics-based model is under development for the measurement noise variance of resolved and unresolved measurements from an optical sensor (preliminary results for resolved measurements in the paper by Balasingam at F 17). Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 17/ 31

17 frames in the FPA (the images are inverted due to the optics while the true trajectories are from the bottom left of the FOV and upwards, the images are from the top right and downwards.) Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 18/ 31

18 THE INTERACTING MULTIPLE MODEL (IMM) ESTIMATOR The Interacting Multiple Model estimation algorithm is a very efficient recursive scheme with fixed requirements for systems with switching models (hybrid systems) a self-adjusting variable-bw estimator. The IMM estimator runs Kalman filter (or EKF) modules simultaneously based on several target models (e.g., non-maneuvering and maneuvering models or thrusting and ballistic) in an interacting manner constantly exchanging information yields the current model" probability conditioned on the available data. The output consists of mode probabilities, combined state estimate weighted by the mode probabilities and covariance of the combined state estimate. The IMM was the key that made it possible for an off-the-shelf fish to intercept and incoming fish in a sea test. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 19/ 31

19 EXTENSIONS TO CLUTTERED ENVIRONMENT The IMM, which has a modular architecture, has been extended (IMMPDAF) for tracking a target in clutter by using the PDAF as the basic filter module and making suitable changes in the model probability calculation to account for the target P D and the clutter. Major advantages simplicity of implementation modest and fixed computational and memory requirements effects soft switching between the models never totally right, never totally wrong Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 20/ 31

20 EXTENSIONS TO CLUTTERED ENVIRONMENT The IMM, which has a modular architecture, has been extended (IMMPDAF) for tracking a target in clutter by using the PDAF as the basic filter module and making suitable changes in the model probability calculation to account for the target P D and the clutter. Major advantages simplicity of implementation modest and fixed computational and memory requirements effects soft switching between the models never totally right, never totally wrong The IMMPDAF has been fielded in an active hull mounted sonar to track low-snr maneuvering targets. The IMM has been successfully used in combination with assignment hard association decision for real ATC data (800 targets, 5 radars). Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 20/ 31

21 LARGE-SCALE ATC USING IMM/ASSIGNMENT ESTIMATOR Scenario: 5 FAA/JSS radars, 800 targets nmi RMS prediction errors 1 sensor.. 2 sensors 3 sensors... 4 sensors time (sec) Solution: 2-D assignment algorithm for data association in conjunction with the IMM estimator for tracking Real-time capability: IMM/Assignment tracker processed 5 minutes worth of data in less than one minute Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 21/ 31

22 LIMITS OF PERFORMANCE Major issue: Is there enough information in the data? Information in the sense of Fisher: a matrix whose inverse, if it exists, yields the lowest achievable covariance in estimation (the CRLB; in general there is no guarantee that one can achieve this bound). If P D < 1 and P F A > 0, one has a new situation: an information reduction factor (IRF) has been quantified there is less information and the CRLB in clutter (CRLBiC) is higher than the conventional CRLB. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 22/ 31

23 LIMITS OF PERFORMANCE Major issue: Is there enough information in the data? Information in the sense of Fisher: a matrix whose inverse, if it exists, yields the lowest achievable covariance in estimation (the CRLB; in general there is no guarantee that one can achieve this bound). If P D < 1 and P F A > 0, one has a new situation: an information reduction factor (IRF) has been quantified there is less information and the CRLB in clutter (CRLBiC) is higher than the conventional CRLB. In real world problems we have to understand the limits due to finite (perhaps insufficient) information in the sensor data the existing information seek efficient algorithms such that the extracted information is equal to the existing information, or as close as possible to it, subject to implementation constraints. Example: The ML-PDA for TBM acquisition is efficient for LO targets down to 4dB SNR in a cell average signal strength is 1.6 times the RMS noise. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 22/ 31

24 slant range (km) LOW OBSERVABLE TBM ACQUISITION USING ML-PDA Time (s) z (km) Scenario: km missile acquisition range, data for 6 s at 10 Hz Difficulty: low SNR high false alarm density (low observability) Solution: ML-PDA estimator with features to initialize tracks Efficient meets the CRLB in clutter (CRLBiC) down to 4 db SNR extracts all available information. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 23/ y (km) x (km)

25 MULTISENSOR TRACKING Prerequisites for successful data fusion: Sensor registration (alignment) Reliable statistical description of the uncertainties in each sensor s data Reliable estimation accuracies track error covariances. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 25/ 31

26 MULTISENSOR TRACKING Prerequisites for successful data fusion: Sensor registration (alignment) Reliable statistical description of the uncertainties in each sensor s data Reliable estimation accuracies track error covariances. An interesting results in fusion from distributed local trackers is that local tracks using independent sensors have correlated errors. This correlation is due to the common process noise the motion uncertainty model is common, only the measurement uncertainties are independent across local trackers and is quantified by crosscovariances". Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 25/ 31

27 SINGLE SENSOR TRACKING FOLLOWED BY TRACK FUSION W/O FEEDBACK Signal Association Filter processing update Tracks Gate computation Track to track association and fusion Signal Association Filter processing update Tracks Gate computation This fusion, even if performed optimally (with the exact cross-correlations between the local state estimation errors), is known to be slightly inferior (10 15%) compared to the centralized configuration. Explanation: optimal fusion of locally optimal tracks is globally suboptimal because the locally optimal filter gains are not globally optimal. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 26/ 31

28 SINGLE SENSOR TRACKING FOLLOWED BY TRACK FUSION W/O FEEDBACK Signal Association Filter processing update Tracks Gate computation Track to track association and fusion Signal Association Filter processing update Tracks Gate computation This fusion, even if performed optimally (with the exact cross-correlations between the local state estimation errors), is known to be slightly inferior (10 15%) compared to the centralized configuration. Explanation: optimal fusion of locally optimal tracks is globally suboptimal because the locally optimal filter gains are not globally optimal. It is critical that each estimate is consistent (has a covariance that is neither optimistic nor pessimistic). Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 26/ 31

29 CENTRALIZED CONFIGURATION FOR MULTISENSOR DATA FUSION In this configuration all the associations and tracking are carried out at a central location. Signal processing Association Filter update Tracks Signal Association processing Gate computation This provides the best performance but it has high communication bandwidth requirements. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 27/ 31

30 SUMMARY The α-β and the NNSKF/SNSKF approaches are overly simplistic and outdated. At the other extreme, the MHT technique is very complex. The use of discrete optimization (rather than enumerative hypothesis evaluation) makes it more efficient and brings it to the stage where real-time implementation is feasible. For a single target, the IMMPDAF is believed to be the best available compromise between complexity and performance. Its capabilities in a realistic cluttered environment have been shown in a series of Navy Benchmark problems. The use of the IMM (combined with PDAF or MHT) has, with its built-in auto-tuning, the potential of overcoming the problem that many filters cannot be tuned for a wide enough range of situations. For VLO targets the ML-PDA is the best algorithm because it can extract all the relevant information from the data it meets the CRLB in clutter down to 4dB SNR. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 28/ 31

31 SUMMARY (Cont d) For multisensor track-to-track fusion, the cross-correlations between local tracking errors have to be accounted for. Optimal track-to-track fusion on demand is slightly inferior to optimal centralized tracking but can save communication BW. Sensor alignment (registration) hinges on observability, which is not always guaranteed. Sensor resolution modeling still needs work. Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 29/ 31

32 Major Achievements in Multisensor Fusion a (very) partial list Distributed filtering algorithms Distributed tracking with data association Distributed fusion (information graph, hierarchical, consensus based) Heterogeneous data fusion (active-passive, collocated stationary passive-passive for ranging, multispectral) Feature, attribute and classification aided fusion Statistical efficiency of passive-passive asynchronous data fusion Sensor networks for security Environmental monitoring Autonomy (ground, sea or air vehicles) Space surveillance Human motion tracking and recognition Yaakov Bar-Shalom TTFMOSTSvb2a (June 12, 2017) Target tracking and data fusion 31/ 31

33 Heterogeneous Track-to-Track Fusion Ting Yuan, Yaakov Bar-Shalom and Xin Tian University of Connecticut, ECE Dept. Storrs, CT {tiy, ybs, T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

34 Motivation and Difficulties of Heterogeneous T2TF Motivation: There is need to fuse tracks from active and passive sensors. Compared with homogeneous track-to-track fusion (T2TF) that assumes the same system model for different local trackers, the heterogeneous case poses two major difficulties: The model heterogeneity problem: fuse tracks from different state spaces (related by a certain nonlinear transformation). The estimation errors dependence problem: recognized as the common process noise effect", which is quantified by the crosscovariance matrix. T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

35 Heterogeneous T2TF Problem state-space models Consider the following state-space models at sensor i x i (k + 1) = f i [x i (k)] + v i (k) (1) z i (k) = h i [x i (k)] + w i (k) (2) at sensor j x j (k + 1) = f j [x j (k)] + v j (k) (3) z j (k) = h j [x j (k)] + w j (k) (4) where x i and x j are in different state spaces (with unequal dimensions). f ( ) and h ( ) are nonlinear in general v ( ) denote the process noises w ( ) denote measurement noises. Note that the two heterogeneous trackers are assumed synchronized and the time index k for sampling time t k will be omitted if there is no ambiguity. T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

36 Heterogeneous T2TF Problem different state vectors Let x i be the larger dimension state (e.g., full Cartesian position and velocity in 2-dimensional space for tracking with an active sensor) x i = [ x ẋ y ẏ ] (5) and x j be the smaller dimension state (e.g., angular position and velocity for tracking with a passive sensor) x j = [ θ θ ] These state vectors have the nonlinear relationship (6) x j = g(x i ) (7) T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

37 Heterogeneous T2TF Problem local tracks From sensor i one has the track ˆx i the covariance matrix P i. From sensor j one has the track ˆx j the covariance matrix P j. The problem is how to carry out the fusion of the track ˆx i with P i and the track ˆx j with P j to achieve improved estimation performance over single sensor track quality. comparable estimation performance to the track quality of centralized measurement tracker/fuser (CTF). T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

38 The LMMSE Fuser The LMMSE fused estimate of x = x i with observation" z = ˆx j (using the fundamental equations of LMMSE) is [ ( ˆx i LMMSE = ˆx i + P xzp 1 ˆx j g ˆx i)] (8) with the corresponding fused covariance matrix where zz P i LMMSE = P i P xzp 1 zz P xz (9) [( P xz = E x i ˆx i) (ˆx ) ] j g(ˆx i ) P i (G i ) P ij (10) [( ) ( ) P ] zz = E ˆx j g(ˆx i ) ˆx j g(ˆx i ) P j G i P ij P ji (G i ) + G i P i (G i ) (11) with G i the Jacobian of g(x i ) G i = [ x ig(x i ) ] (12) x i =ˆx i and P ij the crosscovariance matrix. T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

39 The ML Fuser Under the Gaussian assumption, the heterogeneous T2TF problem can be solved by minimizing the negative log-likelihood function L = ln p(ˆx i, ˆx j x i ) ([ ] [ ]) [ ˆx i x i P i ˆx j x j P ji P ij P j ] 1 ([ ] [ ]) ˆx i x i ˆx j (13) Then, with x j = g(x i ), the ML fused estimate is the solution of x j x il = 0 (14) Because of the nonlinearity of the function g(x i ), we solve (14) by numerical search (or maximize directly (13) w.r.t. x i ). The fusion result is denoted as ˆx i ML with the corresponding covariance matrix ( [ PML i ] [ ] = I G i P i P ij 1 [ ] ) 1 I P ji where G i is defined in (12) and I is the identity matrix (4 4 in our case). P j G i (15) T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

40 LMMSE and ML Fuser are the Same! For the homogeneous case (with the same state x j = x i ), the (Bayesian) MMSE approach yields exactly the same result as the Fisherian (i.e., non-bayesian) ML approach assuming Gaussian errors. MMSE approach: one estimate is the prior, the other is an observation. ML approach: no prior, each estimate is an observation. Bayesian recasting of the ML approach: use a diffuse (non-informational) prior!! T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

41 A Typical Scenario measurement and ground truth The measurements an active sensor located at (x a, y a) with measurements 1 range: r = (x x a) 2 + (y y a) 2 + w r ( ) 2 azimuth angle: θ a = tan 1 y ya x x a + w a a passive sensor located at (x p, y p) with measurements ( ) 1 only azimuth angle: θ p = tan 1 y yp x x p + w p where w r, w a and w p are assumed to be mutually independent zero mean white Gaussian noises with standard deviations (SD) σ r, σ a and σ p, respectively. The ground truth 1 A target moving with a constant speed of 250 m/s with initial state in Cartesian coordinates (with position in m) x(0) = [ x(0) ẋ(0) y(0) ẏ(0) ] = [ ] (16) At k = 10 (t = 100 s) it starts a left turn of 2 /s for 30 s, then continues straight until k = 20, at which time it turns right with 1 /s for 50 s, then left with 1 /s for 90 s, then right with 1 /s for 50 s, then continues straight until 50 s. T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

42 A Typical Scenario overview Y (m) 9 x The Scenario and Sample Active Sensor Measurements 380s 335s 245s 200s Active sensor Passive sensor True trajectory Active sensor measurement Turning point 130s 100s X (m) x 10 4 Figure 1: The scenario, with the target true speed 250 m/s, the active sensor located at ( , ) m with sampling interval T a = 5 s and the passive sensor located at ( , ) m with sampling interval T p = 1 s. T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

43 Local tracker Design the active sensor IMM The active sensor IMM estimator has two modes mode 1 linear nearly constant acceleration (NCA) model: implemented as discretized continuous white noise acceleration (CWNA) model. mode 2 nonlinear nearly coordinate turn (NCT) model: implemented as discretized continuous coordinate turn (CCT) model [Morelande&Gordon, ICASSP 2005]. The (target state-dependent) process noise covariance matrix of the NCT model is (details in [MG2005]) T a 3 3 Q i a [x(k)]= ẋ 2 (k) ẋ 2 (k)+ẏ 2 (k) qv T a 3 ẏ 2 (k) 3 ẋ 2 (k)+ẏ 2 (k) qv T q Ω where q a and q Ω are the power spectral densities (PSDs). Note that the process noise induced RMS change in the velocity and in the turn rate over sampling interval T a are qvt d v= a Ta q d Ω= Ω T a Ta whose physical dimensions are linear acceleration and turn acceleration, respectively. The CTF uses the same IMM design (CTF IMM for short) as the active sensor IMM. (17) (18) T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

44 Local tracker Design the passive sensor KF For the passive sensor, in the scenario considered, the target maneuvering index is very small and the target maneuvers are nearly unobservable by the passive sensor. Consequently, a linear KF (rather than IMM estimator) is used [KB2003]. The motion model used is the discretized continuous Wiener process acceleration (CWPA) model (with angle, angle rate and angle acceleration). The covariance matrix of the process noise is Q j p (k) = Tp 5 20 Tp 4 8 T p 3 6 Tp 4 8 Tp 3 3 T p 2 2 Tp 3 6 Tp 2 2 T p qp (19) where q p is the process noise PSD. The process noise induced RMS change in the angular acceleration over T p are d p = qptp T p (20) whose physical dimension is the angular jerk (derivative of acceleration). Note that d p with d v and d Ω as in (18) are the design values used to select the process noise PSDs for the local trackers. T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

45 The Heterogeneous T2TF The measurement noises: the active sensor σ r = 20 m and σ a = 5 mrad; the passive sensor σ p = 1 mrad. Note An unbiased measurement conversion from polar coordinates to Cartesian coordinates is done for the active sensor measurements for filtering. The process noise intensities settings Active sensor: d a (m/s 2 ) d Ω (mrad/s 2 ) Mode 1 (NCA) 0.2 N/A Mode 2 (NCT) 1 2 Passive sensor: d p = 0.04 mrad/s 3. The IMM transition probability matrix is [ ] π = with initial mode probability vector [ 0.9, 0.1 ]. The estimate ˆx i (k) from the active sensor IMM with the corresponding covariance matrix P i (k) and the estimate ˆx j (k) from the passive KF with the corresponding covariance matrix P j (k) are fused the heterogeneous T2TF. The fusion performance is compared with the corresponding single active sensor IMM track and the CTF IMM track. (21) T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

46 The Sample Crosscorrelation MC simulations In view of the fact that there is no known way to evaluate the crosscovariance of the estimation errors in the case of heterogeneous trackers, a Monte Carlo (MC) investigation of these errors crosscorrelations is carried out. The sample crosscorrelation coefficient between the lth component of x i and the hth component of x j in M MC runs at a particular point in time is M ˆρ M m=1 = (ˆxi l,m xi l )(ˆxj h,m xi h ) x i l xj [ M ] [ h m=1 (ˆxi l,m M ] (22) xi l )2 m=1 (ˆxj h,m xj h )2 T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

47 The Sample Crosscorrelation position-to-position/velocity Sample crosscorrelation coefficient, 1000 MC runs Time (s) Figure 2: The sample crosscorrelation for x and ỹ with θ and θ. (Some are positive and some are negative) x and θ ỹ and θ x and θ ỹ and θ T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

48 The Sample Crosscorrelation velocity-to-position/velocity Sample crosscorrelation coefficient, 1000 MC runs Time (s) Figure 3: The sample crosscorrelation for ẋ and ẏ with θ and θ. (Some are positive and some are negative) ẋ and θ ẏ and θ ẋ and θ ẏ and θ T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

49 The Sample Crosscorrelation negligible crosscorrelation It can be seen from the MC simulations that Some of the crosscorrelations are positive and some are negative. The crosscorrelations depend on the relative geometry of the two sensors and the target, as well as the target maneuvers. For the nonlinear case, neglecting the crosscorrelations makes the fusion sometimes optimistic and sometimes pessimistic, but the effect is small. This supports the approach of ignoring the dependency between the tracks from different local sensors. Thus, since the maneuvers are unknown and scenario dependent, we pursue the heterogeneous T2TF without considering the crosscorrelation between the estimation errors. T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

50 Simulation Results LMMSE fuser (RMSE in position space) Position RMSE (m) Position RMSE, 1000 MC runs Pos. RMSE: active sensor IMM Pos. RMSE: CTF IMM Pos. RMSE: LMMSE fuser Maneuvering interval Time (s) Figure 4: The position RMSE for LMMSE fuser. (Heterogeneouse T2TF is superior to CTF IMM during model switching) (ML fuser has practically the same performance as LMMSE fuser) T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

51 Simulation Results LMMSE fuser (RMSE in velocity space) Velocity RMSE (m/s) Velocity RMSE, 1000 MC runs Vel. RMSE: active sensor IMM Vel. RMSE: CTF IMM Vel. RMSE: LMMSE fuser Maneuvering interval Time (s) Figure 5: The velocity RMSE for LMMSE fuser. (Heterogeneouse T2TF is superior to CTF IMM during model switching) (ML fuser has practically the same performance as LMMSE fuser) T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

52 Simulation Results maneuvering mode probability (NCT) Mode probability Mode probability of NCT NCT(active sensor) NCT(CTF) Maneuvering interval Time (s) Figure 6: Maneuvering mode probability (NCT) in the active sensor IMM and CTF IMM. (Active sensor IMM is superior to CTF IMM!) T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21

53 Conclusions The LMMSE and the ML approaches (equivalent!!) for heterogenous T2TF can effectively achieve improved performance over the single sensor track quality and superior performance to the CTF track. The estimation errors crosscorrelation has been examined by MC simulations. The crosscorrelation of the estimation errors from heterogeneous local sensors is too complicated to capture. The use of the passive measurements in the CTF IMM clouds" the maneuvers it is preferable to have an active sensor IMM (which does detect the maneuvers) and a passive sensor KF (since the passive sensor is almost blind" to the maneuvers) and fuse the outputs of these two local trackers. The freedom available to each local sensor to flexibly design a more suitable local estimator allows the heterogeneous T2TF approach to achieve a better estimation performance than the CTF IMM in the scenario considered. The LMMSE T2TF has practically the same performance as the ML T2TF and can be considered as an effective a simpler alternative for the numerical search required by the ML approach. T. Yuan, Y. Bar-Shalom and X. Tian Heterogeneous TtTF 383V2a June 12, / 21