SENSOR SCHEDULING AND EFFICIENT ALGORITHM IMPLEMENTATION FOR TARGET TRACKING. Amit Singh Chhetri

Transcription

1 SENSOR SCHEDULING AND EFFICIENT ALGORITHM IMPLEMENTATION FOR TARGET TRACKING by Amit Singh Chhetri A Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy ARIZONA STATE UNIVERSITY May 2006

2 SENSOR SCHEDULING AND EFFICIENT ALGORITHM IMPLEMENTATION FOR TARGET TRACKING by Amit Singh Chhetri has been approved April 2006 APPROVED:, Co-Chair, Co-Chair Supervisory Committee ACCEPTED: Department Chair Dean, Division of Graduate Studies

3 ABSTRACT Recent advances in sensor technology coupled with embedded systems and wireless networking has made it possible to deploy sensors for numerous applications including target tracking, environmental science, defense information, and security. Sensor scheduling, a process to allocate sensing resources by optimizing a performance metric over a future time-horizon under constraints, is an effective method to improve performance for such problems. This work investigates myopic (one step ahead) and non-myopic (multiple steps ahead) sensor scheduling algorithms for target tracking applications. Two methods of predicting tracker performance are developed that can be used for target tracking applications. The first is covariance-based, and it can be used with covariancebased scheduler costs. The second is unscented transform-based and it can be used with arbitrary scheduler costs. In application, both methods give a significant improvement in tracking performance over the tracking performance without sensor scheduling. The use of non-myopic sensor scheduling is often restricted due to an exponential dependency of computational and memory requirements on the length of prediction horizon. For non-myopic scheduling, two branch-and-bound based optimal pruning algorithms were investigated. Monte Carlo simulations demonstrated that they significantly reduce the computational and memory requirements of non-myopic scheduling without compromising the tracking accuracy. In networks of tiny inexpensive energy-constrained sensors, tracking involves a natural trade-off between performance and energy consumption. Non-myopic scheduling to minimize the network energy consumption subject to maintaining a desired tracking accuracy in the target s position estimate was studied; non-myopic sensor scheduling significantly iii

4 improved the energy performance of the network compared to myopic scheduling. Using the covariance-based scheduling framework, large sensor scheduling problems involving a trade-off between sensor-usage costs and tracking performance can be posed as binary (0-1) convex programming problems. In some special cases, these problems simplify to 0-1 mixed integer programming problems. The 0-1 convex programming and 0-1 mixed integer programming problems are solved using outer approximation and linear programming relaxation based branch-and-bound algorithms. Simulation results demonstrated that the 0-1 programming allows optimal solution to problems of up to sensors typically in the order of seconds. iv

5 ACKNOWLEDGMENTS I am greatly indebted to my co-advisors, Dr. Antonia Papandreou-Suppappola and Dr. Darryl Morrell for providing me an opportunity of working for them and for continually supporting my Ph.D. study. Working under them in the field of statistical and sensor signal processing has been a very enriching and a memorable experience. I am very grateful for their guidance in both my academic and professional pursuits, and for their ever-helpful and constructive suggestions throughout my Ph.D. study. I am grateful to Dr. Andreas Spanias, Dr. Tom Taylor, and Dr. Junshan Zhang for taking their time to be my committee members. Their comments, questions, and suggestions during my qualifying and comprehensive examinations have significantly aided in improving the quality of this dissertation. I would also like to thank my colleagues of the Sensor and Signal Processing Group at ASU who have been very helpful with their suggestions in my numerous discussions with them. In particular, thanks go to Ioannis Kyriakides, Himanshu Shah, Sandeep Sira, and Fengjun Xi, who have provided valuable inputs during the various stages of my Ph.D. study. I am highly grateful to my parents, my sisters, and my wife who have been very instrumental in the successful completion of my Ph.D. Their constant motivation and support for the cause of my Ph.D. is something that I would always cherish in the years to come. This work was supported by the DARPA Integrated Sensing and Processing program through a contract with the Office of Naval Research N C v

6 TABLE OF CONTENTS Page LIST OF TABLES xii LIST OF FIGURES xiii 1 INTRODUCTION Motivation The Sensor Scheduling Problem and Solution Techniques Target Tracking and Filtering Review of Prior Work in Sensor Scheduling Contributions of this Work Development of Sensor Scheduling Algorithms Application Scenarios for Sensor Scheduling Algorithms Computationally Efficient Methods for Sensor Scheduling Outline of This Dissertation Adopted Notation REVIEW OF TARGET TRACKING Target Tracking Formulations Bayesian Approach to Target Tracking Recursive Filters for Target Tracking Kalman Filter Extended Kalman Filter vi

7 Page Particle Filter Short Overview on Sensors Radar Sensors Infrared Sensors Acoustic Sensors NON-MYOPIC SENSOR SCHEDULING Optimization Framework Types of Cost Functions Covariance-based Costs Information Theoretic Costs Proposed Non-myopic Scheduling Algorithms Covariance-based sensor scheduling Unscented transform-based sensor scheduling Simulation Examples of Tracking Scenarios for Sensor Scheduling Radar and IR Scheduling Problem Bearing-only Sea Surface Tracking using Acoustic-homing Torpedo Comparisons of UTB and CB scheduling BRANCH-AND-BOUND BASED TREE SEARCH ALGORITHMS FOR NON- MYOPIC SENSOR SCHEDULING Overview on Tree Search Techniques Exhaustive Search vii

8 Page Heuristic and Greedy Search Branch-and-Bound Search Branch-and-Bound based pruning algorithms for sensor scheduling Proposed Pruning Algorithms BFS-GS Pruning Algorithm UCS pruning algorithm Suboptimal pruning Non-myopic Sensor Scheduling Simulation using Pruning NON-MYOPIC SENSOR SCHEDULING FOR ENERGY-EFFICIENT TARGET TRACKING IN SENSOR NETWORK Tracking Scenario and Formulations Energy Consumption Model for Sensor Network Non-myopic Sensor Scheduling Problem Optimization Framework Covariance Prediction using the PCRLB Scheduling Procedure Constructing an Energy Tree Uniform-Cost Based Tree Search Simulations and Results Tracking Results Search Results viii

9 Page 6 BINARY PROGRAMMING FRAMEWORK Introduction to 0-1 Programming Problems Complexity of 0-1 Programming Problems Important Categories of 0-1 Programming Problems Algorithms and Techniques For 0-1 Programming Problems Linearization Technique Branch-and-Bound Technique The Outer Approximation Algorithm MYOPIC SENSOR SCHEDULING USING 0-1 PROGRAMMING Formulation of Myopic Sensor Scheduling Problems as 0-1 Programming Problems MinCost Sensor Scheduling Problem MinError Sensor Scheduling Problem Problem Description Perimeter Tracking Scenario Myopic Sensor Scheduling Formulation for MinCost and MinError Problems Sensor-usage Cost Model based on Sensor-Energy Consumption Tracking Error Prediction for Sensor Scheduling Heuristics for Myopic Sensor Scheduling MIP and OA Scheduling Methods for MinCost and MinError Problems MIP Scheduling Method for MinCost-I Problems ix

10 Page OA Scheduling Method for MinCost and MinError Problems Simulations and Results Application of the MIP scheduling method to the MinCost-I Problem Application of OA Scheduling Method to MinCost-II Problem Performance Comparison of MIP and OA Scheduling Methods Application of OA Scheduling Method to MinError Problem Application of OA Scheduling Method to Underwater Tracking Scenario Extension of OA Algorithm to Non-myopic Sensor Scheduling CONCLUSIONS AND FUTURE WORK Conclusions Development of Sensor Scheduling Algorithms Computational Efficient Techniques for Sensor Scheduling Sensor Scheduling in a Network of Sensors Future Work Improvements in the OA and MIP scheduling methods Non-myopic Sensor Scheduling using OA Scheduling Method Approximation Algorithms for Sensor Scheduling Inclusion of Clutter Multiple Target Tracking APPENDIX A DERIVATION OF THE CONDITIONAL KULLBACK-LEIBLER DISTANCE COST FUNCTION x

11 Page APPENDIX B DERIVATION OF THE RECURSIVE WEIGHT UPDATE EQUA- TION APPENDIX C EFFICIENT TECHNIQUE TO COMPUTE THE ESTIMATE ER- ROR COVARIANCE MATRIX USING THE UTB ALGORITHM APPENDIX D PROVING CONVEXITY OF THE CONSTRAINT FUNCTION. 201 APPENDIX E QUADRATIC FORM FOR THE CONSTRAINT FUNCTION APPENDIX F DERIVATION OF THE MINIMUM TRACE INFORMATION MA- TRIX xi

12 LIST OF TABLES Table Page 1.1 List of acronyms used throughout the dissertation Adopted notation The CB algorithm The UTB algorithm Pseudo-code for breadth-first search Pseudo-code for greedy search Pseudo-code for the BFS-GS pruning algorithm Statistics for two pruning algorithms: BFS-GS and UCS Tracking error prediction using the PCRLB Number of times the belief was transferred between sensors s 5 and s 6 over 100 Monte Carlo runs for varying M and ρ T h Number of nodes explored per M-step search averaged over all Monte Carlo runs for varying M and ρ T h The MIP scheduling method for MinCost-I problem OA scheduling method for MinCost and MinError problems Scheduling statistics for MinCost-I problem for varying levels of ρ T h averaged over all Monte Carlo iterations Energy-consumption comparison statistics for greedy and MIP scheduling methods Scheduling statistics for MinCost-II problem for varying levels of ρ T h averaged over all Monte Carlo iterations xii

13 LIST OF FIGURES Figure Page 1.1. Block diagram illustrating tracking and sensor scheduling operations An illustration of a typical target tracking scenario using sensors in a twodimensional plane An illustration of the discrete approximation to a continuous density function using a set of particles Block diagram depicting sensor scheduling for target tracking Sets of particles used to compute the expected future cost for the UTB algorithm Sensor scheduling scenario in which a radar and an IR sensor, co-located at the origin, are used in target tracking: (a) the radar sensor provides good range and range rate measurements of the target, and (b) the IR sensor provides good bearing measurements of the target An example target trajectory generated during Monte Carlo simulations Comparison of the M = 1, 2 and 3 step sensor scheduling cases with the random and no-scheduling (NS) cases for the radar-ir scheduling problem. The sensors are scheduled using the CB algorithm A magnified version of Figure 3.5 to demonstrate the effect of scheduling noise with increasing M due to the unavailability of measurements xiii

14 Figure Page 3.7. Sensor selection for M = 1 and M = 2 step cases for a typical Monte Carlo run. Here, a bar of 1 indicates that the measurement at that time was taken by the radar sensor and a bar of 0 indicates that the measurement was taken by the IR sensor Tracking comparison of M = 1 step scheduling with the NS case for a typical Monte Carlo run Comparison of M = 1, 2 and 3 steps sensor scheduling results obtained with the UTB algorithm with the M = 1 step CB algorithm Tracking scenario: A sea target is tracked by a torpedo. At each time-epoch, the torpedo can change heading by one of nine possible values and then move b meters (a) Comparison of the RMSE of the target position estimate for M = 1, 2,, 5 OL scheduling using the UTB algorithm with the determinant cost. (b) Comparison of the sensor trajectories for the M = 2 and M = 4 OL scheduling using the UTB algorithm with the determinant cost Comparison of the RMSE of the target position estimate for M = 1, 2 and 3 with the KL distance cost, using the UTB algorithm. The M = 3 case with the determinant cost is included for comparison (a) RMSE comparison for OLF scheduling with M = 2, 3 and 4, using the UTB algorithm and the determinant cost. RMSE comparison of OLF and OL scheduling for: (b) M = 2, (c) M = 3, (d) M = xiv

15 Figure Page Comparison of the RMSE of the target position estimate using the UTB and CB algorithms for M = 3 and 4 with the determinant cost An illustrative decision tree with U = 4 sensing options and a time horizon of M = An illustration of node exploration for the breadth-first search in a tree An illustration of node exploration for the depth-first search in a tree An illustration of node exploration for the uniform-cost search in a tree Tree structure obtained from Figure 4.1 by performing a BFS at depth (i) Tree structure obtained from Figure 4.5 by performing GS for node β and pruning nodes e, f, and f, and (ii) tree structure obtained from (i) by performing a GS for node α and pruning nodes a,b, and c (a) Comparison of the suboptimal heuristic and optimal searches with the UTB algorithm and the determinant cost for M = 4 OL scheduling. Here, Sub[d 1, 4] is a heuristic suboptimal algorithm composed of a BFS up to depth d 1 and pure GS from depth d 1 to depth M = 4. (b) Comparison of RMSE of the target position estimate for M = 4 and varying values of suboptimal parameter ɛ using the UCS pruning algorithm (a) Percentage of nodes opened as a function of ɛ for M = 4 with the BFS-GS pruning algorithm. (b) Maximum number of nodes stored as a function of ɛ for M = 4 with the UCS pruning algorithm xv

16 Figure Page 5.1. An illustrative sensor layout of the x and y positions of Type A and Type B sensors. The sensors are labeled as s l, l = 1,, 6. The target trajectory and sensor-communication paths are shown by solid and dashed lines, respectively An illustrative decision tree with U = 4 sensing options and a time horizon of M = An example scenario depicting a Type A sensor (s 1 ) and two Type B sensors (s 2 and s 3 ) used in target tracking. The numbers on the straight dashed line denote measurement transfer cost between the sensors while the number on the curved dashed line denotes the belief transfer cost between the sensors Tree depicting the sensing options available to sensor s 2 at time-step k + 1 and some of the options for time-step k (a) RMSE curves for M = 2 and 3 with ρ T h = 17 m. Here, MC denotes Monte Carlo RMSE, PCRLB denotes PCRLB based predicted RMSE, PF denotes particle filter RMSE, and Th denotes the RMSE threshold. (b) Comparison of RAE for M = 1, 2, and 3 with ρ T h = 300 m (a) Comparison of RAE for M = 1, 2, and 3 with ρ T h = 600 m 2. (b) Comparison of RAE for M = 1, 2, and 3 with ρ T h = 1200 m Average number of nodes explored versus Monte Carlo iterations for M = 1, 2, and 3 with ρ T h = 300 m RAE versus ρ T h at times 14 s, 21 s, 28 s, and 35 s for M = 1, 2, and xvi

17 Figure Page 5.9. Number of nodes explored versus time k for a typical Monte Carlo run with M = 3 and ρ T h = 300 m (a) Geometric interpretation of linear support functions. (b) Geometric interpretation of outer approximation Illustrative convex surface of ˆρ(ã) (where ã = [ã 1 ã 2 ] T ) for a two-sensor problem (a) Layout of the sensor field under consideration. (b) Prototype sensor network for intrusion detection problem An illustration of two equivalent formulations for an integer programming problem MSE curves for (a) ρ T h = 1 m 2. (b) ρ T h = 2 m 2. Here, MC-Upd denotes the updated Monte Carlo MSE, Pred denotes the MSE predicted by the scheduler, and PF-Upd denotes the particle filter updated MSE Simulation statistics for ρ th = 1 m 2 : (a)(i) Statistics for number of sensors scheduled, realized over all problem instances. (a)(ii) Statistics for number of sensors activated, realized over all problem instances. (b)(i) Run time versus k averaged over Monte Carlo iterations. (b)(ii) Number of tree-nodes explored averaged over Monte Carlo iterations xvii

18 Figure Page 7.6. Comparison of greedy and MIP scheduling for ρ T h = 1 m 2 : (a)(i) Percentage suboptimality statistic realized over all problem instances. (a)(ii) Percentage suboptimality versus k averaged over all Monte Carlo iterations. (b) Running average energy comparison of MU heuristic-based and MIP scheduling methods MSE curves for (a) ρ T h = 3 m 2. (b) ρ T h = 4 m 2. Here, MC-Upd denotes the updated Monte Carlo MSE, Pred denotes the MSE predicted by the scheduler, and PF-Upd denotes the particle filter updated MSE Comparison of the number of tree-nodes explored for the OA and MIP scheduling methods averaged over all Monte Carlo iterations: (a) ν = 2, (b) ν = 2.5, (c) ν = 3, and (d) ν = Comparison of search time for OA and MIP scheduling methods for varying L averaged over all Monte Carlo iterations: (a) ν = 2, (b) ν = 2.5, (c) ν = 3, and (d) ν = Monte Carlo statistics for error-minimization scheduling problem for ξ T h = 5, 10, 15, 20, 25, 30, 35, 40, 45, and 50 mj (a) Average energy consumption versus k. (b) Average number of sensors scheduled versus k Monte Carlo statistics for error-minimization scheduling problem. (a) Position MSE versus k for varying values of ξ T h. (b)(i) Average number of nodes explored versus ξ T h. (b)(ii) Average scheduling time versus ξ T h Tracking scenario where a scheduler (airplane) schedules active sensors for target measurements xviii

19 Figure Page Monte Carlo statistics. (a) Average energy consumption versus k. (b) Average number of sensors activated versus k Monte Carlo statistics. (a) Position MSE versus k for varying values of ξ T h. (b)(i) Average number of nodes explored versus ξ T h. (b)(ii) Average scheduling time versus ξ T h F.1. Covariance ellipses for A 1 and E F.2. Information matrix ellipses for B 1 and J T r xix

20 CHAPTER 1 Introduction 1.1. Motivation In recent years, the advancement in sensor technology, coupled with embedded systems and wireless networking, has made it possible to deploy sensors for numerous applications including target tracking, environmental science, defense information, and security [1 3]. The deployed sensors can differ in the type of measurements (seismic, magnetic, temperature, acoustic, angle of arrival), in the cost of obtaining the measurements, in the sensing strategies (active, passive, agile), or in the quality of the information that they provide. Furthermore, compared to the centralized architectures used in multi-sensor applications over the past decade, today sensors (for example, motes) are equipped with tiny radio modules and micro-controllers that can be used to support decentralized and/or hierarchical architectures [4]. In such multi-sensor applications, one is often interested in maximizing the performance of the sensing application while operating within constraints on sensing resources. For example, in a target tracking scenario using active and passive sensors, one may be interested in maximizing the tracking accuracy while minimizing the number of active sensors since

21 2 active sensors can inform the target about their locations [5]. Similarly, for tracking a target in an energy-bandwidth constrained sensor network, one may be interested in maximizing the tracking accuracy subject to constraints on the network bandwidth or energy [1, 2]. Alternatively, in surveillance and monitoring problems, one may be interested in using the minimum number of sensors in a sensor network to maintain a desired network coverage at all times [6]. In addition to these scenarios, there could be a contention in the available sensing resources as different sensing tasks may require to be performed simultaneously; one would need to allocate the sensors in such a way that the overall loss of performance in the sensing tasks is minimized. In all of the above stated problems, a decision needs to be made on the sensor allocation at each time-step in order to execute a desired task while operating under system constraints. Sensor scheduling, a process to allocate sensing resources by optimizing a performance metric under a given set of constraints over a future time-horizon, is an effective method to address such problems. A well-planned sequence of sensor usages obtained from sensor scheduling can result in a large improvement in the system performance [5, 7 15]. Although this dissertation motivates the use of sensor scheduling from a target tracking perspective, solution techniques for the sensor scheduling problems can be applied to other problems. For example, the problem of robot path planning and the problem of optimal sensor placement in robotics and data fusion can be formulated using the sensor scheduling optimization framework [16 18]. Similarly, the problem of finding the optimum channel allocation among various components of a measurement vector to be transmitted over a time-shared communication channel of limited bandwidth can be addressed with the sensor scheduling framework [5]. Other areas where sensor scheduling framework can be applied

22 3 include admission control, power control in telecommunication networks, and scheduling problems in manufacturing systems and computer networks The Sensor Scheduling Problem and Solution Techniques In this dissertation, we investigate sensor scheduling algorithms for target tracking applications. We consider tracking scenarios where a discrete and finite set of sensing options is available at any given time. The discrete sensing options are usually available as a finite set of sensors (e.g., in a sensor network) or as multi-configurable sensors (e.g., a sensor whose motion can be configured in discrete steps in finite directions). We emphasize that we do not address scenarios that involve sensors with continuous-valued configurations as was done in [19]. Also, the target state and target measurements in our work are assumed to be continuous-valued, and they are sampled at discrete time indices. We use the terms scheduler and tracker to denote a device/processor that performs scheduling and tracking, respectively. Central to sensor scheduling is the definition of a scheduler cost (or an objective function) and a set of scheduling constraints. Examples of scheduler costs for tracking applications include the determinant and trace of the track estimate error covariance matrix [7, 10, 11, 14], gain in information about the target state [13, 20], entropy in the target state estimate [21], cost of sensing resources such as communication energy and/or communication bandwidth [8, 15], and risk to sensors [5]. Likewise, the scheduling problem may involve constraints on factors such as communication energy, desired level of tracking accuracy [8, 15, 22], or communication bandwidth [23], etc. As noise is present in both the target dynamics and sensor measurements, state-dependent sched-

23 4 Tracking and Scheduling performed by Processor Sensor Measurements Target Tracking Algorithm Target Estimate Scheduling Decisions Sensor Scheduling Algorithm Figure 1.1. Block diagram illustrating tracking and sensor scheduling operations. uler costs and constraints (e.g., tracking error) in this work are considered to be stochastic in nature while resource-dependent scheduler costs and constraints (e.g., communication energy and bandwidth) are considered to be deterministic in nature. Figure 1.1 depicts tracking and sensor scheduling components as used in this work. In this figure, both the tracker and scheduler are implemented at a common processor such as a computer workstation. At any given time, the tracker provides the current target state estimate that is processed by the scheduler to make future sensor allocation decisions. The sensors corresponding to these decisions are used to obtain target measurements, which are subsequently used by the tracker. The sensor scheduling problem can be formulated as a sequential stochastic optimization problem that involves optimization of an expected scheduler cost under a given set of constraints over time. We emphasize the term sequential to indicate that the scheduling problem is predictive in nature: the scheduler costs and constraints associated with

24 5 a sensing option at a future time-step can be evaluated only after the scheduler cost and constraints of all sensing options leading to the current sensing option have been evaluated. In principle, the optimization is performed by predicting an expected scheduler cost for all possible combinations of sensor uses over a time-horizon or prediction-horizon of length M. Solving the optimization problem yields the best sensor sequence of length M; the sensors of this sequence are then used to obtain target measurements. Dynamic programming provides a natural framework to formulate and obtain optimal solutions to sequential optimization problems [24, 25]. In the past, dynamic programming has been used to formulate sensor scheduling problems with an underlying assumption that the target state and measurements take values (probabilistically) from discrete finite sets [5, 26]. This assumption allows one to develop optimal closed-loop sensor scheduling policies that are functions of the target state at each stage of the scheduling problem. For general sensor scheduling problems that involve continuous-valued state-space and measurementspace, developing optimal closed-loop scheduling policies using the dynamic programming framework usually becomes intractable. Thus, one needs to resort to suboptimal open-loop or open-loop feedback scheduling algorithms that can yield scheduled sequences on-the-fly. Motivated by this, the first part of our work is directed towards developing open-loop and open-loop feedback sensor scheduling algorithms for target tracking applications. Sensor scheduling is denoted myopic if the sensors are scheduled by looking one step ahead in the future (i.e., greedy scheduling) and non-myopic if the sensors are scheduled by looking multiple steps ahead in the future. Although myopic scheduling is attractive due to its low computational complexity and ease of implementation [11, 27], the use of non-myopic scheduling often becomes imperative as it can perform significantly better than

25 6 myopic scheduling [7, 8, 15]. For example, in tracking scenarios that involve maneuvering a sensor platform to increase the observability of the target with finite movement choices, non-myopic sensor scheduling has been shown to perform significantly better than myopic sensor scheduling [7, 28]. This is because myopic maneuvering decisions do not take into account the long term effects of these decisions, which is critical because of the constrained motion of the sensor platform. Non-myopic sensor scheduling has also been recently shown to significantly improve the energy efficiency of an energy-constrained sensor network over myopic scheduling for target tracking applications [8, 15]. In many other tracking scenarios, the use of non-myopic sensor scheduling may simply be unavoidable. For example, in a tracking problem where one needs to dynamically deploy sensors according to the target s location, the deployment strategy must be planned wellahead of time as the sensors cannot be deployed instantaneously [14]. Similarly, many highend sensors (e.g., radar sensors) cannot be used instantaneously as one needs to account for the warm-up delay incurred during the transition between the sleep mode and the active mode of these sensors. The sensor scheduling problem can be posed as a search problem where the objective is to search for the best sensor sequence over a decision-space that contains all allowable sensor sequences of length M, where M is the length of the prediction-horizon. Note that M = 1 and M > 1 correspond to myopic and non-myopic sensor scheduling problems, respectively. As we are considering sensor scheduling problems with discrete options, the decision space is also discrete. The decision-space grows exponentially as the prediction horizon or the number of sensors increases making exhaustive search for the sensor scheduling computationally intractable. Furthermore, as the scheduler costs are predictive and stochastic, standard

26 7 graph-theory based search algorithms [29] or backward dynamic programming [24] are not directly applicable. Thus, we need to develop search algorithms that exploit the problem structure and perform intelligent enumeration on the search space. With this motivation, the second part of our work is directed towards developing efficient search algorithms that can be used to reduce the computational complexity in sensor scheduling problems Target Tracking and Filtering Target tracking is the estimation of target kinematics based on measurements obtained from sensors. It is widely formulated using state-space models where the unknown target parameters of interest, such as its position and velocity, are collectively termed as the target state [30]. In the state-space framework, the evolution of the target state in time is formulated using a process model while a measurement model is used to formulate the dependence of the sensor measurements on the target state. Given an a priori knowledge of the target state, the process and measurement models, and the sequence of target measurements up to the current time-instance, one can estimate the current target state [30, 31]. Filtering is the estimation of the current state of a dynamic system [30]. In this work, we are mainly interested in recursive filtering, i.e., in estimating the state recursively in time. For such problems, Bayesian filtering theory is the most widely accepted paradigm in the literature [32 34]. Depending on the nature of the dynamical models (e.g., linear/nonlinear), several optimal and suboptimal Bayesian filters can be used for recursive state estimation. For linear processes and measurement models, the Kalman filter is optimal in minimizing the mean squared-error of the target state estimate [35]. For moderately nonlinear models,

27 8 one can use an extended Kalman filter [30]. Recently it has been shown that for some nonlinear models, the unscented Kalman filter yields better tracking performance than the extended Kalman filter [36]. For highly nonlinear models in Gaussian or non-gaussian noise, sequential Monte Carlo filters (or particle filters) have recently emerged as robust filtering techniques [32, 33] Review of Prior Work in Sensor Scheduling Research in sensor scheduling for target tracking applications has received considerable research attention in the last few years. Several scheduling frameworks and optimization techniques have been developed that are applicable under a variety of scheduler costs and/or constraints. In this section, we provide a summary on some of the past and ongoing research efforts in the area of discrete sensor scheduling. Sensor scheduling algorithms using scheduler costs that depend on the predicted estimate error covariance matrix were developed in [7, 12, 14, 37]. To the best of our knowledge, the work in [37] signifies the earliest use of sensor scheduling for linear Gaussian dynamic models using covariance based costs. For such models, the predicted estimate error covariance matrix is independent of the future measurements; consequently, scheduling policies obtained with covariance dependent scheduler costs for linear Gaussian dynamic models are closed-loop and optimal. In [12], a scheduling algorithm was developed for linear Gaussian models in which the minimum number of sensors was selected to drive the error covariance matrix to a desired matrix. In [14], a posterior Cramér-Rao lower bound based framework was used to schedule the deployment of sonobuoy sensors to control the tracking accuracy

28 9 of a target within a specified level. In [5], stochastic dynamic programming was used to obtain closed-loop optimal solutions for scheduling problems in discrete probability space. Specifically, the sensor scheduling algorithms developed in [5] used an underlying assumption that both the sensor measurements and target state can take values (probabilistically) from discrete finite sets. In [26], the authors develop an approximate stochastic dynamic programming framework to classify a large number of stationary objects with a multi-modal sensor. In [15], the authors extend this framework to perform non-myopic sensor scheduling for tracking in an energy-constrained sensor network. Sensor scheduling algorithms using information-theoretic based scheduler costs for target tracking were developed in [13, 15, 20, 21, 27, 38, 39]. In [13], the scheduling was myopic and the objective was to maximize the gain in Rényi information for binary-valued measurements. In [38], an information driven sensor query method was proposed to query sensors in a sensor network to obtain the maximum information about a target traveling through the network. In [20, 39], the sensors were scheduled by maximizing the mutual information between the target state and the measurement sequence. In [21], sensors were selected myopically such that the fusion of new measurements provided by these sensors resulted in the maximum reduction of entropy of the current target state. In [15], the authors used the entropy of the target state as a performance metric for sensor scheduling in an energy-constrained sensor network. In [27], sequential Monte Carlo techniques were used to choose sensors by optimizing the expected Kullback-Leibler information gain in the target state over all possible combination of sensors, one step in the future. In the past, several studies have addressed the problem of reducing the computational

29 10 complexity in non-myopic sensor scheduling. For example, in [20], efficient non-myopic sensor scheduling was performed by pruning unwanted sensor sequences using forward dynamic programming. In [40], suboptimal sliding window and threshold methods were proposed to increase search efficiency in non-myopic sensor scheduling. In [14], greedy search, adapted from the simulated annealing approach was used to perform non-myopic scheduling. In [5], Lovejoy s algorithm was used to perform suboptimal non-myopic sensor scheduling. In [15], the authors combined greedy sensor selection, greedy tree pruning, and Lagrangian relaxation techniques to develop a tractable scheduling solution for the non-myopic scheduling problem. Recently, the work in [24, 41] proposed a rollout algorithm for stochastic scheduling and stochastic shortest path problems [24, 41]. This algorithm makes use of a Q-value function to approximate future scheduler costs based on the current state information, and it has recently been applied in some non-myopic sensor scheduling problems [42, 43] Contributions of this Work In this work, we present three main contributions: (a) development of open-loop and open-loop feedback non-myopic sensor scheduling algorithms for nonlinear target tracking problems, (b) implementation of sensor scheduling algorithms in different target tracking settings to illustrate the effectiveness of the developed algorithms, and (c) development of efficient search algorithms to facilitate the implementation of the proposed sensor scheduling algorithms. Our publications related to this work can be noted through the references [7 11, 22, 23, 44, 45].

30 Development of Sensor Scheduling Algorithms Our work is motivated by the fact that closed-loop optimal solutions for sensor scheduling problems are obtainable only for linear and Gaussian dynamical models. However, in most tracking problems, the linear and Gaussian assumptions are not valid. For example, when a sensor measures acoustic energy emitted by the target, the measurements depend nonlinearly on the target state. In such cases, one may develop simulation based suboptimal sensor scheduling algorithms. As a consequence, we develop Monte Carlo based sensor scheduling algorithms that can be applied to nonlinear target tracking problems. We begin by developing a non-myopic sensor scheduling algorithm that uses covariance based costs; we call this the covariance-based sensor scheduling algorithm. In this algorithm, we use a particle filter in conjunction with the linear approximation and information formulation of the filter step of the Kalman filter [30] to predict the estimate error covariance matrix for future time-steps; we compute the scheduler costs based on the predicted estimate error covariance matrix. The linearized Kalman filter framework involves the computation of Jacobian matrices to obtain the information contribution of each sensing option. Under this framework, the contribution of a sensing option is expressed in terms of a positive semidefinite information matrix. In some tracking scenarios, one may not be able to obtain analytical expressions of the Jacobian matrices. For example, in a tracking scenario where the measurements are binary valued (detect or no-detect) and depend probabilistically on the state (e.g., through a probability of detection), it is not possible to obtain an expression of the Jacobian matrix. Additionally, in some tracking scenarios, one may be interested in using information-theoretic

31 12 based scheduler costs that are not functions of the estimate error covariance matrix. To facilitate scheduling in such cases, we developed an unscented transform based non-myopic sensor scheduling algorithm; we call this the unscented transform-based algorithm. The unscented transform-based algorithm works by predicting discrete approximations of state and measurement densities and by computing scheduler costs based on these densities Application Scenarios for Sensor Scheduling Algorithms We apply the covariance-based algorithm to a tracking scenario where we want to schedule between radar and infrared sensors. Our results demonstrate that scheduling using the covariance-based algorithm results in much better performance than no-scheduling or random scheduling [11]. We applied the unscented transform-based algorithm to the radar- IR scheduling problem in [10] and it was found that it performs as well as the CB algorithm. The advantage of the unscented transform-based algorithm however lies in the fact that it can be used with arbitrary state-dependent scheduler costs. It was also found that for the radar-infrared tracking scenario, myopic and non-myopic scheduling led to similar tracking performances. Next, we considered a tracking scenario in which non-myopic sensor scheduling performed significantly better than myopic sensor scheduling. Specifically, we considered a surface-ship tracking scenario in which an acoustic homing torpedo used electro-acoustic transducers and passive beamforming to obtain bearing measurements from the target and estimate the target s position and velocity [46]. The torpedo maneuvered relative to the target (from a discrete set of allowable manoeuvering options) to improve the target observability, and the objective of the sensor scheduling problem was to obtain a sequence

32 13 of torpedo headings that minimize the predicted squared-error in the target position estimate over a future time-horizon. We applied both the covariance-based and the unscented transform-based algorithms for this tracking scenario and we concluded that non-myopic sensor scheduling performs significantly better than myopic scheduling [7] Computationally Efficient Methods for Sensor Scheduling Noting that the use of non-myopic sensor scheduling in the preceding tracking scenario is limited due to the computational complexity and memory requirements of the scheduling algorithms, we proposed two branch-and-bound based pruning algorithms to facilitate the implementation of non-myopic sensor scheduling in the torpedo tracking scenario. The first algorithm combines breadth-first search and greedy search with the branch-and-bound algorithm and is more efficient in memory consumption. The second algorithm is a uniformcost branch-and-bound algorithm and is more efficient in search time. We also investigated the use of sensor scheduling for general problems that involve a trade-off between sensor usage costs and tracking performance. In particular, we considered the application of sensor scheduling to tracking a target in a sensor network with the objective of minimizing the network energy consumption while maintaining a desired tracking accuracy of the target moving through the network. For this scenario, we first considered a prototype small-scale sensor network of bearing-only sensors. Our results demonstrated that non-myopic sensor scheduling can result in significant energy savings over myopic sensor scheduling for the prototype sensor network [8]. However, we also noted that as the number of sensors in the network increases, the scheduling procedure in [8] becomes very computational cumbersome.

33 14 We addressed the use of sensor scheduling in a large scale sensor network that comprised of several hundreds to thousands of sensors, and where at any given time, one may need to schedule sensors. For this scenario, we considered two myopic sensor scheduling problems: minimizing the tracking error subject to constraints on network energy consumption, and minimizing the network energy costs subject to constraints on tracking error. We exploit the structure in the scheduling problems and reformulate them as binary or (0-1) mixed integer programming problems (i.e., problems involving linear objective and constraint functions and a set of binary and real-valued variables) and 0-1 convex programming problems (i.e., problems with binary valued variables and convex objective and convex constraint functions). We used a linear programming relaxation based branch-and-bound technique [47, 48] to obtain optimal solutions to the 0-1 mixed integer programming problem. The 0-1 convex programming algorithm was solved using the outer approximation algorithm in [49] that involves solving a series of 0-1 mixed integer programming problems. Our simulation results demonstrated that the reformulated 0-1 mixed integer programming and 0-1 convex programming problems allow us to solve sensor scheduling problems of up to sensors in the order of seconds Outline of This Dissertation This dissertation is organized as follows. Chapter 2 provides an overview on target tracking. In Chapter 3, we develop the optimization framework for non-myopic sensor scheduling and propose the covariance-based and the unscented transform-based sensor

34 15 scheduling algorithms. We also demonstrate the use of the covariance-based and the unscented transform-based algorithms in two tracking scenarios, namely radar-infrared tracking scenario and mobile bearing-only tracking scenario. In Chapter 4, we provide an overview of tree search techniques relevant to the non-myopic sensor scheduling problem and propose two branch-and-based based tree pruning algorithms that can be used to reduce the complexity of non-myopic sensor scheduling problems. We illustrate the use of these tree pruning algorithms to assist the non-myopic sensor scheduling in the mobile bearing only tracking scenario. In Chapter 5, we provide simulations and results for the application of non-myopic sensor scheduling on a small-scale prototype sensor network of bearing-only sensors. In Chapter 6, we provide an introduction to binary (0-1) programming and in Chapter 7, we develop mixed integer programming and outer approximation sensor scheduling methods using 0-1 programming framework. We apply our proposed sensor scheduling methods to tracking problems that involve a trade-off between sensor costs and tracking performance. We conclude with our extensions and suggested future work in Chapter 8. The acronyms used throughout this dissertation are summarized in Table 1.1. The vairable notation is summarized in Table 1.2.

35 Adopted Notation Table 1.1. List of acronyms used throughout the dissertation. Acronym B&B BFS CB CP CRLB DFS EKF KL LP MIP MMSE MSE NLP NS OA OL OLF Interpretation Branch-and-bound Breadth-first search Covariance-based Convex programming Cramér-Rao lower bound Depth-first search Extended Kalman filter Kullback Leibler Linear programming Mixed integer programming Minimum mean square error Mean square error Nonlinear programming No-scheduling Outer approximation Open-loop Open-loop feedback Continued on next page

36 17 Acronym PCRLB QP RD RMSE UCS UT UTB Interpretation Posterior Cramér-Rao lower bound Quadratic programming Rényi divergence Root mean square error Uniform-cost search Unscented transform Unscented transform-based Table 1.2. Adopted notation Notation a l k Definition Sensor control or sensor configuration associated with the lth sensor a k = [a 1 k a L k ]T A k = a1:k = {a 1 a k } a k+1:k+m = Ak+m f( ) f k ( ) F g r ( ) Sensor-usage vector or sensing option at time k Sequence of control variables from time 1 to k Sensor sequence from time k + 1 to k + m Scheduler cost function used for sensor scheduling Nonlinear function at time k for process model State transition matrix rth constraint function in sensor scheduling, r = 1,, M Continued on next page

37 18 Notation h k ( ) k L M M Definition Nonlinear measurement function at time k Discrete time index Number of sensors available at time k Scheduling time horizon Number of constraint functions in sensor scheduling formulation n x Dimension of x k n w Dimension of w k ( n x = n w ) N Total number of particles in a particle filter P k k ˆP k k Estimate error covariance matrix at time k based on Z k Approximate estimate error covariance matrix at time k based on Z k ˆP k+m k Approximate estimate error covariance matrix at time k + m based on Z k P k+m k+r Approximate estimate error covariance matrix at time k + m based on Z k and the predicted measurements z k+1:k+r ˇP k+m k+r Approximate estimate error covariance matrix at time k + m based on Z k and the use of sensor sequence A k+r, r = 1,, m Q k R k Process noise covariance matrix at time k Measurement noise covariance matrix at time k Continued on next page

38 19 Notation t U v k w k wk i x k x i k Definition Time difference between measurements Number of sensing options available one-step into the future Measurement noise sample at time k Process noise sample at time k Weight of the ith particle Target state vector at time k ith particle of a particle filter ˆx k k State estimate at time k based on Z k ˆx k+m k x k+m k+r State estimate at time k + m based on Z k State estimate at time k + m based on Z k and the predicted measurements z k+1:k+r, r = 1,, m X k = x0:k z k Z k State sequence from time 0 to k: x 0, x 1,, x k Measurement obtained by using a k at time k Measurement obtained by using a k at time k Z k = Z1:k z k+1:k+m = Zk+m Measurement sequence from time 1 to k: z 1, z 2,, z k Measurement sequence from time k + 1 to k + m

39 CHAPTER 2 Review of Target Tracking Target tracking is the estimation of unknown target kinematics based on measurements from sensors. The unknown target kinematics of interest are usually the position, velocity, and acceleration of the target in an appropriate coordinate system. The sensor measurements are usually perturbed by noise and contain information about the target kinematics. For example, while target measurements like range, acoustic, and bearing provide information on the target position, measurements like range rate and bearing rate provide information about the target velocity. Figure 2.1 depicts a typical target tracking scenario in a two-dimensional plane in which the measurements from sensors are used by a tracker to track a target. In the past, various techniques were used for target tracking. For example, matchedfield processing has been extensively used for tracking in underwater signal processing applications [50, 51]. Other target tracking techniques include neural networks [52, 53], fuzzy logic [54], and high speed vision tracking [55]. However, by far, Bayesian theory remains the most widely accepted approach to target tracking, and it constitutes the main crux of the work in this report. In this chapter, we provide a review of target tracking using the Bayesian theoretic

40 21 Figure 2.1. An illustration of a typical target tracking scenario using sensors in a twodimensional plane. approach. We are mainly concerned with tracking in discrete time and processing sensor measurements sequentially rather than in batch. We begin by describing the state-space formulation to model the target tracking problem. We then discuss the Bayesian-theoretic approach to target tracking. Lastly, we review some of the widely used filtering techniques for recursive state estimation in target tracking Target Tracking Formulations We use a state-space approach to formulate the tracking problem. In this framework, the unknown kinematics of the target (that are usually unobservable) are called the state. The evolution of the target state x k with time k is described by a system or process model which is described as [32] x k+1 = f(x k ) + w k+1 (2.1)

41 22 where x k is a vector of size n x, f : R n x R n x is a known nonlinear function 1 of the state, and w k 1 is an additive, independent and identically distributed process-noise vector of dimension n x with covariance matrix Q k 1. The lth sensor, l = 1,, L, at time k is located at (x l k, yl k ) with measurement covariance R l k, where L is the total number of sensors available at time k. Associated with each sensor, is a sensor control or configuration variable a l k, l = 1,, L, that takes values from a discrete finite set. For example, the control variable a l k may be used to specify discrete movement decisions of a mobile sensor, switching decisions (on/off) in a sensor network, or configuration choices for a multi-configurable sensor. The control variable for a sensing [ option is defined as a k = a 1 k a L ] T k, where T denotes matrix transpose, and we define the sequence of control variables from time 1 to k as A k = a1:k = {a 1,, a k }. The target measurement vector z l k of size n l is obtained by using the lth sensor, and it is related to x k by the measurement model [32] z l k = hl k (x k, a l k ) + vl k (2.2) where h l k is a known nonlinear measurement function2 for the lth sensor and v l k is an additive independent and identically distributed measurement noise sequence vector of size n l with covariance matrix R l k. The process and measurement models in (2.1) and (2.2) are collectively called the dynamical model. The models defined in (2.1) and (2.2) represent the general nonlinear tracking scenario in additive noise. Simplifications of these models are often possible by replacing the nonlinear functions f and h l k by known matrices, F and Hl k, respectively, and by replacing w k and 1 Note that f( ) can also be time varying in the general case. 2 Note that for ease of notation, we exclude the explicit dependence of z l k on a l k; the explicit dependence is indicated as necessary.

42 23 v l k by independently and identically distributed sequences of Gaussian random variables. In fact, throughout the rest of this work, we use a constant velocity linear model for tracking. In this case, we denote the target state at time k by x k = [x k ẋ k y k ẏ k ] T, where x k and y k are the target positions, and ẋ k and ẏ k are its corresponding velocities, and we denote the constant velocity model as x k+1 = Fx k + w k+1. (2.3) Here, F is time invariant and is called the state transition matrix; it is given as F = 1 t t , (2.4) where t is the time difference between the measurements, and w k is a zero-mean, white Gaussian sequence with a known covariance matrix Q. This covariance is obtained by converting a continuous time stochastic target model into an equivalent discrete time model [56]: t 3 t t 2 t 0 0 Q = q 2 t 3 t t t 2 where q denotes the intensity of the process noise., (2.5) We denote the measurement vector Z k obtained by using s k at time k as

43 24 Z k = z 1 k z 2 k. z L k = h 1 k (x k, a 1 k ) h 2 k (x k, a 2 k ). h L k (x k, a L k ) + v 1 k v 2 k. v L k = h k (x k, a k ) + v k, (2.6) where z l k, hl k, and vl k are the measurement, measurement function, and noise sequence, respectively, corresponding to the lth sensor, l = 1,, L. The vectors h k and v k are obtained by concatenating the component measurement functions and noise sequences, respectively; v k is assumed to be a zero-mean, white Gaussian noise sequence with a known covariance matrix R k. We denote the sequence of state values from time 0 to k by X k = x0:k, and the sequence of measurement values from time 1 to k by Z k = Z1:k Bayesian Approach to Target Tracking From a Bayesian perspective, the tracking problem is to recursively calculate some degree of belief in the state x k, given a sequence of measurements Z k [32]. This belief is usually expressed in the form of the posterior probability density function p(x k Z k, A k ). It is assumed that the initial probability density function of the state, p(x 0 ), is available. Then, in principle, p(x k Z k, A k ) may be obtained recursively (under a Bayesian paradigm) in two stages: the prediction stage and the update stage. Prediction Stage : Suppose that the probability density function p(x k 1 Z k 1, A k 1 ) at time k 1 is known. The prediction stage involves using the system model to obtain the probability density function of the state at time k based on all the measurements until time

44 25 k 1 using the Chapman-Kolmogorov equation 3 [32]. Specifically, p(x k Z k 1, A k 1 ) = = = p(x k, x k 1 Z k 1, A k 1 )dx k 1 x k 1 p(x k x k 1, Z k 1, A k 1 )p(x k 1 Z k 1, A k 1 )dx k 1 x k 1 p(x k x k 1 ) p(x k 1 Z k 1, A k 1 ) dx k 1, (2.7) x k 1 where p(x k Z k 1, A k 1 ) is also called the prior probability density function. In the above equation, we used the fact that p(x k x k 1, Z k 1, A k 1 ) = p(x k x k 1 ) since (2.1) describes a Markov process of the first order [32]. The probabilistic model of the state evolution p(x k x k 1 ) may be derived from (2.1) and the known statistics of w k 1. Update Stage : At time-step k, we obtain a measurement Z k of the state. This measurement is then used to update the prior probability density function using the Baye s formulation [32] p(x k Z k, A k ) = p(z k x k, A k )p(x k Z k 1, A k 1 ), (2.8) p(z k Z k 1, A k ) where p(z k x k, A k ) is called the likelihood density function. This function can be calculated using the measurement model in (2.2) and the known statistics of v k, and p(z k Z k 1, A k ) is a normalizing constant that can be obtained as p(z k Z k 1, A k ) = p(z k x k, A k ) p(x k z 1:k 1 )dx k. (2.9) x k Using p(x k Z k, A k ), one can obtain the minimum mean square error (MMSE) unbiased estimate of the target state: ˆx k k = E [x k Z k, A k ] = x k p(x k Z k, A k )dx k, (2.10) x k 3 Throughout this work, the limits of integration are from to, unless otherwise stated.

45 26 where E denotes the expectation operator. Note that ˆx k k is also the conditional mean of p(x k Z k, A k ). Since ˆx k k is unbiased (i.e., E[ˆx k k ] = E[x k ]), the estimate error covariance matrix can be expressed as [30] P k k = E [ (x k ˆx k k )(x k ˆx k k ) T ] = Z k x k (x k ˆx k k )(x k ˆx k k ) T p(x k, Z k A k ) dx k dz k, (2.11) where (x k ˆx k k ) is called the estimation error and p(x k, Z k A k ) is the joint distribution of x k and Z k given A k. Equation (2.8) states that, under the Bayesian framework, we can obtain the posterior probability density function by first generating the prior density and then updating it with the likelihood function (obtained when the new measurement becomes available). However, one must note that (2.8) is more of a conceptual formulation as the posterior probability density function can rarely be obtained analytically. Exceptions include the case of tracking with linear Gaussian dynamical models for which the Kalman filter yields the posterior probability density function as a Gaussian density in closed form. For nonlinear models in additive Gaussian noise, one can use an extended Kalman filter to approximate the posterior probability density function with a Gaussian density [30]. For nonlinear dynamic models with Gaussian/non-Gaussian process and measurement noise sequences, one can use a particle filter to obtain discrete approximations to the posterior probability density functions [32]. In the next section, we provide detailed descriptions of three recursive filtering techniques, Kalman filter, extended Kalman filter, and particle filter that are used to obtain the posterior probability density function depending on the type of dynamical model used

46 27 for tracking Recursive Filters for Target Tracking Kalman Filter The Kalman filter was proposed in the early 1960 s by Rudolph E. Kalman in [35]. The Kalman filter assumes linear Gaussian model as described in (2.3) and the measurement model expressed as Z k = H k x k + v k. (2.12) where w k and v k are zero mean, white Gaussian noise sequences with covariances Q k and R k. If w k 1, v k, and x 0 are assumed mutually independent, the Kalman filter provides the following closed form recursive formulations for the Bayesian tracking problem [32] p(x k 1 Z k 1, A k 1 ) = N (ˆx k 1 k 1, P k 1 k 1 ) p(x k Z k 1, A k 1 ) = N (ˆx k k 1, P k k 1 ) p(x k Z k, A k ) = N (ˆx k k, P k k ) where N (ˆx, P) denotes a Gaussian density with mean ˆx and covariance matrix P and, ˆx k k 1 = Fˆx k 1 k 1 P k k 1 = FP k 1 k 1 F T + Q ˆx k k = m k k 1 + K k (Z k H kˆx k k 1 ) P k k = P k k 1 K k H k P k k 1.

47 28 Here, the superscript T denotes the matrix transpose, and K k = P k k 1 H T k ( Hk P k k 1 H T k + R ) 1 k (2.13) is called the gain of the Kalman filter. For linear Gaussian dynamical models, the Kalman filter is the optimal minimum mean square error (MMSE) recursive state estimator [35]. An important feature of the Kalman filter is that the posterior probability density function of the state is obtained in closed form as a Gaussian density; the mean and covariance of this density correspond to the MMSE state estimate and the covariance matrix of the estimate error, respectively. The filter is very powerful in several aspects: it supports estimations of past, present, and even future states, and it can do so even when the precise model of the system is unknown Extended Kalman Filter When the dynamic models are nonlinear in Gaussian noise, one can use an extended Kalman filter (EKF). The EKF is obtained by linearizing the nonlinear models using a first order Taylor series expansion 4 and using the approximated models with the recursive equations of the Kalman filter [30]. The density functions in the EKF are approximated by a Gaussian density function: p(x k 1 Z k 1, A k 1 ) N (ˆx k 1 k 1, P k 1 k 1 ) p(x k Z k 1, A k 1 ) N (ˆx k k 1, P k k 1 ) p(x k Z k, A k ) N (ˆx k k, P k k ) 4 Although second order approximations are possible, they are rarely used as they are computationally intensive.

48 29 where ˆx k k 1 = f(ˆx k 1 k 1 ) P k k 1 = ˆF k P k 1 k 1 ˆFT k + Q ˆx k k = ˆx k k 1 + K k (Z k h k (ˆx k k 1 )) P k k = P k k 1 K k Ĥ k P k k 1. Here, ˆF k and Ĥk are local linearizations of f and h k, i.e., with the EKF gain given as ˆF k = f(x) (2.14) x x=ˆxk 1 k 1 Ĥ k = h k(x), (2.15) x x=ˆxk k 1 K k = P k k 1 Ĥ T k (Ĥk P k k 1 Ĥ T k + R k ) 1. (2.16) Note that the EKF, although widely used, does not yield an MMSE estimator as the state estimate in this framework is not the conditional mean of the posterior probability density function. Furthermore, the use of the first order series expansion in EKF has the potential disadvantage of introducing un-modeled errors. Nevertheless, if these errors are not large, the EKF performs reasonably well [30] Particle Filter A particle filter is a recursive Bayesian filter in which the posterior probability density function at time k is approximated by N samples (called particles) x i k and associated

49 30 Figure 2.2. An illustration of the discrete approximation to a continuous density function using a set of particles. importance weights w i k, i = 1,, N [32]. Specifically, p(x k Z k, A k ) is approximated as p(x k Z k, A k ) N i=1 wk i δ(x k x i k ). (2.17) The particle filter is asymptotically optimal since the discrete approximation in the right hand side of (2.17) asymptotically approaches p(x k Z k, A k ) as N approaches infinity [33]. To illustrate this concept, consider the continuous-valued, one dimensional probability density function p(x) in Figure 2.2. A discrete approximation to p(x) can be obtained using a set of samples or particles described by spheres in Figure 2.2. Each sphere represents a particle with the particle weight corresponding to the size of the sphere; particles whose locations correspond to the peak of p(x) have larger weights, while those near the tail of p(x) have smaller weights. As the number of particles increases, the discrete approximation to p(x) improves. The particle filter yields discrete approximations of the posterior probability

50 31 density function in a way analogous to that described in Figure 2.2. It must be noted that the computation of the MMSE estimate in (2.10) involves computing an integral, which may not be possible in many cases due to the unavailability of an analytical expression of p(x k Z k, A k ) or the intractability of the integral itself. However, using the particles and the weights, one can easily approximate the expected value of any function of the state; in particular, using (2.17), the MMSE estimate and the covariance of the state are given as ˆx k k N wk i xi k (2.18) i=1 ˆP k k N i=1 w i k ( x i k ˆx k k ) ( x i k ˆx k k ) T. (2.19) It is important to note that ˆP k k in (2.19) is actually a conditional covariance matrix. We can obtain P k k by computing an expectation of ˆP k k with respect to the measurement density function p(z k A k ). At each time k, the samples x i k are drawn from a proposal distribution [32], which is chosen as the prior density p(x k x k 1 ) in this work. Upon obtaining a measurement Z k, the weights are updated recursively using the weight update equation: wk+1 i = p(z k x i wi k, a k) k ( N. (2.20) j=1 wj k p(z k x j k k)), a Based on the preceding discussion, the particle filter algorithm also known as the sequential importance sampling (SIS) algorithm [32] is summarized next: 1. Initialize: Set time-step k = 0; intialize the particle filter by generating a set of N independent and identically distributed samples from the given p(x 0 ) that provides a

51 32 rough estimate of the state 5 ; set all the initial weights to w i 0 = 1/N. 2. Iterate: For k = 1 to end time Draw x i k p(x k x i k 1 ) Assign weights to the particles using (2.20) End For Often during the generation and assignment of weights in a particle filter, the weights of the good samples keep increasing and those of the poor samples keep decreasing with time. After a few iterations, all but one particle will have negligible weight; this effect is called degeneracy. In such a scenario, we use the resampling algorithm described in [32]. The idea of resampling is to retain multiple versions of the good samples while discarding the poor samples. Resampling prevents estimation with impoverished samples; a criterion for deciding when to resample is to evaluate the variance of the weights at each time-step, and then to compare it to a set threshold [32] Short Overview on Sensors For the sake of completeness, we present in the following, a short overview on some widely used sensors from an operational (system) point of view Radar Sensors A radar is an active sensor since it initiates the energy which the target reflects. This target energy is subsequently collected by the radar s aperture [57]. The active nature of 5 If there is no information available on the prior probability density function p(x 0), we may start with uniformly generated independent and identically distributed samples from the region of possible states.

52 33 the radar is the primary factor that makes it an effective sensor. Furthermore, as the source of the energy is the radar itself, the nature of the returned target signal is in some sense under control of the radar designer. Thus, possibilities exist for dynamically adapting target energy to changing environments or current needs of the tracking system [57]. At the same time, a disadvantage of active sensing is that if the sensor location is revealed to the target, then the target can take evasive or hostile actions. Modern radars are capable of measuring target angle (azimuth and elevation), range, and range rate. The radar scans its antenna while transmitting and subsequently receiving pulses of energy. It processes the energy reflected from objects to detect their presence and various kinematic parameters such as range and range-rate. The radar operates by transmitting a pulse of radio frequency energy with pulse width τ. This pulse travels to the target, is reflected by the target, and returns to the radar t seconds later. The radar s receiver detects the presence of the returning pulse and measures the round-trip travel time t and hence the range; the mathematical details can be found in [57]. The radar measures angle by noting the azimuth and elevation angles at which the aperture is pointed during the time the returning pulse is detected. The azimuth and elevation along with the measured range provide the full 3-dimensional relative position of the target with respect to the radar [57]. Modern Doppler radars have the ability to measure the range rate of a target by coherently integrating many pulses of returning energy against the known transmitted signal and hence measuring the Doppler shift that the target imposes on the reflected pulse [57].

53 Infrared Sensors Infrared (IR) sensors are passive sensors, i.e., they obtain target measurements in a reactive mode. The advantage of passive sensing is that the target cannot perceive sensor location and thus cannot take evasive actions. Other advantages of an IR sensor are that it provides excellent measurement accuracy and resolution capability for closely spaced targets. An IR sensor measures the energy produced by the target itself (e.g., hot exhaust gasses from an aircraft) or reflected by the target from other sources (e.g., reflected sky and earth shine) [57]. Thus, the IR sensor does not have any control on the nature of the energy to be detected. The primary information that an IR sensor provides is the azimuth and elevation of the detected targets. The angles of the detection are determined based on the state of the scanning optics at the time of the detection and, in the case of a detector array, the detectors in the array which record the target s energy. Due to the relatively short wavelength of IR radiation, very fine angular resolution accuracy is possible with apertures that can be easily incorporated in many platform designs [57] Acoustic Sensors Acoustic sensors measure the intensity of pressure waves that produce sound radiated by the targets. Sources of this radiation include the vibration of machinery, propellers, turbines, and hydrodynamic noise generated by turbulent flow [57]. Typically, the intensity of pressure waves decreases in inverse proportion to the distance from the acoustic source.

54 35 Passive acoustic sensors are used in underwater signal processing (e.g. hydrophones) where low frequency sound carries over long distances. Microphone based acoustic sensors are also supported by miniature sensors such as the Berkeley motes [4]. Other commercially available sensors that can be used in tracking applications include accelerometers, magnetometers, and light sensors.

55 CHAPTER 3 Non-myopic Sensor Scheduling In target tracking applications, one often has several sensing options in the form of multiple sensors or multi-configurable sensors to obtain measurements. An important consideration is the natural trade-off between the cost of sensing resources and the quality of tracking performance. As more sensors are allocated, the tracking quality tends to improve, albeit at an increased sensor-usage cost. In such cases, one needs to decide which of the allowed sensing options to choose for measurements at each time instant. The problem of optimally allocating sensing options for future time-steps is called the sensor scheduling problem. Non-myopic sensor scheduling is the process to schedule sensors by looking ahead multiple steps in the future. Non-myopic scheduling is important when it performs considerably better than myopic scheduling. This can occur since non-myopic scheduling incorporates extra information in its scheduling decisions due to its larger prediction horizon, which in some scenarios, becomes critical to the tracking performance. The non-myopic sensor scheduling problem can be formulated and solved optimally (in principle) using the dynamic programming framework [24]. However, for continuous-valued state-space and measurement variables, the use of dynamic programming often becomes

56 37 intractable. In such cases, one needs to resort to suboptimal scheduling techniques that can provide scheduled sequences on-the-fly. Two scheduling frameworks that can be used in such scenarios are the open-loop (OL) scheduling and the open-loop feedback (OLF) scheduling [24, 58 61]. In the OL scheduling, we perform scheduling only after all the multi-step decisions of the previous scheduling solution have been exhausted [24]. In the OLF scheduling, only the first scheduling decision is executed, and non-myopic scheduling is repeated at each time-step. In this chapter, we propose two non-myopic sensor scheduling algorithms that can be used with nonlinear Gaussian dynamic models. We first formulate the optimization framework for the non-myopic sensor scheduling problem. Thereafter, we provide a description on some of the scheduler costs that can be used in sensor scheduling. Lastly, we present our two proposed non-myopic sensor scheduling algorithms. The first algorithm is called the covariance-based (CB) algorithm, and it can be used with covariance based scheduler costs. The second algorithm is called the unscented transform-based (UTB) algorithm, and it can be used with arbitrary state-dependent scheduler costs such as information theoretic costs or covariance based costs. We conclude this chapter by providing an example tracking scenario that involves scheduling between a radar and an infrared (IR) sensor Optimization Framework Let M denote the length of the prediction horizon of sensor scheduling, i.e., the number of steps we look ahead to perform scheduling. The objective of non-myopic sensor scheduling at any given time k is to obtain the best sequence of sensing options over the next M

57 38 time-steps in order to minimize a scheduler cost. Figure 3.1 depicts a block diagram that integrates an M-step sensor scheduling algorithm with a target tracking algorithm. We denote a sensor sequence 1 at any given time k by an M-tuple: A k+m = [a k+1 a k+2 a k+m ] T, where a k+m is the sensor-usage vector or sensing option at time k + m (m steps in the future). Assuming that the number of allowable sensing options at each time is U, a total of U M distinct sensor sequences of length M are possible. Figure 3.1. Block diagram depicting sensor scheduling for target tracking. We denote the scheduler cost at time k + m by f(a k+m ). Examples of widely used scheduler costs include the tracking accuracy of a target (e.g., root mean squared-error of the target state estimate), information gained or loss of uncertainty in the target state, cost of sensor usage, and the cost of resource consumption such as bandwidth and energy. The scheduler cost for a given sensor sequence A k+m is then defined as M f(a k+m ) = f(a k+m ). (3.1) m=1 1 More precisely, sensor configuration sequence.