Markov Decision Process-Based Resource and Information Management for Sensor Networks

In this chapter, we consider the problem of managing a network of sensors with particular application to multisensor multitarget tracking. We study the problem of decision based control of a network of sensors carrying out surveillance over a region that includes a number of moving targets. The objective is to maximize the information obtained and to track as many targets as possible with the maximum possible accuracy. Uncertainty in the information obtained by each sensor regarding the location of the targets is addressed in the problem formulation. The chapter presents a number of solutions for centralized and decentralized tracking involving sensor management and distributed information flow control. We consider a distributed data fusion system consisting of sensors that are decentralized, heterogenous, and potentially unreliable. The objective function for sensor management is based on the Posterior Cramer-Rao lower bound and constitutes the basis of a reward structure for Markov decision processes that are used, together with decentralized lookup substrate, to control the data fusion process. In distributed sensor network fusion, we analyze three distributed data fusion algorithms: associated measurement fusion, tracklet fusion and track-to-track fusion. The chapter also provides a detailed analysis of communication and computational load in distributed tracking algorithms. In centralized sensor network fusion, we introduce a multi-level hierarchy of MDPs to control each of the sensors in the network. Simulation results are presented on a representative multitarget tracking problem using a network of sensors showing a significant improvement in performance compared to the existing algorithm.

[1]  Yaakov Bar-Shalom,et al.  Multitarget/Multisensor Tracking: Applications and Advances -- Volume III , 2000 .

[2]  Krishna R. Pattipati,et al.  Efficient multisensor fusion using multidimensional data association , 2001 .

[3]  F. Dambreville,et al.  Detection with spatial and temporal optimization of search efforts involving multiple modes and multiple resources management , 2000, Proceedings of the Third International Conference on Information Fusion.

[4]  Chee-Yee Chong,et al.  Joint Probabilistic Data Association in Distributed Sensor Networks , 1985, 1985 American Control Conference.

[5]  Thia Kirubarajan,et al.  Large-Scale Optimal Sensor Array Management for Multitarget Tracking , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[6]  Thia Kirubarajan,et al.  Collaborative distributed data fusion architecture using multi-level Markov decision processes , 2007, 2007 10th International Conference on Information Fusion.

[7]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[8]  Timothy W. McLain,et al.  A decomposition strategy for optimal coordination of unmanned air vehicles , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[9]  B. Leung,et al.  Multibit Sigma - Delta A/D converter incorporating a novel class of dynamic element matching techniques , 1992 .

[10]  Claudia V. Goldman,et al.  Efficient Control of Information Flow for Distributed Multisensor Fusion Using Markov Decision Processes , 2006, 2006 9th International Conference on Information Fusion.

[11]  X. Rong Li.  Optimal linear estimation fusion-part VII: dynamic systems , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[12]  Oliver E. Drummond,et al.  Hybrid sensor fusion algorithm architecture and tracklets , 1997, Optics & Photonics.

[13]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[14]  Y. Bar-Shalom,et al.  Unbiased converted measurements for tracking , 1998 .

[15]  Thiagalingam Kirubarajan,et al.  Optimal cooperative placement of UAVs for ground target tracking with Doppler radar , 2004, SPIE Defense + Commercial Sensing.

[16]  Alan N. Steinberg.  Open Networks: Generalized Multi-Sensor Characterization , 2006, 2006 9th International Conference on Information Fusion.

[17]  Thiagalingam Kirubarajan,et al.  Performance limits of track-to-track fusion versus centralized estimation: theory and application [sensor fusion] , 2003 .

[18]  Anil Kumar,et al.  Image Segmentation Based on Optimal Layering for Precision Tracking , 1993, Partitioning Data Sets.

[19]  Kuo-Chu Chang,et al.  Architectures and algorithms for track association and fusion , 2000 .

[20]  Dimitri P. Bertsekas,et al.  Linear network optimization - algorithms and codes , 1991 .

[21]  Pramod K. Varshney,et al.  Decision fusion in a wireless sensor network with a large number of sensors , 2004 .

[22]  Petter Ögren,et al.  Cooperative control of mobile sensor networks:Adaptive gradient climbing in a distributed environment , 2004, IEEE Transactions on Automatic Control.

[23]  Kathryn B. Laskey,et al.  Credibility Models for Multi-Source Fusion , 2006, 2006 9th International Conference on Information Fusion.

[24]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[25]  W. Dale Blair,et al.  Enhanced electronically scanned array resource management through multisensor integration , 1997, Optics & Photonics.

[26]  Bin Yu,et al.  Learning the Quality of Sensor Data in Distributed Decision Fusion , 2006, 2006 9th International Conference on Information Fusion.

[27]  Huimin Chen,et al.  Multisensor track-to-track association for tracks with dependent errors , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[28]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[29]  Claudia V. Goldman,et al.  Decentralized Control of Cooperative Systems: Categorization and Complexity Analysis , 2004, J. Artif. Intell. Res..

[30]  John N. Tsitsiklis,et al.  The Complexity of Markov Decision Processes , 1987, Math. Oper. Res..

[31]  Y. Bar-Shalom,et al.  Multiassignment for tracking a large number of overlapping objects [and application to fibroblast cells] , 2001 .

[32]  Y. Bar-Shalom,et al.  A generalized S-D assignment algorithm for multisensor-multitarget state estimation , 1997, IEEE Transactions on Aerospace and Electronic Systems.

[33]  H. C. Chan,et al.  An integrated maritime surveillance system based on high-frequency surface-wave radars. 1. Theoretical background and numerical simulations , 2001 .

[34]  Y. Bar-Shalom,et al.  Multisensor resource deployment using posterior Cramer-Rao bounds , 2004, IEEE Transactions on Aerospace and Electronic Systems.

[35]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[36]  Krishna R. Pattipati,et al.  Ground target tracking with variable structure IMM estimator , 2000, IEEE Trans. Aerosp. Electron. Syst..

[37]  M. Farooq,et al.  Estimation and Decision Fusion: A Survey , 2006 .

[38]  Thiagalingam Kirubarajan,et al.  Precision large scale air traffic surveillance using IMM/assignment estimators , 1999 .

[39]  E. Fernandez-Gaucherand,et al.  Cooperative control for multiple autonomous UAV's searching for targets , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[40]  Rudy J. van de Plassche,et al.  Dynamic element matching for high-accuracy monolithic D/A converters , 1976 .

[41]  H. V. Trees.  Detection, Estimation, And Modulation Theory , 2001 .

[42]  T. Kirubarajan,et al.  Track segment association, fine-step IMM and initialization with Doppler for improved track performance , 2004, IEEE Transactions on Aerospace and Electronic Systems.

[43]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM 2001.

[44]  Oliver E. Drummond,et al.  Performance metrics for multiple-sensor multiple-target tracking , 2000, SPIE Defense + Commercial Sensing.

[45]  Carlos H. Muravchik,et al.  Posterior Cramer-Rao bounds for discrete-time nonlinear filtering , 1998, IEEE Trans. Signal Process..

[46]  D. Akselrod,et al.  A mixed signal enhanced WTA tracking system via 2-D dynamic element matching , 2002, 2002 IEEE International Symposium on Circuits and Systems. Proceedings (Cat. No.02CH37353).

[47]  Robert Morris,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM 2001.

[48]  Alan S. Willsky,et al.  Distributed data association for multi-target tracking in sensor networks , 2005 .

[49]  Yakov Bar-Shalom,et al.  Multitarget-Multisensor Tracking: Principles and Techniques , 1995 .

[50]  Krishna R. Pattipati,et al.  Ground-target tracking with topography-based variable-structure IMM estimator , 1998, Defense, Security, and Sensing.

[51]  L. R. Carley,et al.  A noise-shaping coder topology for 15+ bit converters , 1989 .

[52]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[53]  Y. Bar-Shalom,et al.  On optimal track-to-track fusion , 1997, IEEE Transactions on Aerospace and Electronic Systems.

[54]  Thiagalingam Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation , 2001 .

[55]  Samuel S. Blackman,et al.  Design and Analysis of Modern Tracking Systems , 1999 .

[56]  G. Temes.  Delta-sigma data converters , 1994 .

[57]  Timothy W. McLain,et al.  Coordinated target assignment and intercept for unmanned air vehicles , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[58]  Leslie Pack Kaelbling,et al.  On the Complexity of Solving Markov Decision Problems , 1995, UAI.

[59]  Krishna R. Pattipati,et al.  Multiassignment for tracking a large number of overlapping objects , 1997, Optics & Photonics.

[60]  Thia Kirubarajan,et al.  Hierarchical markov decision processes based distributed data fusion and collaborative sensor management for multitarget multisensor tracking applications , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[61]  Nicholas R. Jennings,et al.  Computational Mechanism Design for Information Fusion within Sensor Networks , 2006, 2006 9th International Conference on Information Fusion.

[62]  R. Schreier,et al.  Delta-sigma data converters : theory, design, and simulation , 1997 .

[63]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[64]  Thiagalingam Kirubarajan,et al.  Large-scale optimal sensor array management for target tracking , 2004, SPIE Optics + Photonics.

[65]  T. Kirubarajan,et al.  Performance Evaluation of Multi-platform Distributed Data Fusion Methods for Multi-target Tracking , 2007, 2007 IEEE Aerospace Conference.

[66]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[67]  T. Kirubarajan,et al.  Collaborative sensor management for multitarget tracking using decentralized Markov decision processes , 2006, SPIE Defense + Commercial Sensing.

[68]  Alfonso Farina,et al.  Algorithms for the selection of the active sensors in distributed tracking: comparison between Frisbee and GNS methods , 2006, 2006 9th International Conference on Information Fusion.

[69]  L. R. Carley,et al.  A 16-bit 4'th order noise-shaping D/A converter , 1988, Proceedings of the IEEE 1988 Custom Integrated Circuits Conference.

[70]  Claudia V. Goldman,et al.  The complexity of multiagent systems: the price of silence , 2003, AAMAS '03.

[71]  C.-Y. Chong,et al.  Distributed fusion and communication management for target identification , 2005, 2005 7th International Conference on Information Fusion.

[72]  Feng Zhao,et al.  Distributed tracking in wireless ad hoc sensor networks , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[73]  Yaakov Bar-Shalom,et al.  Multitarget-Multisensor Tracking: Applications and Advances , 1992 .

[74]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[75]  H. Durrant-Whyte,et al.  Management and control in decentralised networks , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[76]  Y. Bar-Shalom,et al.  Autonomous Ground Target Tracking by Multiple Cooperative UAVs , 2005, 2005 IEEE Aerospace Conference.

[77]  T. Kirubarajan,et al.  IMMPDA solution to benchmark for radar resource allocation and tracking in the presence of ECM , 1997, 1997 European Control Conference (ECC).

[78]  Hugh F. Durrant-Whyte,et al.  Dynamic allocation and control of coordinated UAVs to engage multiple targets in a time-optimal manner , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[79]  K. Benameur.  Optimal receiver location for emitter tracking , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[80]  N. Gordon,et al.  Cramer-Rao bounds for non-linear filtering with measurement origin uncertainty , 2002, Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997).

[81]  Abhijit Sinha,et al.  Collaborative distributed sensor management for multitarget tracking using hierarchical Markov decision processes , 2007, SPIE Optical Engineering + Applications.

[82]  Abhijit Sinha,et al.  PCRLB-based multisensor array management for multitarget tracking , 2007 .

[83]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[84]  Aubrey B. Poore,et al.  A New Lagrangian Relaxation Based Algorithm for a Class of Multidimensional Assignment Problems , 1997, Comput. Optim. Appl..

[85]  Lang Hong,et al.  Static/dynamic distributed interacting multiple model fusion algorithms for multiplatform multisensor tracking , 1997 .

[86]  Oliver E. Drummond,et al.  Tracklets and a hybrid fusion with process noise , 1997, Optics & Photonics.

[87]  R. Bellman.  Dynamic programming. , 1957, Science.

[88]  Leon Sterling,et al.  Distributed information fusion agents , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.