Collaborative sensor management for multitarget tracking using decentralized Markov decision processes

In this paper, we consider the problem of collaborative sensor management with particular application to using unmanned aerial vehicles (UAVs) for multitarget tracking. We study the problem of decentralized cooperative control of a group of UAVs carrying out surveillance over a region that includes a number of moving targets. The objective is to maximize the information obtained and to track as many targets as possible with the maximum possible accuracy. Uncertainty in the information obtained by each UAV regarding the location of the ground targets are addressed in the problem formulation. In order to handle these issues, the problem is presented as a decentralized operation of a group of decision-makers lacking full observability of the global state of the system. Recent advances in solving special classes of decentralized Markov Decision Processes (Dec-MDPs) are incorporated into the solution. In these classes of Dec-MDPs, the agents' transitions and observations are independent. Also, the collaborating agents share common goals or objectives. Given the Dec-MDP model, a local policy of actions for a single agent (UAV) is given by a mapping from a current partial view of a global state observed by an agent to actions. The available probability model regarding possible and confirmed locations of the targets is considered in the computations of the UAVs' policies. Simulation results are presented on a representative multisensor-multitarget tracking problem.

[1]  R. Bellman.  Dynamic programming. , 1957, Science.

[2]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[3]  Claudia V. Goldman,et al.  Decentralized Control of Cooperative Systems: Categorization and Complexity Analysis , 2004, J. Artif. Intell. Res..

[4]  Timothy W. McLain,et al.  A decomposition strategy for optimal coordination of unmanned air vehicles , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[5]  Dimitri P. Bertsekas,et al.  Linear network optimization - algorithms and codes , 1991 .

[6]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[7]  H. Durrant-Whyte,et al.  Management and control in decentralised networks , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[8]  Y. Bar-Shalom,et al.  Autonomous Ground Target Tracking by Multiple Cooperative UAVs , 2005, 2005 IEEE Aerospace Conference.

[9]  Claudia V. Goldman,et al.  The complexity of multiagent systems: the price of silence , 2003, AAMAS '03.

[10]  Timothy W. McLain,et al.  Coordinated target assignment and intercept for unmanned air vehicles , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[11]  Leslie Pack Kaelbling,et al.  On the Complexity of Solving Markov Decision Problems , 1995, UAI.

[12]  T. Kirubarajan,et al.  Track segment association, fine-step IMM and initialization with Doppler for improved track performance , 2004, IEEE Transactions on Aerospace and Electronic Systems.

[13]  E. Fernandez-Gaucherand,et al.  Cooperative control for multiple autonomous UAV's searching for targets , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[14]  Petter Ögren,et al.  Cooperative control of mobile sensor networks:Adaptive gradient climbing in a distributed environment , 2004, IEEE Transactions on Automatic Control.

[15]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[16]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[17]  Ronald J. Williams,et al.  Tight Performance Bounds on Greedy Policies Based on Imperfect Value Functions , 1993 .

[18]  Hugh F. Durrant-Whyte,et al.  Dynamic allocation and control of coordinated UAVs to engage multiple targets in a time-optimal manner , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[19]  Thiagalingam Kirubarajan,et al.  Optimal cooperative placement of UAVs for ground target tracking with Doppler radar , 2004, SPIE Defense + Commercial Sensing.

[20]  Yakov Bar-Shalom,et al.  Multitarget-Multisensor Tracking: Principles and Techniques , 1995 .