Robust UAV path planning using POMDP with limited FOV sensor

Significant development in path planning algorithms for unmanned aerial vehicles (UAVs) has been performed using numerous different methods. One such method, Partially Observable Markov Decision Processes (POMDP), has been used effectively for tracking fixed and moving targets. One limitation of those efforts has been the assumption that the UAVs could always see the targets, with a few unique exceptions, e.g., building obscuration. In reality, there will be times when a vehicle will not be able to observe a target due to constraints such as turn requirements or tracking multiple targets that are not within a single field of view (FOV). The POMDP formulation proposed in this paper is robust enough to handle those missed observations. Monte Carlo runs of 1000 iterations per configuration are run to provide statistical confidence in the performance of the algorithm. UAV altitude and sensor configuration are varied to show robustness across multiple configurations. A sensor with a limited FOV is assumed and changes in fixed look angle are evaluated. Changes in altitude provide results equivalent to changes in sensor window or focal length. Results show that the POMDP algorithm is capable of tracking single and multiple moving targets successfully with limited FOV sensors across a range of conditions.

[1]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC , 2005, Eur. J. Control.

[2]  Edwin K. P. Chong,et al.  A POMDP Framework for Coordinated Guidance of Autonomous UAVs for Multitarget Tracking , 2009, EURASIP J. Adv. Signal Process..

[3]  A. Hero,et al.  Efficient methods of non-myopic sensor management for multitarget tracking , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[4]  Alfred O. Hero,et al.  Partially Observable Markov Decision Process Approximations for Adaptive Sensing , 2009, Discret. Event Dyn. Syst..

[5]  D.A. Castanon,et al.  Rollout Algorithms for Stochastic Scheduling Problems , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[6]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Vol. II , 1976 .

[8]  Edwin K. P. Chong,et al.  UAVGuidance Algorithms via Partially Observable Markov Decision Processes , 2014 .

[9]  Anthony A. Maciejewski,et al.  Multiple-Scenario Unmanned Aerial System Control: A Systems Engineering Approach and Review of Existing Control Methods , 2016 .

[10]  Gang Wu,et al.  Burst-level congestion control using hindsight optimization , 2002, IEEE Trans. Autom. Control..

[11]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[12]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[13]  Robert Givan,et al.  A framework for simulation-based network control via hindsight optimization , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[14]  Anna Freud,et al.  Design And Analysis Of Modern Tracking Systems , 2016 .

[15]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[16]  Edwin K. P. Chong,et al.  UAV Path Planning in a Dynamic Environment via Partially Observable Markov Decision Process , 2013, IEEE Transactions on Aerospace and Electronic Systems.

[17]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[18]  Y. Bar-Shalom.  Tracking and data association , 1988 .