Multi-Target Detection and Recognition by UAVs Using Online POMDPs

This paper tackles high-level decision-making techniques for robotic missions, which involve both active sensing and symbolic goal reaching, under uncertain probabilistic environments and strong time constraints. Our case study is a POMDP model of an online multi-target detection and recognition mission by an autonomous UAV. The POMDP model of the multi-target detection and recognition problem is generated online from a list of areas of interest, which are automatically extracted at the beginning of the flight from a coarse-grained high altitude observation of the scene. The POMDP observation model relies on a statistical abstraction of an image processing algorithm's output used to detect targets. As the POMDP problem cannot be known and thus optimized before the beginning of the flight, our main contribution is an "optimize-while-execute" algorithmic framework: it drives a POMDP sub-planner to optimize and execute the POMDP policy in parallel under action duration constraints. We present new results from real outdoor flights and SAIL simulations, which highlight both the benefits of using POMDPs in multi-target detection and recognition missions, and of our "optimize-while-execute" paradigm.

[1]  David Hsu,et al.  SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces , 2008, Robotics: Science and Systems.

[2]  Brahim Chaib-draa,et al.  AEMS: An Anytime Online Search Algorithm for Approximate Policy Refinement in Large POMDPs , 2007, IJCAI.

[3]  Mykel J. Kochenderfer,et al.  Unmanned Aircraft Collision Avoidance using Continuous-State POMDPs , 2011, Robotics: Science and Systems.

[4]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[5]  Kendall E. Nygard,et al.  POMDP Planning for High Level UAV Decisions: Search vs. Strike , 2003, CAINE.

[6]  Jianjiang Lu,et al.  A Framework for Moving Target Detection, Recognition and Tracking in UAV Videos , 2012 .

[7]  Jaime Valls Miró,et al.  A POMDP framework for modelling human interaction with assistive robots , 2011, 2011 IEEE International Conference on Robotics and Automation.

[8]  Joelle Pineau,et al.  Point-based value iteration: An anytime algorithm for POMDPs , 2003, IJCAI.

[9]  M. Spaan.  Cooperative Active Perception using POMDPs , 2008 .

[10]  Charles Lesire,et al.  A generic framework for anytime execution-driven planning in robotics , 2011, 2011 IEEE International Conference on Robotics and Automation.

[11]  Seth Hutchinson,et al.  Minimum uncertainty robot navigation using information-guided POMDP planning , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Edwin K. P. Chong,et al.  A POMDP Framework for Coordinated Guidance of Autonomous UAVs for Multitarget Tracking , 2009, EURASIP J. Adv. Signal Process..

[13]  Bertrand Le Saux,et al.  Robust vehicle categorization from aerial images by 3D-template matching and multiple classifier system , 2011, 2011 7th International Symposium on Image and Signal Processing and Analysis (ISPA).

[14]  Jérôme Lang,et al.  Purely Epistemic Markov Decision Processes , 2007, AAAI.

[15]  Reid G. Simmons,et al.  Point-Based POMDP Algorithms: Improved Analysis and Implementation , 2005, UAI.

[16]  Blai Bonet,et al.  Solving POMDPs: RTDP-Bel vs. Point-based Algorithms , 2009, IJCAI.

[17]  Thilo Grundmann,et al.  Probabilistic action planning for active scene modeling in continuous high-dimensional domains , 2009, 2009 IEEE International Conference on Robotics and Automation.