Collision Avoidance for Unmanned Aircraft using Markov Decision Processes

we investigate the automatic generation of collision avoidance algorithms given models of aircraft dynamics, sensor performance, and intruder behavior. By formulating the problem of collision avoidance as a Markov Decision Process (MDP) for sensors that provide precise localization of the intruder aircraft, or a Partially Observable Markov Decision Process (POMDP) for sensors that have positional uncertainty or limited eld-of-view constraints, generic MDP/POMDP solvers can be used to generate avoidance strategies that optimize a cost function that balances ight-plan deviation with collision. Experimental results demonstrate the suitability of such an approach using four dierent sensor modalities and a parametric aircraft performance model.

[1]  Mykel J. Kochenderfer,et al.  A decision-theoretic approach to developing robust collision avoidance logic , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[2]  Joelle Pineau,et al.  Spoken Dialogue Management Using Probabilistic Reasoning , 2000, ACL.

[3]  Reid G. Simmons,et al.  Probabilistic Robot Navigation in Partially Observable Environments , 1995, IJCAI.

[4]  Leslie Pack Kaelbling,et al.  Model-Based Optimization of Airborne Collision Avoidance Logic , 2010 .

[5]  Marie-Odile Cordier,et al.  Supply Restoration in Power Distribution Systems: A Case Study in Integrating Model-Based Diagnosis and Repair Planning , 1996, UAI.

[6]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[7]  Mykel J. Kochenderfer,et al.  Hazard Alerting Based on Probabilistic Models , 2011 .

[8]  Jeff G. Schneider,et al.  Autonomous helicopter control using reinforcement learning policy search methods , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[9]  Amir Averbuch,et al.  Interacting Multiple Model Methods in Target Tracking: A Survey , 1988 .

[10]  Joelle Pineau,et al.  High-level robot behavior control using POMDPs , 2002 .

[11]  James K. Kuchar.  A unified methodology for the evaluation of hazard alerting systems , 1995 .

[12]  Leslie Pack Kaelbling,et al.  Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[13]  Joelle Pineau,et al.  Point-based value iteration: An anytime algorithm for POMDPs , 2003, IJCAI.

[14]  A. Cassandra.  A Survey of POMDP Applications , 2003 .

[15]  David Hsu,et al.  SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces , 2008, Robotics: Science and Systems.

[16]  Alex Pentland,et al.  Active gesture recognition using partially observable Markov decision processes , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[17]  Barbara J. Chludzinzki.  Lincoln Laboratory Evaluation of TCAS II Logic Version 7. Volume 1 , 1999 .

[18]  Nan Rong,et al.  A point-based POMDP planner for target tracking , 2008, 2008 IEEE International Conference on Robotics and Automation.

[19]  Craig Boutilier,et al.  A POMDP formulation of preference elicitation problems , 2002, AAAI/IAAI.

[20]  Milos Hauskrecht,et al.  Planning and control in stochastic domains with imperfect information , 1997 .

[21]  David Hsu,et al.  Planning under Uncertainty for Robotic Tasks with Mixed Observability , 2010, Int. J. Robotics Res..

[22]  Eric Horvitz,et al.  Deeplistener: harnessing expected utility to guide clarification dialog in spoken language systems , 2000, INTERSPEECH.

[23]  Mykel J. Kochenderfer,et al.  Accounting for State Uncertainty in Collision Avoidance , 2011 .

[24]  J E Lebron,et al.  System Safety Study of Minimum TCAS II (Traffic Alert and Collision Avoidance System) for Instrument Weather Conditions. , 1983 .

[25]  James K. Kuchar,et al.  Evaluation of Collision Avoidance Maneuvers for Parallel Approach , 1999 .

[26]  B J Chludzinski.  LINCOLN LABORATORY EVALUATION OF TCAS II LOGIC VERSION 7 APPENDICES.. , 1997 .

[27]  Hisashi Tanizaki,et al.  Nonlinear Filters: Estimation and Applications , 1993 .

[28]  James K. Kuchar.  Methodology for Alerting-System Performance Evaluation , 1996 .

[29]  S. Shankar Sastry,et al.  Autonomous Helicopter Flight via Reinforcement Learning , 2003, NIPS.

[30]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[31]  Mykel J. Kochenderfer,et al.  Correlated Encounter Model for Cooperative Aircraft in the National Airspace System Version 1.0 , 2008 .

[32]  Thomas B Billingsley.  Safety Analysis of TCAS on Global Hawk Using Airspace Encounter Models , 2006 .

[33]  Reid G. Simmons,et al.  Point-Based POMDP Algorithms: Improved Analysis and Implementation , 2005, UAI.

[34]  Nan Rong,et al.  What makes some POMDP problems easy to approximate? , 2007, NIPS.

[35]  Francisco J. Vico,et al.  Residual Q-Learning Applied to Visual Attention , 1996, ICML.

[36]  B. Abdul-Baki,et al.  Independent validation and verification of the TCAS II collision avoidance subsystem , 1999, Gateway to the New Millennium. 18th Digital Avionics Systems Conference. Proceedings (Cat. No.99CH37033).

[37]  Mykel J. Kochenderfer,et al.  Airspace Encounter Models for Estimating Collision Risk , 2010 .

[38]  Michael P. Wellman,et al.  Planning and Control , 1991 .