An information theoretic approach for generating an aircraft avoidance Markov Decision Process

Developing a collision avoidance system that can meet safety standards required of commercial aviation is challenging. A dynamic programming approach to collision avoidance has been developed to optimize and generate logics that are robust to the complex dynamics of the national airspace. The current approach represents the aircraft avoidance problem as Markov Decision Processes and independently optimizes a horizontal and vertical maneuver avoidance logics. This is a result of the current memory requirements for each logic, simply combining the logics will result in a significantly larger representation. The “curse of dimensionality” makes it computationally inefficient and infeasible to optimize this larger representation. However, existing and future collision avoidance systems have mostly defined the decision process by hand. In response, a simulation-based framework was built to better understand how each potential state quantifies the aircraft avoidance problem with regards to safety and operational components. The framework leverages recent advances in signals processing and database, while enabling the highest fidelity analysis of Monte Carlo aircraft encounter simulations to date. This framework enabled the calculation of how well each state of the decision process quantifies the collision risk and the associated memory requirements. Using this analysis, a collision avoidance logic that leverages both horizontal and vertical actions was built and optimized using this simulationbased approach.

[1]  Larry M. Kinnan,et al.  Use of multicore processors in avionics systems and its potential impact on implementation and certification , 2009, 2009 IEEE/AIAA 28th Digital Avionics Systems Conference.

[2]  John S. Lai A hidden Markov model and relative entropy rate approach to vision-based dim target detection for UAV sense-and-avoid , 2010 .

[3]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[4]  Jonathan P. How,et al.  Optimized Airborne Collision Avoidance , 2015 .

[5]  N. Metropolis,et al.  The Monte Carlo method. , 1949 .

[6]  Mykel J. Kochenderfer,et al.  Optimizing the Next Generation Collision Avoidance System for Safe, Suitable, and Acceptable Operational Performance , 2013 .

[7]  Wilson C. Hsieh,et al.  Bigtable: A Distributed Storage System for Structured Data , 2006, TOCS.

[8]  Benjamin Van Roy,et al.  A Cost-Shaping Linear Program for Average-Cost Approximate Dynamic Programming with Performance Guarantees , 2006, Math. Oper. Res..

[9]  Amy R. Pritchett,et al.  Pilot’s Information Use During TCAS Events, and Relationship to Compliance to TCAS Resolution Advisories , 2012 .

[10]  Kevin W Williams An Assessment of Pilot Control Interfaces for Unmanned Aircraft , 2007 .

[11]  J E Lebron,et al.  System Safety Study of Minimum TCAS II (Traffic Alert and Collision Avoidance System) for Instrument Weather Conditions. , 1983 .

[12]  Robert Kohl Algorithms And Data Structures The Basic Toolbox , 2016 .

[13]  Mykel J. Kochenderfer,et al.  Efficiently Estimating Ambient Near Mid-Air Collision Risk for Unmanned Aircraft* , 2010 .

[14]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[15]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[16]  Jens Rasmussen,et al.  Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  Rick Cattell,et al.  Scalable SQL and NoSQL data stores , 2011, SGMD.

[18]  T.D. Hall,et al.  SAFETY ANALYSIS METHODOLOGY FOR UNMANNED AERIAL VEHICLE ( UAV ) COLLISION AVOIDANCE SYSTEMS , 2005 .

[19]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[20]  Mykel J. Kochenderfer,et al.  Collision avoidance system optimization with probabilistic pilot response models , 2011, Proceedings of the 2011 American Control Conference.

[21]  James Doebbler,et al.  Real-Time Path Planning and Terrain Obstacle Avoidance for General Aviation Aircraft , 2005 .

[22]  Elliott Irving Organick A Fortran IV Primer , 1966 .

[23]  Fred Glover,et al.  Practical introduction to simulation optimization , 2003, Proceedings of the 2003 Winter Simulation Conference, 2003..

[24]  Mykel J. Kochenderfer,et al.  Airspace Encounter Models for Conventional and Unconventional Aircraft , 2009 .

[25]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[26]  G. Box,et al.  On the Experimental Attainment of Optimum Conditions , 1951 .

[27]  Carlos Parra,et al.  Designing public safety mobile applications for disconnected, interrupted, and low bandwidth communication environments , 2013, 2013 IEEE International Conference on Technologies for Homeland Security (HST).

[28]  Daniela Pucci de Farias,et al.  Choosing the cost vector of the linear programming approach to approximate dynamic programming , 2008, 2008 47th IEEE Conference on Decision and Control.

[29]  Michael Stonebraker,et al.  SQL databases v. NoSQL databases , 2010, CACM.

[30]  Reid G. Simmons,et al.  Heuristic anytime approaches to stochastic decision processes , 2006, J. Heuristics.

[31]  James K. Kuchar,et al.  The Traffic Alert and Collision Avoidance System , 2007 .

[32]  Alex Fung,et al.  Multi-Sensor Data Integration for Autonomous Sense and Avoid , 2011 .

[33]  D. W. Burgess,et al.  TCAS III Bearing Error Evaluation. , 1995 .

[34]  Mathématiques,et al.  Big-O Notation , 2010, Definitions.

[35]  David R. Maroney,et al.  UNMANNED AIRCRAFT COLLISION AVOIDANCE – TECHNOLOGY ASSESSMENT AND EVALUATION METHODS , 2007 .

[36]  Selim Temizer Unmanned Aircraft Collision Avoidance Using Partially Observable Markov Decision Processes , 2009 .

[37]  N. A. Spencer,et al.  Development and operation of the Traffic Alert and Collision Avoidance System (TCAS) , 1989, Proc. IEEE.

[38]  S. Fang,et al.  Entropy Optimization and Mathematical Programming , 1997 .

[39]  Leslie Pack Kaelbling,et al.  Model-Based Optimization of Airborne Collision Avoidance Logic , 2010 .

[40]  F. R. Taylor,et al.  Helicopter User Survey: Traffic Alert Collision Avoidance System (TCAS) , 1985 .

[41]  Mykel J. Kochenderfer,et al.  Electro-Optical System Analysis for Sense and Avoid , 2008 .

[42]  Peter Brooker,et al.  Introducing Unmanned Aircraft Systems into a High Reliability ATC System , 2013, Journal of Navigation.

[43]  Wright-Patterson Afb,et al.  Multiple Intruder Autonomous Avoidance Flight Test , 2011 .

[44]  Mykel J. Kochenderfer,et al.  Robust Airborne Collision Avoidance through Dynamic Programming , 2011 .

[45]  L P Espindle,et al.  Safety Analysis of Upgrading to TCAS Version 7.1 Using the 2008 U.S. Correlated Encounter Model , 2009 .

[46]  Kyle A. Smith,et al.  Collision avoidance system optimization for closely spaced parallel operations through surrogate modeling , 2013 .

[47]  Mykel J. Kochenderfer,et al.  Uncorrelated Encounter Model of the National Airspace System, Version 1.0 , 2008 .

[48]  Xuejun Zhang,et al.  A conflict avoidance approach based on memetic algorithm under 4D-Trajectory operation concept , 2013, 2013 IEEE/AIAA 32nd Digital Avionics Systems Conference (DASC).

[49]  A. Sinsky,et al.  Enhanced TCAS II tracking accuracy , 1984 .

[50]  Mykel J. Kochenderfer,et al.  A New Approach for Designing Safer Collision Avoidance Systems , 2012 .

[51]  Mykel J. Kochenderfer,et al.  Airspace Encounter Models for Estimating Collision Risk , 2010 .

[52]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[53]  Andrew W. Moore,et al.  Variable Resolution Discretization in Optimal Control , 2002, Machine Learning.

[54]  Don-lin Mon,et al.  Evaluating weapon system using fuzzy analytic hierarchy process based on entropy weight , 1994, Proceedings of 1995 IEEE International Conference on Fuzzy Systems..

[55]  A. J. Weinert,et al.  Providing communication capabilities during disaster response: Airborne remote communication (ARC) platform , 2012, 2012 IEEE Conference on Technologies for Homeland Security (HST).

[56]  Jeremy Kepner,et al.  Driving big data with big compute , 2012, 2012 IEEE Conference on High Performance Extreme Computing.

[57]  Mykel J. Kochenderfer,et al.  On Estimating Mid-Air Collision Risk , 2010 .

[58]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[59]  M. Laguna,et al.  Neural network prediction in a system for optimizing simulations , 2002 .

[60]  M. Edwards Encounter Models for the Littoral Regions of the National Airspace System , 2010 .

[61]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[62]  Sergio Pissanetzky,et al.  Sparse Matrix Technology , 1984 .

[63]  Raphael Yuster,et al.  Fast sparse matrix multiplication , 2004, TALG.

[64]  Mykel J. Kochenderfer,et al.  Correlated Encounter Model for Cooperative Aircraft in the National Airspace System Version 1.0 , 2008 .

[65]  A L McFarland,et al.  INITIAL COLLISION AVOIDANCE ALGORITHMS FOR THE BEACON-BASED COLLISION AVOIDANCE SYSTEM , 1977 .

[66]  Jeremy Kepner,et al.  'pMATLAB Parallel MATLAB Library' , 2007, Int. J. High Perform. Comput. Appl..

[67]  D. W. Burgess,et al.  TCAS: maneuvering aircraft in the horizontal plane , 1995 .

[68]  Omid Shakernia,et al.  Passive Ranging for UAV Sense and Avoid Applications , 2005 .

[69]  R. D. Grappel ASR-9 processor augmentation card (9-PAC) phase II scan-scan correlator algorithms , 2001 .

[70]  D. Rajan Probability, Random Variables, and Stochastic Processes , 2017 .

[71]  W. Harman TCAS: A system for preventing midair collisions , 1989 .

[72]  D Griffith,et al.  Coordinating General Aviation Maneuvers with TCAS Resolution Advisories , 2001 .

[73]  Jeremy Kepner Parallel MATLAB - for Multicore and Multinode Computers , 2009, Software, environments, tools.

[74]  Joelle Pineau,et al.  Online Planning Algorithms for POMDPs , 2008, J. Artif. Intell. Res..

[75]  R. Abeyratne Rules of the Air , 2012 .

[76]  Matthew W. M. Edwards,et al.  Establishing a Risk-Based Separation Standard for Unmanned Aircraft Self Separation , 2011 .

[77]  M. Pechoucek,et al.  Integration of Probability Collectives for Collision Avoidance in AGENTFLY , 2009 .

[78]  Milos Hauskrecht,et al.  Value-Function Approximations for Partially Observable Markov Decision Processes , 2000, J. Artif. Intell. Res..

[79]  Jeremy Kepner,et al.  D4M 2.0 schema: A general purpose high performance schema for the Accumulo database , 2013, 2013 IEEE High Performance Extreme Computing Conference (HPEC).

[80]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[81]  Jeffrey L. Gertz Mode S Surveillance Netting , 1983 .

[82]  Leslie Pack Kaelbling,et al.  Learning Policies for Partially Observable Environments: Scaling Up , 1997, ICML.

[83]  Ching-Hsue Cheng Evaluating naval tactical missile systems by fuzzy AHP based on the grade value of membership function , 1997 .

[84]  Neil Schlager,et al.  When technology fails : significant technological disasters, accidents, and failures of the twentieth century , 1994 .