Scalable Multi-Agent Computational Guidance with Separation Assurance for Autonomous Urban Air Mobility

Electric vertical takeoff and landing vehicles are becoming promising for on-demand air transportation in urban air mobility (UAM). However, successfully bringing such vehicles and airspace operati...

[1]  B. Faverjon,et al.  Probabilistic Roadmaps for Path Planning in High-Dimensional Con(cid:12)guration Spaces , 1996 .

[2]  Kenneth H. Goodrich,et al.  Enabling Airspace Integration for High-Density On-Demand Mobility Operations , 2017 .

[3]  Mykel J. Kochenderfer,et al.  Deep Neural Network Compression for Aircraft Collision Avoidance Systems , 2018, Journal of Guidance, Control, and Dynamics.

[4]  Guodong Zhu,et al.  Pre-Departure Planning for Urban Air Mobility Flights with Dynamic Airspace Reservation , 2019 .

[5]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[6]  M. Hossein Mansur,et al.  A Practical Approach to Obstacle Field Route Planning for Unmanned Rotorcraft , 2004 .

[7]  Hyochoong Bang,et al.  Proportional navigation-based collision avoidance for UAVs , 2009 .

[8]  B. Moor,et al.  Mixed integer programming for multi-vehicle path planning , 2001, 2001 European Control Conference (ECC).

[9]  Maria Consiglio,et al.  DAIDALUS: Detect and avoid alerting logic for unmanned systems , 2015, 2015 IEEE/AIAA 34th Digital Avionics Systems Conference (DASC).

[10]  A. Richards,et al.  Decentralized model predictive control of cooperating UAVs , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[11]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[12]  Michael D. Patterson,et al.  System-Level Urban Air Mobility Transportation Modeling and Determination of Energy-Related Constraints , 2018, 2018 Aviation Technology, Integration, and Operations Conference.

[13]  E. Feron,et al.  Resolution of Conflicts Involving Many Aircraft via Semidefinite Programming , 2001 .

[14]  Colin Camerer.  Behavioral Game Theory: Experiments in Strategic Interaction , 2003 .

[15]  Jonathan P. How,et al.  Decentralized path planning for multi-agent teams in complex environments using rapidly-exploring random trees , 2011, 2011 IEEE International Conference on Robotics and Automation.

[16]  Jonathan P. How,et al.  Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Rémi Coulom,et al.  Computing "Elo Ratings" of Move Patterns in the Game of Go , 2007, J. Int. Comput. Games Assoc..

[18]  Sergey Levine,et al.  PLATO: Policy learning using adaptive trajectory optimization , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[20]  Sheng Li,et al.  Optimizing Collision Avoidance in Dense Airspace using Deep Reinforcement Learning , 2019, ArXiv.

[21]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[22]  Mykel J. Kochenderfer,et al.  Robust Airborne Collision Avoidance through Dynamic Programming , 2011 .

[23]  Josh Bertram,et al.  Distributed Computational Guidance for High-Density Urban Air Mobility with Cooperative and Non-Cooperative Collision Avoidance , 2020, AIAA Scitech 2020 Forum.

[24]  John-Paul Clarke,et al.  A mixed integer program for flight-level assignment and speed control for conflict resolution , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[25]  A. Shwartz,et al.  Handbook of Markov decision processes : methods and applications , 2002 .

[26]  Antonio A. Trani,et al.  Preliminary Considerations for ODM Air Traffic Management based on Analysis of Commuter Passenger Demand and Travel Patterns for the Silicon Valley Region of California , 2017 .

[27]  Mykel J. Kochenderfer,et al.  Aircraft Collision Avoidance Using Monte Carlo Real-Time Belief Space Search , 2011, J. Intell. Robotic Syst..

[28]  Roland Siegwart,et al.  Introduction to Autonomous Mobile Robots , 2004 .

[29]  Aníbal Ollero,et al.  Path planning based on Genetic Algorithms and the Monte-Carlo method to avoid aerial vehicle collisions under uncertainties , 2011, 2011 IEEE International Conference on Robotics and Automation.

[30]  Vijay Kumar,et al.  Mixed-integer quadratic program trajectory generation for heterogeneous quadrotor teams , 2012, 2012 IEEE International Conference on Robotics and Automation.

[31]  Jonathan P. How,et al.  Aircraft trajectory planning with collision avoidance using mixed integer linear programming , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[32]  Bart De Schutter,et al.  Multi-agent Reinforcement Learning: An Overview , 2010 .

[33]  Michal Pechoucek,et al.  Multi-agent RRT: sampling-based cooperative pathfinding , 2013, AAMAS.

[34]  Jan H. van Schuppen.  Jump linear systems in automatic control : M. Mariton , 1992, Autom..

[35]  Dinesh Manocha,et al.  Reciprocal n-Body Collision Avoidance , 2011, ISRR.

[36]  Shahab Hasan,et al.  JPDO Case Study of NextGen High Density Operations , 2009 .

[37]  D. Stahl,et al.  On Players' Models of Other Players: Theory and Experimental Evidence , 1995 .

[38]  D. J. White,et al.  A Survey of Applications of Markov Decision Processes , 1993 .

[39]  Jonathan P. How,et al.  Decision Making Under Uncertainty: Theory and Application , 2015 .

[40]  Bruce A. Conway,et al.  Discrete approximations to optimal trajectories using direct transcription and nonlinear programming , 1992 .

[41]  Steven L. Waslander,et al.  The Stanford testbed of autonomous rotorcraft for multi agent control (STARMAC) , 2004, The 23rd Digital Avionics Systems Conference (IEEE Cat. No.04CH37576).

[42]  Craig Boutilier,et al.  Decision-Theoretic Planning: Structural Assumptions and Computational Leverage , 1999, J. Artif. Intell. Res..

[43]  Groupe Pdmia.  Markov Decision Processes In Artificial Intelligence , 2009 .

[44]  Stephen P. Cook,et al.  A Quantitative Metric to Enable Unmanned Aircraft Systems to Remain Well Clear , 2015 .

[45]  S. Kahne,et al.  Air traffic management: evolution with technology , 1996 .

[46]  Min-Jea Tahk,et al.  UAV collision avoidance based on geometric approach , 2008, 2008 SICE Annual Conference.

[47]  Lorenz T. Biegler,et al.  Dynamic Optimization Strategies for Three-Dimensional Conflict Resolution of Multiple Aircraft , 2004 .

[48]  Bruce A. Conway,et al.  Particle Swarm Optimization Applied to Space Trajectories , 2010 .

[49]  D. Koditschek,et al.  Robot navigation functions on manifolds with boundary , 1990 .

[50]  James K. Kuchar,et al.  A review of conflict detection and resolution modeling methods , 2000, IEEE Trans. Intell. Transp. Syst..

[51]  Fast-Forwarding to a Future of On-Demand Urban Air Transportation , 2016 .

[52]  Jan Willemson,et al.  Improved Monte-Carlo Search , 2006 .

[53]  Henk A. P. Blom,et al.  Mathematical Models for Air Traffic Conflict and Collision Probability Estimation , 2019, IEEE Transactions on Intelligent Transportation Systems.

[54]  Mark Peters,et al.  A DECENTRALIZED CONTROL STRATEGY FOR DISTRIBUTED AIR/GROUND TRAFFIC SEPARATION , 2000 .

[55]  S. LaValle.  Rapidly-exploring random trees : a new tool for path planning , 1998 .

[56]  Parimal Kopardekar,et al.  Distributed Air/Ground Traffic Management - Technology and Concept Demonstration Report , 2002 .

[57]  S. Shankar Sastry,et al.  Conflict resolution for air traffic management: a study in multiagent hybrid systems , 1998, IEEE Trans. Autom. Control..

[58]  J. Brian Burns,et al.  Path planning using Laplace's equation , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[59]  Jia Liu,et al.  Multi-Agent Autonomous Operations in Urban Air Mobility with Communication Constraints , 2020 .

[60]  Paul Lee,et al.  Joint NASA Ames/Langley Experimental Evaluation of Integrated Air/Ground Operations for En Route Free Maneuvering , 2005 .

[61]  H. Jaap van den Herik,et al.  Progressive Strategies for Monte-Carlo Tree Search , 2008 .

[62]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[63]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[64]  Dou Long,et al.  Air Transportation Strategic Trade Space Modeling and Assessment Through Analysis of On-Demand Air Mobility with Electric Aircraft , 2012 .

[65]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[66]  Mykel J. Kochenderfer,et al.  Policy compression for aircraft collision avoidance systems , 2016, 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC).

[67]  Demis Hassabis,et al.  Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm , 2017, ArXiv.

[68]  Marcus Johnson,et al.  Unmanned Aircraft System Traffic Management (UTM) Concept of Operations , 2016 .

[69]  韓國航空大學 航空 械工學科.  美聯邦航空廳(Federal Aviation Administration)의 航空機 製作檢査 制度의 現況 , 1979 .

[70]  Craig Boutilier,et al.  Sequential Optimality and Coordination in Multiagent Systems , 1999, IJCAI.

[71]  Antonio Bicchi,et al.  Conflict resolution problems for air traffic management systems solved with mixed integer programming , 2002, IEEE Trans. Intell. Transp. Syst..

[72]  Paul W. Goldberg,et al.  The complexity of computing a Nash equilibrium , 2006, STOC '06.

[73]  Angela P. Schoellig,et al.  Generation of collision-free trajectories for a quadrocopter fleet: A sequential convex programming approach , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[74]  Craig Boutilier,et al.  Planning, Learning and Coordination in Multiagent Decision Processes , 1996, TARK.

[75]  Rafael Apaza,et al.  Urban Air Mobility Airspace Integration Concepts and Considerations , 2018, 2018 Aviation Technology, Integration, and Operations Conference.

[76]  Brian T. Baxley,et al.  Safety Performance of Airborne Separation: Preliminary Baseline Testing , 2007 .

[77]  Josh Bertram,et al.  An Efficient Algorithm for Self-Organized Terminal Arrival in Urban Air Mobility , 2020 .

[78]  Hendrikus G. Visser,et al.  Cost-Benefit Study of Free Flight with Airborne Separation Assurance , 2001 .

[79]  Gerald Tesauro,et al.  On-line Policy Improvement using Monte-Carlo Search , 1996, NIPS.

[80]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[81]  Thomas R. Ioerger,et al.  Automated Conflict Resolution for Air Trac Management Using Cooperative Multiagent Negotiation , 2004 .

[82]  Rémi Munos,et al.  Algorithms for Infinitely Many-Armed Bandits , 2008, NIPS.

[83]  Leslie Pack Kaelbling,et al.  Collision Avoidance for Unmanned Aircraft using Markov Decision Processes , 2010 .

[84]  R. Bellman.  A Markovian Decision Process , 1957 .

[85]  Eric Feron,et al.  Decentralized Cooperative Trajectory Planning of Multiple Aircraft with Hard Safety Guarantees , 2004 .

[86]  Mykel J. Kochenderfer,et al.  Neural Network Guidance for UAVs , 2017 .

[87]  Valentin Polishchuk,et al.  Geometric Approach Towards Airspace Assessment for Emerging Operations , 2020 .

[88]  S. Shankar Sastry,et al.  Decentralized nonlinear model predictive control of multiple flying robots , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[89]  John Canny,et al.  The complexity of robot motion planning , 1988 .

[90]  Mykel J. Kochenderfer,et al.  Markov Decision Process-Based Distributed Conflict Resolution for Drone Air Traffic Management , 2017 .

[91]  Lillian Gipson.  NASA Embraces Urban Air Mobility, Calls for Market Study , 2017 .

[92]  D. Stahl,et al.  Experimental evidence on players' models of other players , 1994 .

[93]  Michael D. Patterson,et al.  A Proposed Approach to Studying Urban Air Mobility Missions Including an Initial Exploration of Mission Requirements , 2018 .

[94]  J Wing David,et al.  Airborne Use of Traffic Intent Information in a Distributed Air-Ground Traffic Management Concept: Experiment Design and Preliminary Results , 2001 .

[95]  Kenneth H. Goodrich,et al.  High Speed Mobility Through On-Demand Aviation , 2013 .

[96]  Antonio Bicchi,et al.  Probabilistic verification of a decentralized policy for conflict resolution in multi-agent systems , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[97]  C. Tomlin,et al.  Decentralized optimization, with application to multiple aircraft coordination , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[98]  Joseph Post,et al.  Assessing the Benefits of NextGen Performance-Based Navigation , 2013 .

[99]  Mykel J. Kochenderfer,et al.  Next-Generation Airborne Collision Avoidance System , 2012 .

[100]  Kapil Sheth,et al.  Performance evaluation of airborne separation assurance for free flight , 2000 .

[101]  Peng Wei,et al.  eVTOL Arrival Sequencing and Scheduling for On-Demand Urban Air Mobility , 2018, 2018 IEEE/AIAA 37th Digital Avionics Systems Conference (DASC).

[102]  Robert A. Vivona,et al.  Autonomous Operations Planner: A Flexible Platform for Research in Flight-Deck Support for Airborne Self-Separation , 2012 .

[103]  Eugene A. Feinberg,et al.  Handbook of Markov Decision Processes , 2002 .

[104]  Marcel Mongeau,et al.  Aircraft Conflict Resolution by Genetic Algorithm and B-Spline Approximation , 2010 .

[105]  Jaewoo Jung,et al.  Applying Required Navigation Performance Concept for Traffic Management of Small Unmanned Aircraft Systems , 2016 .

[106]  Christabelle S. Bosson,et al.  Simulation Evaluations of an Autonomous Urban Air Mobility Network Management and Separation Service , 2018, 2018 Aviation Technology, Integration, and Operations Conference.

[107]  Maria Consiglio,et al.  ICAROUS: Integrated configurable algorithms for reliable operations of unmanned systems , 2016, 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC).

[108]  Xuxi Yang.  Autonomous On-Demand Free Flight Operations in Urban Air Mobility using Monte Carlo Tree Search , 2018 .

[109]  Raffaello D'Andrea,et al.  Theory and implementation of path planning by negotiation for decentralized agents , 2008, Robotics Auton. Syst..

[110]  Jacco M. Hoekstra,et al.  Designing for safety: the 'free flight' air traffic management concept , 2002, Reliab. Eng. Syst. Saf..

[111]  Peng Wei,et al.  Energy Optimal Speed Profile for Arrival of Tandem Tilt-Wing eVTOL Aircraft with RTA Constraint , 2018, 2018 IEEE CSAA Guidance, Navigation and Control Conference (CGNCC).

[112]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[113]  Daniel E. Koditschek,et al.  Exact robot navigation using cost functions: the case of distinct spherical boundaries in E/sup n/ , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[114]  S. Sastry,et al.  An Evasive Maneuvering Algorithm for UAVs in See-and-Avoid Situations , 2007, 2007 American Control Conference.

[115]  Henk A. P. Blom,et al.  Safety Evaluation of Advanced Self-Separation Under Very High En Route Traffic Demand , 2015, J. Aerosp. Inf. Syst..

[116]  Fedja Netjasov,et al.  A review of research on risk and safety modelling in civil aviation , 2008 .

[117]  Nataliya Sokolovska,et al.  Continuous Upper Confidence Trees , 2011, LION.

[118]  Soon-Jo Chung,et al.  Decentralized Model Predictive Control of Swarms of Spacecraft Using Sequential Convex Programming , 2013 .

[119]  Peng Wei,et al.  Heuristic Approach for Arrival Sequencing and Scheduling for eVTOL Aircraft in On-Demand Urban Air Mobility , 2018, 2018 IEEE/AIAA 37th Digital Avionics Systems Conference (DASC).

[120]  Peng Wei,et al.  Autonomous Separation Assurance in An High-Density En Route Sector: A Deep Multi-Agent Reinforcement Learning Approach , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).