Multiagent Reinforcement Learning Methods for Resolving Demand - Capacity Imbalances

In this article, we explore the computation of joint policies for autonomous agents, representing flights, to resolve congestions problems in the Air Traffic Management (ATM) domain in the context of Demand-Capacity Balance (DCB) process. We formalize the problem as a multi-agent Markov Decision Process (MDP) towards deciding flight ground delays to resolve imbalances, during the pre-tactical phase. To this end, we present and evaluate multi-agent reinforcement learning methods. An experimental study on real-world cases confirms the effectiveness of our approach.

[1]  Peter Stone,et al.  Multiagent traffic management: a reservation-based intersection control mechanism , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[2]  Nikos A. Vlassis,et al.  Collaborative Multiagent Reinforcement Learning by Payoff Propagation , 2006, J. Mach. Learn. Res..

[3]  Sam Devlin,et al.  Resource Abstraction for Reinforcement Learning in Multiagent Congestion Problems , 2016, AAMAS.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Igal Milchtaich,et al.  Social optimality and cooperation in nonatomic congestion games , 2004, J. Econ. Theory.

[6]  Kagan Tumer,et al.  A multiagent approach to managing air traffic flow , 2010, Autonomous Agents and Multi-Agent Systems.

[7]  D. Koller,et al.  Planning under uncertainty in complex structured environments , 2003 .

[8]  Ana L. C. Bazzan,et al.  Agents in Traffic Modelling - From Reactive to Social Behaviour , 1999, KI.

[9]  Michail G. Lagoudakis,et al.  Coordinated Reinforcement Learning , 2002, ICML.

[10]  Moshe Tennenholtz,et al.  Congestion games with failures , 2011, Discret. Appl. Math..

[11]  Andreas S. Schulz,et al.  Network flow problems and congestion games: complexity and approximation results , 2006 .

[12]  Georgios Chalkiadakis,et al.  Learning Policies for Resolving Demand-Capacity Imbalances During Pre-tactical Air Traffic Management , 2017, MATES.

[13]  George A. Vouros,et al.  Multiagent Reinforcement Learning Methods to Resolve Demand Capacity Balance Problems , 2018, SETN.

[14]  R. Rosenthal.  A class of games possessing pure-strategy Nash equilibria , 1973 .