Reward functions for learning to control in air traffic flow management

Air Traffic Flow Management (ATFM) is a complex decision-making process with multiple stakeholders involved. In this decision loop, a Multi-agent system is developed for both simulation and daily operations to support human decisions. Considering human factors in ATFM, the method of Reinforcement Learning (RL) is suitable in the acquirement of the knowledge and experience of the controllers to assist them in the next control activities. The paper presents the recent development of reinforcement learning and its reward structure for ATFM decision making. Two types of reward functions are proposed for agent-based RL in the application of air traffic management: (1) Reward function considering safety separation and fairness impact among different commercial entities in Ground Holding Problem (GHP) and (2) Reward function considering safety separation in Air Holding Problem (AHP). Real case studies in Brazil are described to show the effectiveness and efficiency of the developed reward functions in the controller decision process of ATFM.