Strategic air traffic flow management under uncertainties using scalable sampling-based dynamic programming and Q-learning approaches

Many Cyber-Physical Systems (CPSs) demonstrate high-dimensional environmental uncertainties that modulate the physical system dynamics and complicate the computational decision-making tasks. As an example, strategic air traffic flow management (ATFM) aims to resolve air traffic congestion through managing traffic flows at a long look-ahead time. The convective weather, a dominant factor of traffic congestion, is very uncertain and significantly complicates the decision-making process. Other uncertainties such as traffic demands further expand the uncertainty space and complicate the design of optimal control solutions that are robust to uncertainties. In this paper, we formulate the strategic ATFM as a stochastic optimal control problem, and use scalable sampling based dynamic programming and Q-learning approaches to address high-dimensional uncertainties. Simulation studies demonstrate the effectiveness of these approaches. These approaches generally apply to CPSs that operate under high-dimensional environmental uncertainties.