Accuracy of reinforcement learning algorithms for predicting aircraft taxi-out times: A case-study of Tampa Bay departures

Taxi-out delay is a significant portion of the block time of a flight. Uncertainty in taxi-out times reduces predictability of arrival times at the destination. This in turn results in inefficient use of airline resources such as aircraft, crew, and ground personnel. Taxi-out time prediction is also a first step in enabling schedule modifications that would help mitigate congestion and reduce emissions. The dynamically changing operation at the airport makes it difficult to accurately predict taxi-out time. In this paper we investigate the accuracy of taxi out time prediction using a nonparametric reinforcement learning (RL) based method, set in the probabilistic framework of stochastic dynamic programming. A case-study of Tampa International Airport (TPA) shows that on an average, with 93.7% probability, on any given day, our predicted mean taxi-out time for any given quarter, matches the actual mean taxi-out time for the same quarter with a standard error of 1.5 min. Also, for individual flights, the taxi-out time of 81% of them were predicted accurately within a standard error of 2 min. The predictions were done 15 min before gate departure. Gate OUT, wheels OFF, wheels ON, and gate IN (OOOI) data available in the Aviation System Performance Metric (ASPM) database maintained by the Federal Aviation Administration (FAA) was used to model and analyze the problem. The prediction accuracy is high even without the use of detailed track data.

[1]  Arnon M. Hurwitz,et al.  Run-to-Run Process Control: Literature Review and Extensions , 1997 .

[2]  Robert A. Shumsky.  Real-Time Forecasts of Aircraft Departure Queues , 1997 .

[3]  Abhijit Gosavi,et al.  An algorithm for solving semi-markov decision problems using reinforcement learning: convergence analysis and numerical results , 1999 .

[4]  N. S. Patel,et al.  Information state for robust control of set-valued discrete time systems , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[5]  Craig Wanke,et al.  Predeparture Uncertainty and Prediction Performance in Collaborative Routing Coordination Tools , 2005 .

[6]  Robert A. Shumsky.  Dynamic statistical models for the prediction of aircraft take-off times , 1995 .

[7]  Andrey V. Savkin,et al.  Qualitative Theory of Hybrid Dynamical Systems , 2012 .

[8]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.

[9]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[10]  S. Atkins,et al.  Prediction and control of departure runway balancing at Dallas/Fort Worth airport , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[11]  Richard M. Murray,et al.  Flat systems, equivalence and feedback , 2001 .

[12]  A. Futer.  Improving Etms' Ground Time Predictions , 2006, 2006 ieee/aiaa 25TH Digital Avionics Systems Conference.

[13]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[14]  John-Paul Clarke,et al.  A Conceptual Design of A Departure Planner Decision Aid , 2000 .

[15]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[16]  B.S. Levy,et al.  Accurate OOOI Data: Implications for Efficient Resource Utilization , 2006, 2006 ieee/aiaa 25TH Digital Avionics Systems Conference.

[17]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[18]  John-Paul Clarke,et al.  Identification of flow constraint and control points in departure operations at airport systems , 1998 .

[19]  Paul J. Werbos,et al.  Stable adaptive control using new critic designs , 1998, Other Conferences.

[20]  Yu-Chi Ho,et al.  Ordinal optimization of DEDS , 1992, Discret. Event Dyn. Syst..

[21]  John-Paul Clarke,et al.  Modeling and control of airport queueing dynamics under severe flow restrictions , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[22]  W. Wonham.  Linear Multivariable Control: A Geometric Approach , 1974 .

[23]  Ning Xu,et al.  Propagation of Delays in the National Airspace System , 2006, UAI.

[24]  M. Ball,et al.  Estimating Flight Departure Delay Distributions—A Statistical Approach With Long-Term Trend and Short-Term Pattern , 2008 .

[25]  John-Paul Clarke,et al.  Observations of Departure Processes at Logan Airport to Support the Development of Departure Planning Tools , 1999 .

[26]  Stephen Atkins,et al.  Using surface surveillance to help reduce taxi delays , 2001 .

[27]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[28]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[29]  H Oliver Gao,et al.  Aircraft Taxi-Out Emissions at Congested Hub Airports and Implications for Aviation Emissions Reduction in the United States , 2007 .

[30]  John-Paul Clarke,et al.  Queuing Model for Taxi-Out Time Estimation , 2002 .

[31]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[32]  D. S. Bayard.  A forward method for optimal stochastic nonlinear and adaptive control , 1991 .

[33]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[34]  M. E. Ahmed,et al.  Neural-net-based direct self-tuning control of nonlinear plants , 1997 .

[35]  J. Spall,et al.  Model-free control of nonlinear stochastic systems with discrete-time measurements , 1998, IEEE Trans. Autom. Control..

[36]  Yufeng Tu.  Air Transportation System Performance- Estimation and Comparative Analysis of Departure Delays , 2007 .

[37]  Johannes Schumacher,et al.  An Introduction to Hybrid Dynamical Systems, Springer Lecture Notes in Control and Information Sciences 251 , 1999 .