Estimating Taxi-out times with a reinforcement learning algorithm

Flight delays have a significant impact on the nationpsilas economy. Taxi-out delays in particular constitute a significant portion of the block time of a flight. In the future, it can be expected that accurate predictions of dasiawheels-offpsila time may be used in determining whether an aircraft can meet its allocated slot time, thereby fitting into an en-route traffic flow. Without an accurate taxi-out time prediction for departures, there is no way to effectively manage fuel consumption, emissions, or cost. Dynamically changing operations at the airport makes it difficult to accurately predict taxi-out time. This paper describes a method for estimating average taxi-out times at the airport in 15 minute intervals of the day and at least 15 minutes in advance of aircraft scheduled gate push-back time. A probabilistic framework of stochastic dynamic programming with a learning-based solution strategy called Reinforcement Learning (RL) has been applied. Historic data from the Federal Aviation Administrationpsilas (FAA) Aviation System Performance Metrics (ASPM) database were used to train and test the algorithm. The algorithm was tested on John F. Kennedy International airport (JFK), one of the busiest, challenging, and difficult to predict airports in the United States that significantly influences operations across the entire National Airspace System (NAS). Due to the nature of departure operations at JFK the prediction accuracy of the algorithm for a given day was analyzed in two separate time periods (1) before 4:00 P.M and (2) after 4:00 P.M. On an average across 15 days, the predicted average taxi-out times matched the actual average taxi-out times within plusmn5 minutes for about 65 % of the time (for the period before 4:00 P.M) and 53 % of the time (for the period after 4:00 P.M). The prediction accuracy over the entire day within plusmn5 minutes range of accuracy was about 60 %. Further, application of the RL algorithm to estimate taxi-out times at airports with multi-dependent static surface surveillance data will likely improve the accuracy of prediction. The implications of these results for airline operations and network flow planning are discussed.

[1]  S. W. Dareing,et al.  Traffic management and airline operations , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[2]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[3]  Lance Sherry,et al.  Airport Taxi-Out Prediction Using Approximate Dynamic Programming , 2008 .

[4]  John-Paul Clarke,et al.  Modeling and control of airport queueing dynamics under severe flow restrictions , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[5]  H. Robbins A Stochastic Approximation Method , 1951 .

[6]  Craig Wanke,et al.  Predeparture Uncertainty and Prediction Performance in Collaborative Routing Coordination Tools , 2005 .

[7]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[8]  A. Futer Improving Etms' Ground Time Predictions , 2006, 2006 ieee/aiaa 25TH Digital Avionics Systems Conference.

[9]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[10]  H Oliver Gao,et al.  Aircraft Taxi-Out Emissions at Congested Hub Airports and Implications for Aviation Emissions Reduction in the United States , 2007 .

[11]  John-Paul Clarke,et al.  Queuing Model for Taxi-Out Time Estimation , 2002 .

[12]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[13]  Dimitri Jeltsema,et al.  Proceedings Of The 2000 American Control Conference , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[14]  John-Paul Clarke,et al.  A Conceptual Design of A Departure Planner Decision Aid , 2000 .

[15]  Stephen Atkins,et al.  Using surface surveillance to help reduce taxi delays , 2001 .

[16]  Robert A. Shumsky Dynamic statistical models for the prediction of aircraft take-off times , 1995 .

[17]  Lance Sherry,et al.  Accuracy of reinforcement learning algorithms for predicting aircraft taxi-out times: A case-study of Tampa Bay departures , 2010 .

[18]  John F. MacGregor,et al.  Proceedings of the American Control Conference , 1985 .

[19]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[20]  S. Atkins,et al.  Prediction and control of departure runway balancing at Dallas/Fort Worth airport , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[21]  B.S. Levy,et al.  Accurate OOOI Data: Implications for Efficient Resource Utilization , 2006, 2006 ieee/aiaa 25TH Digital Avionics Systems Conference.

[22]  B.S. Levy,et al.  Objective and automatic estimation of excess taxi-times , 2008, 2008 Integrated Communications, Navigation and Surveillance Conference.

[23]  Nicolas Pujet Modeling and control of the departure process of congested airports , 1999 .

[24]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[25]  Robert A. Shumsky Real-Time Forecasts of Aircraft Departure Queues , 1997 .

[26]  R. John Hansman,et al.  Observation and Analysis of Departure Operations at Boston Logan International Airport , 2000 .

[27]  Wayne W. Cooper,et al.  Determination of Minimum PushBack Time Predictability Needed for Near-Term Departure Scheduling using DEPARTS , 2001 .

[28]  John-Paul Clarke,et al.  Observations of Departure Processes at Logan Airport to Support the Development of Departure Planning Tools , 1999 .

[29]  Ning Xu,et al.  Propagation of Delays in the National Airspace System , 2006, UAI.

[30]  M. Ball,et al.  Estimating Flight Departure Delay Distributions—A Statistical Approach With Long-Term Trend and Short-Term Pattern , 2008 .

[31]  M. Grassi,et al.  AIAA Guidance, Navigation, and Control Conference , 2008 .