The Effect of Robust Decisions on the Cost of Uncertainty in Military Airlift Operations

There are a number of sources of randomness that arise in military airlift operations. However, the cost of uncertainty can be difficult to estimate, and is easy to overestimate if we use simplistic decision rules. Using data from Canadian military airlift operations, we study the effect of uncertainty in customer demands as well as aircraft failures, on the overall cost. The system is first analyzed using the types of myopic decision rules widely used in the research literature. The performance of the myopic policy is then compared to the results obtained using robust decisions that account for the uncertainty of future events. These are obtained by modeling the problem as a dynamic program, and solving Bellman’s equations using approximate dynamic programming. The experiments show that even approximate solutions to Bellman’s equations produce decisions that reduce the cost of uncertainty.

[1]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[2]  Tongqiang Tony Wu.  The Optimizing Simulator: An Intelligent Analysis Tool for the Military Airlift Problem , 2003 .

[3]  Warren B. Powell,et al.  Dynamic-Programming Approximations for Stochastic Time-Staged Integer Multicommodity-Flow Problems , 2006, INFORMS J. Comput..

[4]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[5]  BoukhtoutaAbdeslem,et al.  The Effect of Robust Decisions on the Cost of Uncertainty in Military Airlift Operations , 2011 .

[6]  Jacques Desrosiers,et al.  Routing with time windows by column generation , 1983, Networks.

[7]  Robert J. Vanderbei,et al.  Robust Optimization of Large-Scale Systems , 1995, Oper. Res..

[8]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[9]  Warren B. Powell,et al.  An Adaptive Dynamic Programming Algorithm for the Heterogeneous Resource Allocation Problem , 2002, Transp. Sci..

[10]  Jacques Desrosiers,et al.  Time Constrained Routing and Scheduling , 1992 .

[11]  David A. Goggins.  Stochastic Modeling for Airlift Mobility. , 1995 .

[12]  Warren B. Powell,et al.  The optimizing-simulator: An illustration using the military airlift problem , 2009, TOMC.

[13]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[14]  Warren B. Powell,et al.  An Approximate Dynamic Programming Algorithm for Large-Scale Fleet Management: A Case Application , 2009, Transp. Sci..

[15]  J. L. Midler,et al.  Stochastic programming models for scheduling airlift operations , 1969 .

[16]  Michail G. Lagoudakis,et al.  Least-Squares Policy Iteration , 2003, J. Mach. Learn. Res..

[17]  David P. Morton,et al.  A Stochastic Program for Optimizing Military Sealift Subject to Attack , 2009 .

[18]  Warren B. Powell,et al.  An algorithm for approximating piecewise linear concave functions from sample gradients , 2003, Oper. Res. Lett..

[19]  David P. Morton,et al.  Application and extension of the Thruput II optimization model for airlift mobility , 1997 .

[20]  W.B. Powell,et al.  Approximate dynamic programming for high dimensional resource allocation problems , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[21]  David P. Morton,et al.  Optimizing Military Airlift , 2002, Oper. Res..

[22]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[23]  Shie Mannor,et al.  Basis Function Adaptation in Temporal Difference Reinforcement Learning , 2005, Ann. Oper. Res..

[24]  G. Dantzig,et al.  The Allocation of Aircraft to Routes—An Example of Linear Programming Under Uncertain Demand , 1956 .

[25]  Alan R. Washburn,et al.  The LP/POMDP marriage: Optimization with imperfect information , 2000 .

[26]  John M. Mulvey,et al.  A New Scenario Decomposition Method for Large-Scale Stochastic Optimization , 1995, Oper. Res..

[27]  Warrren B Powell,et al.  An Adaptive, Distribution-Free Algorithm for the Newsvendor Problem with Censored Demands, with Applications to Inventory and Distribution , 2001 .

[28]  Warren B. Powell,et al.  Adaptive stepsizes for recursive estimation with applications in approximate dynamic programming , 2006, Machine Learning.

[29]  Dean A. Jones,et al.  Robust optimization for fleet planning under uncertainty , 2003 .

[30]  Warren B. Powell,et al.  Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics) , 2007 .

[31]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[32]  Warren B. Powell,et al.  Using Low-Dimensional Patterns in Optimizing Simulators: An Illustration for the Airlift Mobility Problem , 2003 .

[33]  Warren B. Powell,et al.  An Adaptive Dynamic Programming Algorithm for Dynamic Fleet Management, I: Single Period Travel Times , 2002, Transp. Sci..

[34]  T. Ralphs,et al.  Decomposition Methods , 2010 .

[35]  Warren B. Powell,et al.  The Dynamic Assignment Problem , 2004, Transp. Sci..

[36]  R. Tyrrell Rockafellar,et al.  Scenarios and Policy Aggregation in Optimization Under Uncertainty , 1991, Math. Oper. Res..

[37]  J. R Crino,et al.  Solving the theater distribution vehicle routing and scheduling problem using group theoretic tabu search , 2004 .

[38]  S. Sen.  Algorithms for Stochastic Mixed-Integer Programming Models , 2005 .

[39]  Warren B. Powell,et al.  Learning Algorithms for Separable Approximations of Discrete Stochastic Optimization Problems , 2004, Math. Oper. Res..

[40]  Warren B. Powell,et al.  Approximate dynamic programming for high dimensional resource allocation problems , 2005 .

[41]  Dimitri P. Bertsekas,et al.  Approximate Dynamic Programming , 2017, Encyclopedia of Machine Learning and Data Mining.

[42]  Charles M. Macal,et al.  Modelling force deployments from army installations using the Transportation System Capability (TRANSCAP) model: A standardized approach , 2004 .

[43]  Warren B. Powell,et al.  Solving Parallel Machine Scheduling Problems by Column Generation , 1999, INFORMS J. Comput..

[44]  Julia L. Higle,et al.  Stochastic Decomposition: An Algorithm for Two-Stage Linear Programs with Recourse , 1991, Math. Oper. Res..

[45]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[46]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[47]  Jacques Desrosiers,et al.  Chapter 2 Time constrained routing and scheduling , 1995 .

[48]  Paul S. Killingsworth,et al.  Should C-17s Be Used to Carry In-Theater Cargo During Major Deployments? , 1997 .

[49]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.

[50]  Csaba Szepesvári,et al.  Algorithms for Reinforcement Learning , 2010, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[51]  Peter Kall,et al.  Stochastic Programming , 1995 .

[52]  John R. Birge,et al.  Introduction to Stochastic Programming , 1997 .