Predicting Solution Summaries to Integer Linear Programs under Imperfect Information with Machine Learning

The paper provides a methodological contribution at the intersection of machine learning and operations research. Namely, we propose a methodology to quickly predict solution summaries (i.e., solution descriptions at a given level of detail) to discrete stochastic optimization problems. We approximate the solutions based on supervised learning and the training dataset consists of a large number of deterministic problems that have been solved independently and offline. Uncertainty regarding a missing subset of the inputs is addressed through sampling and aggregation methods. Our motivating application concerns booking decisions of intermodal containers on double-stack trains. Under perfect information, this is the so-called load planning problem and it can be formulated by means of integer linear programming. However, the formulation cannot be used for the application at hand because of the restricted computational budget and unknown container weights. The results show that standard deep learning algorithms allow one to predict descriptions of solutions with high accuracy in very short time (milliseconds or less).

[1]  Barry L. Nelson,et al.  Control variates for quantile estimation , 1987, WSC '87.

[2]  R. Wets,et al.  Stochastic programming , 1989 .

[3]  James R. Wilson,et al.  Correlation-induction techniques for estimating quantiles in simulation experiments , 1995, WSC '95.

[4]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[5]  B. Nelson,et al.  Control Variates for Probability and Quantile Estimation , 1998 .

[6]  Kate Smith-Miles,et al.  Neural Networks for Combinatorial Optimization: A Review of More Than a Decade of Research , 1999, INFORMS J. Comput..

[7]  S. Keleş,et al.  Statistical Applications in Genetics and Molecular Biology Asymptotic Optimality of Likelihood-Based Cross-Validation , 2011 .

[8]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  G. Evans,et al.  Learning to Optimize , 2008 .

[11]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[12]  John Langford,et al.  Search-based structured prediction , 2009, Machine Learning.

[13]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[14]  Csaba Szepesvári,et al.  Algorithms for Reinforcement Learning , 2010, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[15]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[16]  B. F. J. La Maire,et al.  Comparison of neural networks for solving the travelling salesman problem , 2012, 11th Symposium on Neural Network Applications in Electrical Engineering.

[17]  A. A. Bhatti,et al.  Critical analysis of hopfield's neural network model for TSP and its comparison with heuristic algorithm for shortest path computation , 2012, Proceedings of 2012 9th International Bhurban Conference on Applied Sciences & Technology (IBCAST).

[18]  Alexander Shapiro,et al.  Lectures on Stochastic Programming - Modeling and Theory, Second Edition , 2014, MOS-SIAM Series on Optimization.

[19]  Li Zhou,et al.  A Survey on Contextual Multi-armed Bandits , 2015, ArXiv.

[20]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[21]  John Langford,et al.  Learning to Search Better than Your Teacher , 2015, ICML.

[22]  E Weinan,et al.  Deep Learning Approximation for Stochastic Control Problems , 2016, ArXiv.

[23]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[24]  Dimitris Bertsimas,et al.  Inventory Management in the Era of Big Data , 2016 .

[25]  Le Song,et al.  Learning to Branch in Mixed Integer Programming , 2016, AAAI.

[26]  Marco Fraccaro,et al.  Using OR + AI to Predict the Optimal Production of Offshore Wind Parks: A Preliminary Study , 2017 .

[27]  Marco E. Lübbecke,et al.  Learning When to Use a Decomposition , 2017, CPAIOR.

[28]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[29]  Sara Ahmadian,et al.  Approximation Algorithms for Clustering and Facility Location Problems , 2017 .

[30]  Andrea Lodi,et al.  On learning and branching: a survey , 2017 .

[31]  Samy Bengio,et al.  Neural Combinatorial Optimization with Reinforcement Learning , 2016, ICLR.

[32]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[33]  Wouter Kool,et al.  Attention Solves Your TSP, Approximately , 2018 .

[34]  Lior Wolf,et al.  Learning the Multiple Traveling Salesmen Problem with Permutation Invariant Pooling Networks , 2018, ArXiv.

[35]  Teodor Gabriel Crainic,et al.  The load planning problem for double-stack intermodal trains , 2017, Eur. J. Oper. Res..

[36]  Dimitris Bertsimas,et al.  From Predictive to Prescriptive Analytics , 2014, Manag. Sci..

[37]  Lawrence V. Snyder,et al.  Applying deep learning to the newsvendor problem , 2016, IISE Trans..