Learning to Select Cuts for Efficient Mixed-Integer Programming

Cutting plane methods play a significant role in modern solvers for tackling mixed-integer programming (MIP) problems. Proper selection of cuts would remove infeasible solutions in the early stage, thus largely reducing the computational burden without hurting the solution accuracy. However, the major cut selection approaches heavily rely on heuristics, which strongly depend on the specific problem at hand and thus limit their generalization capability. In this paper, we propose a data-driven and generalizable cut selection approach, named Cut Ranking , in the settings of multiple instance learning. To measure the quality of the candidate cuts, a scoring function, which takes the instance-specific cut features as inputs, is trained and applied in cut ranking and selection. In order to evaluate our method, we conduct extensive experiments on both synthetic datasets and real-world datasets. Compared with commonly used heuristics for cut selection, the learning-based policy has shown to be more effective, and is capable of generalizing over multiple problems with different properties. Cut Ranking has been deployed in an industrial solver for large-scale MIPs. In the online A/B testing of the product planning problems with more than 10 7 variables and constraints daily, Cut Ranking has achieved the average speedup ratio of 12.42% over the production solver without any accuracy loss of solution. © 2021 Elsevier Ltd. All rights reserved.

[1]  M. E. Gershon Mine scheduling optimization with mixed integer programming , 1983 .

[2]  Ahmet B. Keha,et al.  Mixed integer programming formulations for single machine scheduling problems , 2009, Comput. Ind. Eng..

[3]  Timo Berthold,et al.  Rounding and Propagation Heuristics for Mixed Integer Programming , 2011, OR.

[4]  Robert E. Bixby,et al.  A Brief History of Linear and Mixed-Integer Programming Computation , 2012 .

[5]  Tobias Achterberg,et al.  Mixed Integer Programming: Analyzing 12 Years of Progress , 2013 .

[6]  ● Pytorch,et al.  Attention! , 1998, Trends in Cognitive Sciences.

[7]  Laurence A. Wolsey,et al.  Cutting planes in integer and mixed integer programming , 2002, Discret. Appl. Math..

[8]  Lawrence V. Snyder,et al.  Reinforcement Learning for Solving the Vehicle Routing Problem , 2018, NeurIPS.

[9]  Leyuan Shi,et al.  Mixed integer programming in production planning with backlogging and setup carryover: modeling and algorithms , 2013, Discret. Event Dyn. Syst..

[10]  Max Welling,et al.  Attention, Learn to Solve Routing Problems! , 2018, ICLR.

[11]  Boris Babenko Multiple Instance Learning: Algorithms and Applications , 2008 .

[12]  Laurence A. Wolsey,et al.  Mixed Integer Programming , 2008, Wiley Encyclopedia of Computer Science and Engineering.

[13]  B. Moor,et al.  Mixed integer programming for multi-vehicle path planning , 2001, 2001 European Control Conference (ECC).

[14]  Christian Tjandraatmadja,et al.  Solving Mixed Integer Programs Using Neural Networks , 2020, ArXiv.

[15]  Sheldon H. Jacobson,et al.  Branch-and-bound algorithms: A survey of recent advances in searching, branching, and pruning , 2016, Discret. Optim..

[16]  Le Song,et al.  Learning to Branch in Mixed Integer Programming , 2016, AAAI.

[17]  J. How,et al.  Mixed-integer programming for control , 2005, Proceedings of the 2005, American Control Conference, 2005..

[18]  Matthew J. Saltzman,et al.  Coin-Or: An Open-Source Library for Optimization , 2002 .

[19]  Zoran Obradovic,et al.  Aerosol Optical Depth Prediction from Satellite Obsercations by Multiple Instance Regression , 2008, SDM.

[20]  James R. Foulds,et al.  A review of multi-instance learning assumptions , 2010, The Knowledge Engineering Review.

[21]  Samy Bengio,et al.  Neural Combinatorial Optimization with Reinforcement Learning , 2016, ICLR.

[22]  Yoshua Bengio,et al.  Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon , 2018, Eur. J. Oper. Res..

[23]  Gérard Cornuéjols,et al.  Valid inequalities for mixed integer linear programs , 2007, Math. Program..

[24]  P. Alam,et al.  H , 1887, High Explosives, Propellants, Pyrotechnics.

[25]  Yori Zwols,et al.  Linear and integer optimization: Theory and practice , 2015 .

[26]  George L. Nemhauser,et al.  Learning to Run Heuristics in Tree Search , 2017, IJCAI.

[27]  J. Clausen,et al.  Branch and Bound Algorithms-Principles and Examples , 2003 .

[28]  David Correa Martins,et al.  U-curve: A branch-and-bound optimization algorithm for U-shaped cost functions on Boolean lattices applied to the feature selection problem , 2010, Pattern Recognit..

[29]  Kiri L. Wagstaff,et al.  Salience Assignment for Multiple-Instance Regression , 2007 .

[30]  He He,et al.  Learning to Search in Branch and Bound Algorithms , 2014, NIPS.

[31]  André R. S. Amaral A mixed-integer programming formulation for the double row layout of machines in manufacturing systems , 2019 .

[32]  J. Mitchell Branch-and-Cut Algorithms for Combinatorial Optimization Problems , 1988 .

[33]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[34]  Marc Peter Deisenroth,et al.  Deep Reinforcement Learning: A Brief Survey , 2017, IEEE Signal Processing Magazine.

[35]  Marco Loog,et al.  Multiple instance learning with bag dissimilarities , 2013, Pattern Recognit..

[36]  John K. Karlof,et al.  Integer programming : theory and practice , 2005 .

[37]  Robert E. Bixby,et al.  Mixed-Integer Programming: A Progress Report , 2004, The Sharpest Cut.

[38]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[39]  Andrei Popescu-Belis,et al.  Explaining the Stars: Weighted Multiple-Instance Learning for Aspect-Based Sentiment Analysis , 2014, EMNLP.

[40]  E. L. Lawler,et al.  Branch-and-Bound Methods: A Survey , 1966, Oper. Res..

[41]  Laurence A. Wolsey,et al.  Production Planning by Mixed Integer Programming , 2010 .

[42]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[43]  Eric Granger,et al.  Multiple instance learning: A survey of problem characteristics and applications , 2016, Pattern Recognit..

[44]  P. Alam ‘T’ , 2021, Composites Engineering: An A–Z Guide.

[45]  Yunhao Tang,et al.  Reinforcement Learning for Integer Programming: Learning to Cut , 2019, ICML.

[46]  A. Lodi,et al.  Heuristics in Mixed Integer Programming , 2011 .

[47]  Ted K. Ralphs,et al.  Integer and Combinatorial Optimization , 2013 .

[48]  Maria-Florina Balcan,et al.  Learning to Branch , 2018, ICML.

[49]  Andrea Lodi,et al.  Exact Combinatorial Optimization with Graph Convolutional Neural Networks , 2019, NeurIPS.