Learning To Cut By Looking Ahead: Cutting Plane Selection via Imitation Learning

Cutting planes are essential for solving mixed-integer linear problems (MILPs), because they facilitate bound improvements on the optimal solution value. For selecting cuts, modern solvers rely on manually designed heuristics that are tuned to gauge the potential effectiveness of cuts. We show that a greedy selection rule explicitly looking ahead to select cuts that yield the best bound improvement delivers strong decisions for cut selection - but is too expensive to be deployed in practice. In response, we propose a new neural architecture (NeuralCut) for imitation learning on the lookahead expert. Our model outperforms standard baselines for cut selection on several synthetic MILP benchmarks. Experiments with a B&C solver for neural network verification further validate our approach, and exhibit the potential of learning methods in this setting.

[1]  Maria-Florina Balcan,et al.  Sample Complexity of Tree Search Configuration: Cutting Planes and Beyond , 2021, NeurIPS.

[2]  Yong Yu,et al.  Learning to Select Cuts for Efficient Mixed-Integer Programming , 2021, Pattern Recognit..

[3]  Timo Berthold,et al.  MIPLIB 2017: data-driven compilation of the 6th mixed-integer programming library , 2021, Mathematical Programming Computation.

[4]  Andrea Lodi,et al.  Ecole: A Gym-like Library for Machine Learning in Combinatorial Optimization Solvers , 2020, ArXiv.

[5]  Yu Sun,et al.  Masked Label Prediction: Unified Massage Passing Model for Semi-Supervised Classification , 2020, IJCAI.

[6]  Yoshua Bengio,et al.  Parameterizing Branch-and-Bound Search Trees to Learn Branching Policies , 2020, AAAI.

[7]  Andrea Lodi,et al.  Exact Combinatorial Optimization with Graph Convolutional Neural Networks , 2019, NeurIPS.

[8]  Timothy A. Mann,et al.  On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models , 2018, ArXiv.

[9]  Marco Molinaro,et al.  Theoretical challenges towards cutting-plane selection , 2018, Math. Program..

[10]  Sebastian Stiller,et al.  Cuts, Primal Heuristics, and Learning to Branch for the Time-Dependent Traveling Salesman Problem , 2018, 1805.01415.

[11]  Russ Tedrake,et al.  Evaluating Robustness of Neural Networks with Mixed Integer Programming , 2017, ICLR.

[12]  Chih-Hong Cheng,et al.  Maximum Resilience of Artificial Neural Networks , 2017, ATVA.

[13]  Louis Wehenkel,et al.  A Machine Learning-Based Approximation of Strong Branching , 2017, INFORMS J. Comput..

[14]  Stefano Coniglio,et al.  On the Generation of Cutting Planes which Maximize the Bound Improvement , 2015, SEA.

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Edoardo Amaldi,et al.  Coordinated cutting plane generation via multi-objective separation , 2014, Math. Program..

[17]  Jeff T. Linderoth,et al.  Lookahead Branching for Mixed Integer Programming , 2011, ICS 2011.

[18]  Tobias Achterberg,et al.  Constraint integer programming , 2007 .

[19]  William J. Cook,et al.  The Traveling Salesman Problem: A Computational Study , 2007 .

[20]  Ailsa H. Land,et al.  An Automatic Method of Solving Discrete Programming Problems , 1960 .

[21]  R. Gomory AN ALGORITHM FOR THE MIXED INTEGER PROBLEM , 1960 .

[22]  Christoph Schubert,et al.  Multi-Level Lookahead Branching , 2017 .

[23]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.