Learning to Branch in Mixed Integer Programming

The design of strategies for branching in Mixed Integer Programming (MIP) is guided by cycles of parameter tuning and offline experimentation on an extremely heterogeneous testbed, using the average performance. Once devised, these strategies (and their parameter settings) are essentially input-agnostic. To address these issues, we propose a machine learning (ML) framework for variable branching in MIP. Our method observes the decisions made by Strong Branching (SB), a time-consuming strategy that produces small search trees, collecting features that characterize the candidate branching variables at each node of the tree. Based on the collected data, we learn an easy-to-evaluate surrogate function that mimics the SB strategy, by means of solving a learning-to-rank problem, common in ML. The learned ranking function is then used for branching. The learning is instance-specific, and is performed on-the-fly while executing a branch-and-bound search to solve the instance. Experiments on benchmark instances indicate that our method produces significantly smaller search trees than existing heuristics, and is competitive with a state-of-the-art commercial solver.

[1]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988 .

[2]  John N. Hooker,et al.  Testing heuristics: We have it all wrong , 1995, J. Heuristics.

[3]  Martin W. P. Savelsbergh,et al.  A Computational Study of Search Strategies for Mixed Integer Programming , 1999, INFORMS J. Comput..

[4]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[5]  Dan Roth,et al.  Integer linear programming inference for conditional random fields , 2005, ICML.

[6]  Thorsten Koch,et al.  Branching rules revisited , 2005, Oper. Res. Lett..

[7]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[8]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[9]  Tobias Achterberg,et al.  Constraint integer programming , 2007 .

[10]  Sheila A. McIlraith,et al.  Using Expectation Maximization to Find Likely Assignments for Solving CSP's , 2007, AAAI.

[11]  Tao Qin,et al.  LETOR: Benchmark Dataset for Research on Learning to Rank for Information Retrieval , 2007 .

[12]  John W. Chinneck,et al.  Active-constraint variable ordering for faster feasibility of mixed integer linear programs , 2007, Math. Program..

[13]  Jonathan Schaeffer,et al.  Learning from Multiple Heuristics , 2008, AAAI.

[14]  F. Hutter,et al.  ParamILS: An Automatic Algorithm Configuration Framework , 2009, J. Artif. Intell. Res..

[15]  Cynthia Rudin,et al.  The P-Norm Push: A Simple Convex Ranking Algorithm that Concentrates at the Top of the List , 2009, J. Mach. Learn. Res..

[16]  Timo Berthold,et al.  Hybrid Branching , 2009, CPAIOR.

[17]  Martin W. P. Savelsbergh,et al.  Information-based branching schemes for binary linear mixed integer problems , 2009, Math. Program. Comput..

[18]  Yuri Malitsky,et al.  ISAC - Instance-Specific Algorithm Configuration , 2010, ECAI.

[19]  F. Hutter,et al.  Hydra-MIP : Automated Algorithm Configuration and Selection for Mixed Integer Programming , 2011 .

[20]  Andrea Lodi,et al.  MIPLIB 2010 , 2011, Math. Program. Comput..

[21]  Tie-Yan Liu,et al.  Learning to Rank for Information Retrieval , 2011 .

[22]  Matteo Fischetti,et al.  Branching on nonchimerical fractionalities , 2012, Oper. Res. Lett..

[23]  Andrea Lodi,et al.  Performance Variability in Mixed-Integer Programming , 2013 .

[24]  Tobias Achterberg,et al.  Mixed Integer Programming: Analyzing 12 Years of Progress , 2013 .

[25]  He He,et al.  Learning to Search in Branch and Bound Algorithms , 2014, NIPS.

[26]  Louis Wehenkel,et al.  A Supervised Machine Learning Approach to Variable Branching in Branch-And-Bound , 2014 .

[27]  Yuri Malitsky,et al.  Instance-specific algorithm configuration , 2014, Constraints.

[28]  D. Bertsimas,et al.  Best Subset Selection via a Modern Optimization Lens , 2015, 1507.03133.

[29]  Louis Wehenkel,et al.  Machine Learning to Balance the Load in Parallel Branch-and-Bound , 2015 .