On learning and branching: a survey

This paper surveys learning techniques to deal with the two most crucial decisions in the branch-and-bound algorithm for Mixed-Integer Linear Programming, namely variable and node selections. Because of the lack of deep mathematical understanding on those decisions, the classical and vast literature in the field is inherently based on computational studies and heuristic, often problem-specific, strategies. We will both interpret some of those early contributions in the light of modern (machine) learning techniques, and give the details of the recent algorithms that instead explicitly incorporate machine learning paradigms.

[1]  Cheng Soon Ong,et al.  Multivariate spearman's ρ for aggregating ranks using copulas , 2016 .

[2]  Giovanni Rinaldi,et al.  A Branch-and-Cut Algorithm for the Resolution of Large-Scale Symmetric Traveling Salesman Problems , 1991, SIAM Rev..

[3]  A. Land,et al.  An Automatic Method for Solving Discrete Programming Problems , 1960, 50 Years of Integer Programming.

[4]  H. Abdi,et al.  Principal component analysis , 2010 .

[5]  Bernd Bischl,et al.  mlr: Machine Learning in R , 2016, J. Mach. Learn. Res..

[6]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[7]  Andrea Lodi,et al.  MIPLIB 2010 , 2011, Math. Program. Comput..

[8]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[9]  Martin W. P. Savelsbergh,et al.  Information-based branching schemes for binary linear mixed integer problems , 2009, Math. Program. Comput..

[10]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[11]  Louis Wehenkel,et al.  Online Learning for Strong Branching Approximation in Branch-and-Bound , 2016 .

[12]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[13]  Tuomas Sandholm,et al.  Information-theoretic approaches to branching in search , 2006, AAMAS '06.

[14]  He He,et al.  Learning to Search in Branch and Bound Algorithms , 2014, NIPS.

[15]  Kevin Leyton-Brown,et al.  Algorithm runtime prediction: Methods & evaluation , 2012, Artif. Intell..

[16]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[17]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[18]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[19]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Yuri Malitsky,et al.  Non-Model-Based Search Guidance for Set Partitioning Problems , 2012, AAAI.

[22]  Martin W. P. Savelsbergh,et al.  A Computational Study of Search Strategies for Mixed Integer Programming , 1999, INFORMS J. Comput..

[23]  A. Schrijver,et al.  The Traveling Salesman Problem , 2011 .

[24]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[25]  Louis Wehenkel,et al.  Machine Learning to Balance the Load in Parallel Branch-and-Bound , 2015 .

[26]  Louis Wehenkel,et al.  A Machine Learning-Based Approximation of Strong Branching , 2017, INFORMS J. Comput..

[27]  Jeff T. Linderoth,et al.  Lookahead Branching for Mixed Integer Programming , 2011, ICS 2011.

[28]  Matteo Fischetti,et al.  Improving branch-and-cut performance by random sampling , 2016, Math. Program. Comput..

[29]  George L. Nemhauser,et al.  An abstract model for branching and its application to mixed integer programming , 2015, Math. Program..

[30]  Elias Boutros Khalil.  Machine Learning for Integer Programming , 2016, IJCAI.

[31]  Andrea Lodi.  The Heuristic (Dark) Side of MIP Solvers , 2013, Hybrid Metaheuristics.

[32]  William J. Cook,et al.  The Traveling Salesman Problem: A Computational Study , 2007 .

[33]  D. Knuth.  Estimating the efficiency of backtrack programs. , 1974 .

[34]  Matteo Fischetti,et al.  Exploiting Erraticism in Search , 2014, Oper. Res..

[35]  Gérard Cornuéjols,et al.  Early Estimates of the Size of Branch-and-Bound Trees , 2006, INFORMS J. Comput..

[36]  Greg Hamerly,et al.  Learning the k in k-means , 2003, NIPS.

[37]  Matteo Fischetti,et al.  Backdoor Branching , 2011, INFORMS J. Comput..

[38]  Thorsten Koch,et al.  Konrad-zuse-zentrum F ¨ Ur Informationstechnik Berlin Miplib 2003 , 2022 .

[39]  G. Ribiere,et al.  Experiments in mixed-integer linear programming , 1971, Math. Program..

[40]  Timo Berthold,et al.  Hybrid Branching , 2009, CPAIOR.

[41]  Ailsa H. Land,et al.  An Automatic Method of Solving Discrete Programming Problems , 1960 .

[42]  H. Robbins.  Some aspects of the sequential design of experiments , 1952 .

[43]  R. Gomory.  AN ALGORITHM FOR THE MIXED INTEGER PROBLEM , 1960 .

[44]  Csaba Szepesvári,et al.  Algorithms for Reinforcement Learning , 2010, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[45]  Alejandro Marcos Alvarez,et al.  Computational and Theoretical Synergies between Linear Optimization and Supervised Machine Learning , 2016 .

[46]  Andrea Lodi,et al.  Performance Variability in Mixed-Integer Programming , 2013 .

[47]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[48]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[49]  Pedro M. Domingos.  A few useful things to know about machine learning , 2012, Commun. ACM.

[50]  Le Song,et al.  Learning to Branch in Mixed Integer Programming , 2016, AAAI.

[51]  Carlos Ansótegui,et al.  A Gender-Based Genetic Algorithm for the Automatic Configuration of Algorithms , 2009, CP.

[52]  Matteo Fischetti,et al.  Branching on nonchimerical fractionalities , 2012, Oper. Res. Lett..

[53]  Ashish Sabharwal,et al.  Guiding Combinatorial Optimization with UCT , 2012, CPAIOR.

[54]  Andrea Lodi,et al.  Mixed Integer Programming Computation , 2010, 50 Years of Integer Programming.

[55]  Martin W. P. Savelsbergh,et al.  An Updated Mixed Integer Programming Library: MIPLIB 3.0 , 1998 .

[56]  Thorsten Koch,et al.  Branching rules revisited , 2005, Oper. Res. Lett..

[57]  Yuri Malitsky,et al.  DASH: Dynamic Approach for Switching Heuristics , 2013, Eur. J. Oper. Res..

[58]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[59]  R. Bixby.  An Updated Mixed Integer Programming Library MIPLIB , 1998 .

[60]  Louis Wehenkel,et al.  A Supervised Machine Learning Approach to Variable Branching in Branch-And-Bound , 2014 .

[61]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.