Learning to Configure Mathematical Programming Solvers by Mathematical Programming

We discuss the issue of finding a good mathematical programming solver configuration for a particular instance of a given problem, and we propose a two-phase approach to solve it. In the first phase we learn the relationships between the instance, the configuration and the performance of the configured solver on the given instance. A specific difficulty of learning a good solver configuration is that parameter settings may not all be independent; this requires enforcing (hard) constraints, something that many widely used supervised learning methods cannot natively achieve. We tackle this issue in the second phase of our approach, where we use the learnt information to construct and solve an optimization problem having an explicit representation of the dependency/consistency constraints on the configuration parameter settings. We discuss computational results for two different instantiations of this approach on a unit commitment problem arising in the short-term planning of hydro valleys. We use logistic regression as the supervised learning methodology and consider CPLEX as the solver of interest.

[1]  D. Cox The Regression Analysis of Binary Sequences , 1958 .

[2]  Milind Tambe,et al.  MIPaaL: Mixed Integer Program as a Layer , 2019, AAAI.

[3]  Mauro Brunato,et al.  Reactive Search: Machine Learning for Memory-Based Heuristics , 2005, Handbook of Approximation Algorithms and Metaheuristics.

[4]  Milind Tambe,et al.  Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization , 2018, AAAI.

[5]  A. Borghetti,et al.  An MILP Approach for Short-Term Hydro Scheduling and Unit Commitment With Head-Dependent Reservoir , 2008, IEEE Transactions on Power Systems.

[6]  Andrea Lodi,et al.  Performance Variability in Mixed-Integer Programming , 2013 .

[7]  Benjamin Doerr,et al.  Optimal Parameter Choices via Precise Black-Box Analysis , 2016, GECCO.

[8]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[9]  R. Geoff Dromey,et al.  An algorithm for the selection problem , 1986, Softw. Pract. Exp..

[10]  Michael I. Jordan Why the logistic function? A tutorial discussion on probabilities and neural networks , 1995 .

[11]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[12]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[13]  Marius Lindauer,et al.  Pitfalls and Best Practices in Algorithm Configuration , 2017, J. Artif. Intell. Res..

[14]  Leslie Pérez Cáceres,et al.  The irace package: Iterated racing for automatic algorithm configuration , 2016 .

[15]  J. Zico Kolter,et al.  OptNet: Differentiable Optimization as a Layer in Neural Networks , 2017, ICML.

[16]  Pierre-Louis Poirion,et al.  Algorithms and applications for a class of bilevel MILPs , 2020, Discret. Appl. Math..

[17]  Teofilo F. Gonzalez,et al.  Reactive Search: Machine Learning for Memory-Based Heuristics , 2007 .

[18]  Kevin Leyton-Brown,et al.  Automated Configuration of Mixed Integer Programming Solvers , 2010, CPAIOR.

[19]  Michèle Sebag,et al.  Alors: An algorithm recommender system , 2017, Artif. Intell..

[20]  John N. Hooker,et al.  Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems: 10th International Conference, CPAIOR 2013, Yorktown Heights, NY, USA, May 18-22, 2013. Proceedings , 2013, CPAIOR.

[21]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[22]  Holger H. Hoos,et al.  Programming by optimization , 2012, Commun. ACM.

[23]  W. Vach,et al.  Neural networks and logistic regression: Part I , 1996 .