CapsAndRuns: An Improved Method for Approximately Optimal Algorithm Configuration

We consider the problem of configuring generalpurpose solvers to run efficiently on problem instances drawn from an unknown distribution, a problem of major interest in solver autoconfiguration. Following previous work, we focus on designing algorithms that find a configuration with near-optimal expected capped runtime while doing the least amount of work, with the cap chosen in a configuration-specific way so that most instances are solved. In this paper we present a new algorithm, CAPSANDRUNS, which finds a near-optimal configuration while using time that scales (in a problem dependent way) with the optimal expected capped runtime, significantly strengthening previous results which could only guarantee a bound that scaled with the potentially much larger optimal expected uncapped runtime. The new algorithm is simpler and more intuitive than the previous methods: first it estimates the optimal runtime cap for each configuration, then it uses a Bernstein race to find a near optimal configuration given the caps. Experiments verify that our method can significantly outperform its competitors.

[1]  Kevin Leyton-Brown,et al.  Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[2]  András György,et al.  Efficient Multi-Start Strategies for Local Search Algorithms , 2009, J. Artif. Intell. Res..

[3]  Kevin Leyton-Brown,et al.  Efficiency Through Procrastination: Approximately Optimal Algorithm Configuration with Runtime Guarantees , 2017, IJCAI.

[4]  Yuri Malitsky,et al.  Model-Based Genetic Algorithms for Algorithm Configuration , 2015, IJCAI.

[5]  Kevin Leyton-Brown,et al.  Bayesian Optimization With Censored Response Data , 2013, ArXiv.

[6]  Ameet Talwalkar,et al.  Efficient Hyperparameter Optimization and Infinitely Many Armed Bandits , 2016, ArXiv.

[7]  Thomas Stützle,et al.  A Racing Algorithm for Configuring Metaheuristics , 2002, GECCO.

[8]  András György,et al.  LEAPSANDBOUNDS: A Method for Approximately Optimal Algorithm Configuration , 2018, ICML.

[9]  Niklas Een,et al.  MiniSat v1.13 - A SAT Solver with Conflict-Clause Minimization , 2005 .

[10]  F. Hutter.  On the Potential of Automatic Algorithm Configuration , 2007 .

[11]  Klaus Adam.  Learning While Searching for the Best Alternative , 2001, J. Econ. Theory.

[12]  Po-Ling Loh,et al.  Faster Hoeffding Racing: Bernstein Races via Jackknife Estimates , 2013, ALT.

[13]  Csaba Szepesvari,et al.  Efficient Stopping Rules , 2008 .

[14]  Mauro Birattari,et al.  The irace Package: Iterated Race for Automatic Algorithm , 2011 .

[15]  Carlos Ansótegui,et al.  A Gender-Based Genetic Algorithm for the Automatic Configuration of Algorithms , 2009, CP.

[16]  David Zuckerman,et al.  Optimal speedup of Las Vegas algorithms , 1993, [1993] The 2nd Israel Symposium on Theory and Computing Systems.

[17]  Stefano Ermon,et al.  Best arm identification in multi-armed bandits with delayed feedback , 2018, AISTATS.

[18]  Csaba Szepesvári,et al.  Empirical Bernstein stopping , 2008, ICML '08.