F-Race and Iterated F-Race: An Overview

Algorithms for solving hard optimization problems typically have several parameters that need to be set appropriately such that some aspect of performance is optimized. In this chapter, we review F-Race, a racing algorithm for the task of automatic algorithm configuration. F-Race is based on a statistical approach for selecting the best configuration out of a set of candidate configurations under stochastic evaluations. We review the ideas underlying this technique and discuss an extension of the initial F-Race algorithm, which leads to a family of algorithms that we call iterated F-Race. Experimental results comparing one specific implementation of iterated F-Race to the original F-Race algorithm confirm the potential of this family of algorithms.

[1]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[2]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[3]  A. Stuart,et al.  Non-Parametric Statistics for the Behavioral Sciences. , 1957 .

[4]  P. Billingsley,et al.  Probability and Measure , 1980 .

[5]  T. Obremski.  Practical Nonparametric Statistics (2nd ed.) , 1981 .

[6]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[7]  Patrick Billingsley,et al.  Probability and Measure. , 1986 .

[8]  Shirley Dex,et al.  JR 旅客販売総合システム（マルス）における運用及び管理について , 1991 .

[9]  Andrew W. Moore,et al.  Hoeffding Races: Accelerating Model Selection Search for Classification and Function Approximation , 1993, NIPS.

[10]  Oded Maron,et al.  Hoeffding Races--model selection for MRI classification , 1994 .

[11]  Alexander J. Smola,et al.  Neural Information Processing Systems , 1997, NIPS 1997.

[12]  David J. Groggel,et al.  Practical Nonparametric Statistics , 2000, Technometrics.

[13]  Thomas Stützle,et al.  MAX-MIN Ant System , 2000, Future Gener. Comput. Syst..

[14]  Maliha S. Nash,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 2001, Technometrics.

[15]  Ben Paechter,et al.  A Comparison of the Performance of Different Metaheuristics on the Timetabling Problem , 2002, PATAT.

[16]  Thomas Stützle,et al.  A Racing Algorithm for Configuring Metaheuristics , 2002, GECCO.

[17]  Edmund K. Burke,et al.  Practice and Theory of Automated Timetabling IV , 2002, Lecture Notes in Computer Science.

[18]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[19]  T. Stützle,et al.  An algorithm for the car sequencing problem of the ROADEF 2005 Challenge , 2004 .

[20]  Handbook of Parametric and Nonparametric Statistical Procedures , 2004 .

[21]  Thomas Stützle,et al.  The linear ordering problem: Instances, search space analysis and algorithms , 2004, J. Math. Model. Algorithms.

[22]  Mauro Birattari,et al.  The problem of tuning metaheuristics: as seen from the machine learning perspective , 2004 .

[23]  Matthijs Leendert den Besten,et al.  Simple metaheuristics for scheduling: an empirical investigation into the application of iterated local search to deterministic scheduling problems with tardiness penalties , 2004 .

[24]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[25]  M. Birattari,et al.  Artificielle On the Estimation of the Expected Performance of a Metaheuristic on a Class of Instances How many instances , how many runs ? , 2004 .

[26]  Mauro Birattari,et al.  Model-Based Search for Combinatorial Optimization: A Critical Survey , 2004, Ann. Oper. Res..

[27]  Marcus Gallagher,et al.  Statistical Racing Techniques for Improved Empirical Evaluation of Evolutionary Algorithms , 2004, PPSN.

[28]  Marco Chiarandini,et al.  Experimental Evaluation of Course Timetabling Algorithms , 2004 .

[29]  Andrew W. Moore,et al.  The Racing Algorithm: Model Selection for Lazy Learners , 1997, Artificial Intelligence Review.

[30]  M. Besten.  Simple metaheuristics for scheduling: an empirical investigation into the application of iterated local search to deterministic scheduling problems with tardiness penalties , 2005 .

[31]  Marco Chiarandini,et al.  Stochastic local search methods for highly constrained combinatorial optimisation problems: graph colouring, generalisations, and applications , 2005 .

[32]  Marcus Gallagher,et al.  A hybrid approach to parameter tuning in genetic algorithms , 2005, 2005 IEEE Congress on Evolutionary Computation.

[33]  Paola Pellegrini.  Application of Two Nearest Neighbor Approaches to a Rich Vehicle Routing Problem , 2005 .

[34]  Christian Blum,et al.  Training feed-forward neural networks with ant colony optimization: an application to pattern classification , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[35]  Thomas Stützle,et al.  Applications of Racing Algorithms: An Industrial Perspective , 2005, Artificial Evolution.

[36]  Gianluca Bontempi,et al.  How to allocate a restricted budget of leave-one-out assessments for effective model selection in machine learning: a comparison of state-of-the-art techniques , 2005, BNAIC.

[37]  Manuel Laguna,et al.  Fine-Tuning of Algorithms Using Fractional Experimental Designs and Local Search , 2006, Oper. Res..

[38]  Mauro Birattari,et al.  An effective hybrid algorithm for university course timetabling , 2006, J. Sched..

[39]  Marco Dorigo,et al.  Path formation in a robot swarm , 2008, Swarm Intelligence.

[40]  Prasanna Balaprakash,et al.  The ACO/F-Race Algorithm for Combinatorial Optimization Under Uncertainty , 2007, Metaheuristics.

[41]  Michel Gendreau,et al.  Metaheuristics: Progress in Complex Systems Optimization , 2007 .

[42]  Marcus Gallagher,et al.  Combining Meta-EAs and Racing for Difficult EA Parameter Tuning Tasks , 2007, Parameter Setting in Evolutionary Algorithms.

[43]  Thomas Stützle,et al.  A study of stochastic local search algorithms for the quadratic assignment problem , 2007 .

[44]  Thomas Bartz-Beielstein,et al.  Experimental research in evolutionary computation , 2007, GECCO '07.

[45]  Christian Blum,et al.  An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training , 2007, Neural Computing and Applications.

[46]  Thomas Stützle,et al.  Improvement Strategies for the F-Race Algorithm: Sampling Design and Iterative Refinement , 2007, Hybrid Metaheuristics.

[47]  Andrea Roli,et al.  Hybrid Local Search for Constrained Financial Portfolio Selection Problems , 2007, CPAIOR.

[48]  Thomas Stützle,et al.  Stochastic Local Search Algorithms for Graph Set T -colouring and Frequency Assignment , 2005 .

[49]  Thomas Stützle,et al.  Automatic Algorithm Configuration Based on Local Search , 2007, AAAI.

[50]  Marco Dorigo,et al.  Teamwork in a swarm of robots: an experiment in search and retrieval , 2008 .

[51]  H. Bersini,et al.  The Gestalt heuristic : learning the right level of abstraction to better search the optima , 2008 .

[52]  Thomas Stützle,et al.  Reactive Stochastic Local Search Algorithms for the Genomic Median Problem , 2007, EvoCOP.

[53]  Andrea Roli,et al.  Stochastic local search for large-scale instances of the haplotype inference problem by pure parsimony , 2008, J. Algorithms.

[54]  Thomas Stützle,et al.  Iterated Greedy Algorithms for a Real-World Cyclic Train Scheduling Problem , 2008, Hybrid Metaheuristics.

[55]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[56]  Thomas Stützle,et al.  Estimation-based ant colony optimization and local search for the probabilistic traveling salesman problem , 2009, Swarm Intelligence.

[57]  Mauro Birattari,et al.  Tuning Metaheuristics - A Machine Learning Perspective , 2009, Studies in Computational Intelligence.

[58]  Prasanna Balaprakash,et al.  Adaptive sample size and importance sampling in estimation-based local search for the probabilistic traveling salesman problem , 2009, Eur. J. Oper. Res..

[59]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.

[60]  Mario Cortina-Borja,et al.  Handbook of Parametric and Nonparametric Statistical Procedures, 5th edn , 2012 .