Modern Continuous Optimization Algorithms for Tuning Real and Integer Algorithm Parameters

To obtain peak performance from optimization algorithms, it is required to set appropriately their parameters. Frequently, algorithm parameters can take values from the set of real numbers, or from a large integer set. To tune this kind of parameters, it is interesting to apply state-of-the-art continuous optimization algorithms instead of using a tedious, and error-prone, hands-on approach. In this paper, we study the performance of several continuous optimization algorithms for the algorithm parameter tuning task. As case studies, we use a number of optimization algorithms from the swarm intelligence literature.

[1]  Anne Auger,et al.  Experimental Comparisons of Derivative Free Optimization Algorithms , 2009, SEA.

[2]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[3]  Mauro Birattari,et al.  The problem of tuning metaheuristics: as seen from the machine learning perspective , 2004 .

[4]  Roberto Rossi,et al.  Synthesizing Filtering Algorithms for Global Chance-Constraints , 2009, CP.

[5]  Nicolás García-Pedrajas,et al.  Trends in Applied Intelligent Systems - 23rd International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2010, Cordoba, Spain, June 1-4, 2010, Proceedings, Part I , 2010, IEA/AIE.

[6]  Mauro Birattari,et al.  Tuning Metaheuristics - A Machine Learning Perspective , 2009, Studies in Computational Intelligence.

[7]  Thomas Stützle,et al.  MADS/F-Race: Mesh Adaptive Direct Search Meets F-Race , 2010, IEA/AIE.

[8]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[9]  Carlos Ansótegui,et al.  A Gender-Based Genetic Algorithm for the Automatic Configuration of Algorithms , 2009, CP.

[10]  Alex S. Fukunaga,et al.  Automated Discovery of Local Search Heuristics for Satisfiability Testing , 2008, Evolutionary Computation.

[11]  Thomas Stützle,et al.  F-Race and Iterated F-Race: An Overview , 2010, Experimental Methods for the Analysis of Optimization Algorithms.

[12]  Thomas Bartz-Beielstein,et al.  Experimental Research in Evolutionary Computation - The New Experimentalism , 2010, Natural Computing Series.

[13]  E. D. Taillard,et al.  Ant Systems , 1999 .

[14]  Chuan-Kang Ting,et al.  Varying Number of Difference Vectors in Differential Evolution , 2009, 2009 IEEE Congress on Evolutionary Computation.

[15]  A. E. Eiben,et al.  Efficient relevance estimation and value calibration of evolutionary algorithm parameters , 2007, 2007 IEEE Congress on Evolutionary Computation.

[16]  Pedro Larrañaga,et al.  Towards a New Evolutionary Computation - Advances in the Estimation of Distribution Algorithms , 2006, Towards a New Evolutionary Computation.

[17]  Thomas Stützle,et al.  Ant Colony Optimization Theory , 2004 .

[18]  Robert E. Mercer,et al.  ADAPTIVE SEARCH USING A REPRODUCTIVE META‐PLAN , 1978 .

[19]  Charles Audet,et al.  Erratum: Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[20]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[21]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[22]  Thomas Stützle,et al.  Improvement Strategies for the F-Race Algorithm: Sampling Design and Iterative Refinement , 2007, Hybrid Metaheuristics.

[23]  Mihai Oltean,et al.  Evolving Evolutionary Algorithms Using Linear Genetic Programming , 2005, Evolutionary Computation.

[24]  Kevin P. Murphy,et al.  An experimental investigation of model-based parameter optimisation: SPO and beyond , 2009, GECCO.

[25]  M. Powell.  The BOBYQA algorithm for bound constrained optimization without derivatives , 2009 .

[26]  Manuel Laguna,et al.  Fine-Tuning of Algorithms Using Fractional Experimental Designs and Local Search , 2006, Oper. Res..

[27]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[28]  Thomas Bartz-Beielstein,et al.  Experimental Methods for the Analysis of Optimization Algorithms , 2010 .

[29]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[30]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[31]  M. Powell.  The NEWUOA software for unconstrained optimization without derivatives , 2006 .

[32]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[33]  Thomas Stützle,et al.  MAX-MIN Ant System , 2000, Future Gener. Comput. Syst..

[34]  Terry Jones,et al.  Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms , 1995, ICGA.