Computer-Aided Design of High-Performance Algorithms

High-performance algorithms play an important role in many areas of computer science and are core components of many software systems used in real-world applications. Traditionally, the creation of these algorithms requires considerable expertise and experience, often in combination with a substantial amount of trial and error. Here, we outline a new approach to the process of designing high-performance algorithms that is based on the use of automated procedures for exploring potentially very large spaces of candidate designs. We contrast this computer-aided design approach with the traditional approach and discuss why it can be expected to yield better performing, yet simpler algorithms. Finally, we sketch out the high-level design of a software environment that supports our new design approach. Existing work on algorithm portfolios, algorithm selection, algorithm configuration and parameter tuning, but also on general methods for discrete and continuous optimisation methods fits naturally into our design approach and can be integrated into the proposed software

[1]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[2]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[3]  John R. Rice,et al.  The Algorithm Selection Problem , 1976, Adv. Comput..

[4]  Jon Louis Bentley,et al.  Engineering a sort function , 1993, Softw. Pract. Exp..

[5]  Sebastian Thrun,et al.  Explanation-based neural network learning a lifelong learning approach , 1995 .

[6]  Ulrich W. Thonemann,et al.  Optimizing simulated annealing schedules with genetic programming , 1996 .

[7]  Roberto Battiti,et al.  Reactive Search, a history-based heuristic for MAX-SAT , 1996 .

[8]  Myra Spiliopoulou,et al.  Genetic programming in database query optimization , 1996 .

[9]  Tad Hogg,et al.  An Economics Approach to Hard Computational Problems , 1997, Science.

[10]  M. J. D. Powell,et al.  Direct search algorithms for optimization calculations , 1998, Acta Numerica.

[11]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[12]  Holger H. Hoos,et al.  Stochastic local search - methods, models, applications , 1998, DISKI.

[13]  Alex Fukunaga,et al.  Genetic algorithm portfolios , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[14]  Douglas R. Smith,et al.  Synthesis of efficient constraint-satisfaction programs , 2001, The Knowledge Engineering Review.

[15]  Bart Selman,et al.  Algorithm portfolios , 2001, Artif. Intell..

[16]  Jack J. Dongarra,et al.  Automated empirical optimizations of software and the ATLAS project , 2001, Parallel Comput..

[17]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[18]  Graham Kendall,et al.  Hyperheuristics: A Tool for Rapid Prototyping in Scheduling and Optimisation , 2002, EvoWorkshops.

[19]  Thomas Stützle,et al.  A Racing Algorithm for Configuring Metaheuristics , 2002, GECCO.

[20]  Alex S. Fukunaga,et al.  Automated discovery of composite SAT variable-selection heuristics , 2002, AAAI/IAAI.

[21]  Graham Kendall,et al.  Hyper-Heuristics: An Emerging Direction in Modern Search Technology , 2003, Handbook of Metaheuristics.

[22]  Ivana Kruijff-Korbayová,et al.  A Portfolio Approach to Algorithm Selection , 2003, IJCAI.

[23]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[24]  Michela Milano,et al.  Learning Techniques for Automatic Algorithm Portfolio Selection , 2004, ECAI.

[25]  Alex S. Fukunaga,et al.  Evolving Local Search Heuristics for SAT Using Genetic Programming , 2004, GECCO.

[26]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[27]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[28]  David A. Padua,et al.  Optimizing sorting with genetic algorithms , 2005, International Symposium on Code Generation and Optimization.

[29]  J. Christopher Beck,et al.  APPLYING MACHINE LEARNING TO LOW‐KNOWLEDGE CONTROL OF OPTIMIZATION ALGORITHMS , 2005, Comput. Intell..

[30]  Thomas Bartz-Beielstein,et al.  Experimental Research in Evolutionary Computation - The New Experimentalism , 2010, Natural Computing Series.

[31]  Rudolf Eigenmann,et al.  Fast and effective orchestration of compiler optimizations for automatic performance tuning , 2006, International Symposium on Code Generation and Optimization (CGO'06).

[32]  Manuel Laguna,et al.  Fine-Tuning of Algorithms Using Fractional Experimental Designs and Local Search , 2006, Oper. Res..

[33]  Jürgen Schmidhuber,et al.  Dynamic Algorithm Portfolios , 2006, AI&M.

[34]  Janez Puhan,et al.  Grid Restrained Nelder-Mead Algorithm , 2006, Comput. Optim. Appl..

[35]  Pascal Van Hentenryck,et al.  Synthesis of Constraint-Based Local Search Algorithms from High-Level Models , 2007, AAAI.

[36]  Kevin Leyton-Brown,et al.  : The Design and Analysis of an Algorithm Portfolio for SAT , 2007, CP.

[37]  Michael F. P. O'Boyle,et al.  Rapidly Selecting Good Compiler Optimizations using Performance Counters , 2007, International Symposium on Code Generation and Optimization (CGO'07).

[38]  Luca Di Gaspero,et al.  EasySyn++: A Tool for Automatic Synthesis of Stochastic Local Search Algorithms , 2007, SLS.

[39]  Peter Sanders,et al.  Engineering Fast Route Planning Algorithms , 2007, WEA.

[40]  Peter Sanders,et al.  Engineering Algorithms for Approximate Weighted Matching , 2007, WEA.

[41]  Thomas Stützle,et al.  Improvement Strategies for the F-Race Algorithm: Sampling Design and Iterative Refinement , 2007, Hybrid Metaheuristics.

[42]  Thomas Stützle,et al.  Automatic Algorithm Configuration Based on Local Search , 2007, AAAI.

[43]  Hongbin Dong,et al.  Genetic algorithms for large join query optimization , 2007, GECCO '07.

[44]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[45]  Kevin Leyton-Brown,et al.  SATzilla: Portfolio-based Algorithm Selection for SAT , 2008, J. Artif. Intell. Res..

[46]  Gerth Stølting Brodal,et al.  Engineering a cache-oblivious sorting algorithm , 2008, JEAL.

[47]  Mauro Brunato,et al.  Reactive Search and Intelligent Optimization , 2008 .

[48]  F. Hutter,et al.  ParamILS: An Automatic Algorithm Configuration Framework , 2009, J. Artif. Intell. Res..

[49]  Pascal Van Hentenryck,et al.  Aeon: Synthesizing Scheduling Algorithms from High-Level Models , 2009 .