Optimization of algorithms with OPAL

Opal is a general-purpose system for modeling and solving algorithm optimization problems. Opal takes an algorithm as input, and as output it suggests parameter values that maximize some user-defined performance measure. In order to achieve this, the user provides a Python script describing how to launch the target algorithm, and defining the performance measure. Opal then models this question as a blackbox optimization problem which is then solved by a state-of-the-art direct search solver. Opal handles a wide variety of parameter types, it can exploit multiple processors in parallel at different levels, and can take advantage of a surrogate blackbox. Several features of Opal are illustrated on a problem consisting in the design of a hybrid sort strategy.

[1]  James Demmel,et al.  The PHiPAC v1.0 Matrix-Multiply Distribution , 1998 .

[2]  Charles Audet,et al.  Algorithmic Parameter Optimization of the DFO Method with the OPAL Framework , 2010, Software Automatic Tuning, From Concepts to State-of-the-Art Results.

[3]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[4]  Charles Audet,et al.  Multiobjective Optimization Through a Series of Single-Objective Formulations , 2008, SIAM J. Optim..

[5]  J. Dennis,et al.  Filter Pattern Search Algorithms for Mixed Variable Constrained Optimization Problems , 2004 .

[6]  CHARLES AUDET,et al.  Finding Optimal Algorithmic Parameters Using Derivative-Free Optimization , 2006, SIAM J. Optim..

[7]  Panu Somervuo,et al.  How to make large self-organizing maps for nonvectorial data , 2002, Neural Networks.

[8]  Sébastien Le Digabel,et al.  The mesh adaptive direct search algorithm with treed Gaussian process surrogates , 2011 .

[9]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[10]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[11]  Luís N. Vicente,et al.  Analysis of direct searches for discontinuous functions , 2012, Math. Program..

[12]  Charles Audet,et al.  Efficient use of parallelism in algorithmic parameter optimization applications , 2013, Optim. Lett..

[13]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[14]  Sébastien Le Digabel,et al.  Algorithm xxx : NOMAD : Nonlinear Optimization with the MADS algorithm , 2010 .

[15]  O. SIAMJ.,et al.  ON THE CONVERGENCE OF PATTERN SEARCH ALGORITHMS , 1997 .

[16]  Charles Audet,et al.  Analysis of Generalized Pattern Searches , 2000, SIAM J. Optim..

[17]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[18]  George E. P. Box,et al.  Evolutionary Operation: a Method for Increasing Industrial Productivity , 1957 .

[19]  Jack J. Dongarra,et al.  Automated empirical optimizations of software and the ATLAS project , 2001, Parallel Comput..

[20]  Charles Audet,et al.  Trade-off studies in blackbox optimization , 2012, Optim. Methods Softw..

[21]  Pierre Hansen,et al.  Variable Neighborhood Search , 2018, Handbook of Heuristics.

[22]  Nicholas I. M. Gould,et al.  Sensitivity of trust-region algorithms to their parameters , 2005, 4OR.

[23]  Charles Audet,et al.  A MADS Algorithm with a Progressive Barrier for Derivative-Free Nonlinear Programming , 2007 .

[24]  Yuri Kagolovsky,et al.  Cluster analysis of Wisconsin Breast Cancer dataset using self-organizing maps. , 2002, Studies in health technology and informatics.

[25]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[26]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[27]  Katherine Yelick,et al.  OSKI: A library of automatically tuned sparse matrix kernels , 2005 .

[28]  John L. Nazareth,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[29]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[30]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[31]  Yuefan Deng,et al.  New trends in high performance computing , 2001, Parallel Computing.

[32]  Dominique Orban,et al.  Templating and Automatic Code Generation for Performance with Python , 2011 .

[33]  Charles Audet,et al.  Nonsmooth optimization through Mesh Adaptive Direct Search and Variable Neighborhood Search , 2006, J. Glob. Optim..

[34]  Charles Audet,et al.  Optimization of algorithms with the opal framework , 2012 .

[35]  J. Dennis,et al.  Mixed Variable Optimization of the Number and Composition of Heat Intercepts in a Thermal Insulation System , 2001 .

[36]  Charles Audet,et al.  Convergence of Mesh Adaptive Direct Search to Second-Order Stationary Points , 2006, SIAM J. Optim..

[37]  Jack J. Dongarra,et al.  A comparison of search heuristics for empirical code optimization , 2008, 2008 IEEE International Conference on Cluster Computing.

[38]  Alan Edelman,et al.  PetaBricks: a language and compiler for algorithmic choice , 2009, PLDI '09.

[39]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[40]  Charles Audet,et al.  Globalization strategies for Mesh Adaptive Direct Search , 2008, Comput. Optim. Appl..

[41]  Sébastien Le Digabel,et al.  Use of quadratic models with mesh-adaptive direct search for constrained black box optimization , 2011, Optim. Methods Softw..