Comprehensive Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems Using the R-Package Flacco

Choosing the best-performing optimizer(s) out of a portfolio of optimization algorithms is usually a difficult and complex task. It gets even worse, if the underlying functions are unknown, i.e., so-called black-box problems, and function evaluations are considered to be expensive. In case of continuous single-objective optimization problems, exploratory landscape analysis (ELA), a sophisticated and effective approach for characterizing the landscapes of such problems by means of numerical values before actually performing the optimization task itself, is advantageous. Unfortunately, until now it has been quite complicated to compute multiple ELA features simultaneously, as the corresponding code has been—if at all—spread across multiple platforms or at least across several packages within these platforms. This article presents a broad summary of existing ELA approaches and introduces flacco, an R-package for feature-based landscape analysis of continuous and constrained optimization problems. Although its functions neither solve the optimization problem itself nor the related algorithm selection problem (ASP), it offers easy access to an essential ingredient of the ASP by providing a wide collection of ELA features on a single platform—even within a single package. In addition, flacco provides multiple visualization techniques, which enhance the understanding of some of these numerical features, and thereby make certain landscape properties more comprehensible. On top of that, we will introduce the package’s built-in, as well as web-hosted and hence platform-independent, graphical user interface (GUI). It facilitates the usage of the package—especially for people who are not familiar with R—and thus makes flacco a very convenient toolbox when working towards algorithm selection of continuous single-objective optimization problems.

[1]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[2]  Scott Chamberlain,et al.  Create Interactive Web Graphics via Plotly's JavaScript GraphingLibrary , 2015 .

[3]  Raymond Ros,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Experimental Setup , 2009 .

[4]  Christian L. Müller,et al.  Global Characterization of the CEC 2005 Fitness Landscapes Using Fitness-Distance Analysis , 2011, EvoApplications.

[5]  F. Kianifard Applied Multivariate Data Analysis: Volume II: Categorical and Multivariate Methods , 1994 .

[6]  Tim Jones Evolutionary Algorithms, Fitness Landscapes and Search , 1995 .

[7]  Weiqiang Dong On Bias , Variance , 0 / 1-Loss , and the Curse of Dimensionality RK April 13 , 2014 .

[8]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[9]  Saman K. Halgamuge,et al.  Exploratory Landscape Analysis of Continuous Space Optimization Problems Using Information Content , 2015, IEEE Transactions on Evolutionary Computation.

[10]  Yuri Malitsky,et al.  Features for Exploiting Black-Box Optimization Problem Structure , 2013, LION.

[11]  Josef Pihera,et al.  Application of Machine Learning to Algorithm Selection for TSP , 2014, 2014 IEEE 26th International Conference on Tools with Artificial Intelligence.

[12]  Terry Jones,et al.  Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms , 1995, ICGA.

[13]  Andries Petrus Engelbrecht,et al.  Characterising constrained continuous optimisation problems , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[14]  John R. Rice,et al.  The Algorithm Selection Problem , 1976, Adv. Comput..

[15]  Michael T. Wolfinger,et al.  Barrier Trees of Degenerate Landscapes , 2002 .

[16]  Bernd Bischl,et al.  Algorithm selection based on exploratory landscape analysis and cost-sensitive learning , 2012, GECCO '12.

[17]  Jakob Bossek,et al.  smoof: Single- and Multi-Objective Optimization Test Functions , 2017, R J..

[18]  Bernd Bischl,et al.  Exploratory landscape analysis , 2011, GECCO '11.

[19]  Bernd Bischl,et al.  mlr: Machine Learning in R , 2016, J. Mach. Learn. Res..

[20]  Bernd Bischl,et al.  A novel feature-based approach to characterize algorithm performance for the traveling salesperson problem , 2012, Annals of Mathematics and Artificial Intelligence.

[21]  Marcus Gallagher,et al.  Analysing and characterising optimization problems using length scale , 2017, Soft Comput..

[22]  Bernd Bischl,et al.  Cell Mapping Techniques for Exploratory Landscape Analysis , 2014 .

[23]  Kiyoshi Tanaka,et al.  Problem features vs. algorithm performance on rugged multi-objective combinatorial fitness landscapes , 2017, SEVO.

[24]  Ramana V. Grandhi,et al.  Improved Distributed Hypercube Sampling , 2002 .

[25]  Jj Allaire,et al.  Web Application Framework for R , 2016 .

[26]  Sébastien Vérel,et al.  Local Optima Networks: A New Model of Combinatorial Fitness Landscapes , 2014, ArXiv.

[27]  Heike Trautmann,et al.  Detecting Funnel Structures by Means of Exploratory Landscape Analysis , 2015, GECCO.

[28]  Heike Trautmann,et al.  The R-Package FLACCO for exploratory landscape analysis with applications to multi-objective optimization problems , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[29]  L. Darrell Whitley,et al.  The dispersion metric and the CMA evolution strategy , 2006, GECCO.

[30]  Tomoharu Nagao,et al.  Bag of local landscape features for fitness landscape analysis , 2016, Soft Comput..

[31]  Mario A. Muñoz,et al.  Landscape characterization of numerical optimization problems using biased scattered data , 2012, 2012 IEEE Congress on Evolutionary Computation.

[32]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[33]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[34]  Sébastien Vérel,et al.  Problem Features versus Algorithm Performance on Rugged Multiobjective Combinatorial Fitness Landscapes , 2016, Evolutionary Computation.

[35]  Andries Petrus Engelbrecht,et al.  Quantifying ruggedness of continuous landscapes using entropy , 2009, 2009 IEEE Congress on Evolutionary Computation.

[36]  Kevin Leyton-Brown,et al.  Algorithm runtime prediction: Methods & evaluation , 2012, Artif. Intell..

[37]  Mario A. Muñoz,et al.  Algorithm selection for black-box continuous optimization problems: A survey on methods and challenges , 2015, Inf. Sci..

[38]  Heike Trautmann,et al.  Low-Budget Exploratory Landscape Analysis on Multiple Peaks Models , 2016, GECCO.

[39]  Pascal Kerschke,et al.  flaccogui: exploratory landscape analysis for everyone , 2017, GECCO.