Can Single Solution Optimisation Methods Be Structurally Biased?

This paper investigates whether optimisation methods with the population made up of one solution can suffer from structural bias just like their multisolution variants. Following recent results highlighting the importance of choice of strategy for handling solutions generated outside the domain, a selection of single solution methods are considered in conjunction with several such strategies. Obtained results are tested for the presence of structural bias by means of a traditional approach from literature and a newly proposed here statistical approach. These two tests are demonstrated to be not fully consistent. All tested methods are found to be structurally biased with at least one of the tested strategies. Confirming results for multisolution methods, it is such strategy that is shown to control the emergence of structural bias in single solution methods. Some of the tested methods exhibit a kind of structural bias that has not been observed before.

[1]  Xinchao Zhao,et al.  Simulated annealing algorithm with adaptive neighborhood , 2011, Appl. Soft Comput..

[2]  Christian Igel,et al.  A computational efficient covariance matrix update and a (1+1)-CMA for evolution strategies , 2006, GECCO.

[3]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[4]  J. Shaffer.  Multiple Hypothesis Testing , 1995 .

[5]  M. J. D. Powell,et al.  An efficient method for finding the minimum of a function of several variables without calculating derivatives , 1964, Comput. J..

[6]  Edmund K. Burke,et al.  A Separability Prototype for Automatic Memes with Adaptive Operator Selection , 2014, 2014 IEEE Symposium on Foundations of Computational Intelligence (FOCI).

[7]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[8]  Roger J.-B. Wets,et al.  Minimization by Random Search Techniques , 1981, Math. Oper. Res..

[9]  Fabio Caraffini,et al.  Structural Bias in Optimisation Algorithms: Extended Results , 2020 .

[10]  Adam P. Piotrowski,et al.  Some metaheuristics should be simplified , 2018, Inf. Sci..

[11]  Ji Zhen.  A Novel Intelligent Single Particle Optimizer , 2010 .

[12]  Fabio Caraffini.  The SOS Platform: Designing, Tuning and Statistically Benchmarking Optimisation Algorithms , 2020, Mathematics.

[13]  Ralph B. D'Agostino,et al.  Goodness-of-Fit-Techniques , 2020 .

[14]  David W. Corne,et al.  Structural bias in population-based algorithms , 2014, Inf. Sci..

[15]  D. Anderson,et al.  Algorithms for minimization without derivatives , 1974 .

[16]  Adam P. Piotrowski,et al.  Across Neighborhood Search algorithm: A comprehensive analysis , 2018, Inf. Sci..

[17]  J. R. Palmer.  An Improved Procedure for Orthogonalising the Search Vectors in Rosenbrock's and Swann's Direct Search Optimisation Methods , 1969, Comput. J..

[18]  Adam Prügel-Bennett,et al.  Benefits of a Population: Five Mechanisms That Advantage Population-Based Algorithms , 2010, IEEE Transactions on Evolutionary Computation.

[19]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[20]  J. Spall.  A Stochastic Approximation Technique for Generating Maximum Likelihood Parameter Estimates , 1987, 1987 American Control Conference.

[21]  Giovanni Iacca,et al.  Single particle algorithms for continuous optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[22]  Giovanni Iacca,et al.  Re-sampled inheritance search: high performance despite the simplicity , 2013, Soft Comput..

[23]  Günter Rudolph,et al.  Contemporary Evolution Strategies , 1995, ECAL.

[24]  David H. Wolpert,et al.  Coevolutionary free lunches , 2005, IEEE Transactions on Evolutionary Computation.

[25]  Giovanni Iacca,et al.  Improving (1+1) covariance matrix adaptation evolution strategy: a simple yet efficient approach , 2019 .

[26]  Anne Auger,et al.  Benchmarking the (1+1) evolution strategy with one-fifth success rule on the BBOB-2009 function testbed , 2009, GECCO '09.

[27]  Adam P. Piotrowski,et al.  Searching for structural bias in particle swarm optimization and differential evolution algorithms , 2016, Swarm Intelligence.

[28]  Francisco Herrera,et al.  MA-SW-Chains: Memetic algorithm based on local search chains for large scale continuous global optimization , 2010, IEEE Congress on Evolutionary Computation.

[29]  Adam P. Piotrowski,et al.  Step-by-step improvement of JADE and SHADE-based algorithms: Success or failure? , 2018, Swarm Evol. Comput..

[30]  Fabio Caraffini,et al.  Structural bias in differential evolution: A preliminary study , 2019 .

[31]  Ferrante Neri,et al.  Differential Evolution with Scale Factor Local Search for Large Scale Problems , 2010 .

[32]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[33]  Y. Benjamini.  Discovering the false discovery rate , 2010 .

[34]  J. Spall.  Implementation of the simultaneous perturbation algorithm for stochastic optimization , 1998 .

[35]  David W. Corne,et al.  Infeasibility and structural bias in Differential Evolution , 2019, Inf. Sci..

[36]  Alden H. Wright,et al.  Emergent Behaviour, Population-based Search and Low-pass Filtering , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[37]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..

[38]  Alfred Inselberg,et al.  The plane with parallel coordinates , 1985, The Visual Computer.

[39]  Warren Hare,et al.  Best practices for comparing optimization algorithms , 2017, Optimization and Engineering.

[40]  Michael G. Epitropakis,et al.  HyperSPAM: A study on hyper-heuristic coordination strategies in the continuous domain , 2019, Inf. Sci..

[41]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[42]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[43]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[44]  Fabio Caraffini,et al.  The Stochastic Optimisation Software (SOS) platform , 2019 .