Principled Design of Translation, Scale, and Rotation Invariant Variation Operators for Metaheuristics

In the past three decades, a large number of metaheuristics have been proposed and shown high performance in solving complex optimization problems. While most variation operators in existing metaheuristics are empirically designed, this paper aims to design new operators automatically, which are expected to be search space independent and thus exhibit robust performance on different problems. For this purpose, this work first investigates the influence of translation invariance, scale invariance, and rotation invariance on the search behavior and performance of some representative operators. Then, we deduce the generic form of translation, scale, and rotation invariant operators. Afterwards, a principled approach is proposed for the automated design of operators, which searches for highperformance operators based on the deduced generic form. The experimental results demonstrate that the operators generated by the proposed approach outperform state-of-theart ones on a variety of problems with complex landscapes and up to 1000 decision variables.

[1]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[2]  Stefan Roth,et al.  Covariance Matrix Adaptation for Multi-objective Optimization , 2007, Evolutionary Computation.

[3]  Xingyi Zhang,et al.  A Recommender System for Metaheuristic Algorithms for Continuous Optimization Based on Deep Recurrent Neural Networks , 2020, IEEE Transactions on Artificial Intelligence.

[4]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[5]  Fabio Caraffini,et al.  A study on rotation invariance in differential evolution , 2019, Swarm Evol. Comput..

[6]  Qingfu Zhang,et al.  Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II , 2009, IEEE Transactions on Evolutionary Computation.

[7]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[8]  Yaochu Jin,et al.  A Competitive Swarm Optimizer for Large Scale Optimization , 2015, IEEE Transactions on Cybernetics.

[9]  Samy Bengio,et al.  The Vehicle Routing Problem with Time Windows Part II: Genetic Search , 1996, INFORMS J. Comput..

[10]  Alex S. Fukunaga,et al.  Success-history based parameter adaptation for Differential Evolution , 2013, 2013 IEEE Congress on Evolutionary Computation.

[11]  A. E. Eiben,et al.  On Evolutionary Exploration and Exploitation , 1998, Fundam. Informaticae.

[12]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[13]  D. Fogel.  ASYMPTOTIC CONVERGENCE PROPERTIES OF GENETIC ALGORITHMS AND EVOLUTIONARY PROGRAMMING: ANALYSIS AND EXPERIMENTS , 1994 .

[14]  Xin Yao,et al.  A Survey of Automatic Parameter Tuning Methods for Metaheuristics , 2020, IEEE Transactions on Evolutionary Computation.

[15]  LiHui,et al.  Multiobjective optimization problems with complicated Pareto sets, MOEA/D and NSGA-II , 2009 .

[16]  Ye Tian,et al.  EMODMI: A Multi-Objective Optimization Based Method to Identify Disease Modules , 2021, IEEE Transactions on Emerging Topics in Computational Intelligence.

[17]  Kalyanmoy Deb,et al.  A combined genetic adaptive search (GeneAS) for engineering design , 1996 .

[18]  Ran Cheng,et al.  Adaptive simulated binary crossover for rotated multi-objective optimization , 2021, Swarm Evol. Comput..

[19]  Michèle Sebag,et al.  Toward comparison-based adaptive operator selection , 2010, GECCO '10.

[20]  Mark Hoogendoorn,et al.  Parameter Control in Evolutionary Algorithms: Trends and Challenges , 2015, IEEE Transactions on Evolutionary Computation.

[21]  Ye Tian,et al.  A non-revisiting genetic algorithm based on a novel binary space partition tree , 2020, Inf. Sci..

[22]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[23]  Thomas Stützle,et al.  A Racing Algorithm for Configuring Metaheuristics , 2002, GECCO.

[24]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[25]  Yunfeng Zhang,et al.  Process planning optimization for the manufacture of injection moulds using a genetic algorithm , 2003, Int. J. Comput. Integr. Manuf..

[26]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[27]  Tae Yoon Kim,et al.  Using genetic algorithm to support portfolio optimization for index fund management , 2005, Expert Syst. Appl..

[28]  Marco Dorigo,et al.  Ant colony optimization for continuous domains , 2008, Eur. J. Oper. Res..

[29]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[30]  Michael J. Ryan,et al.  Improved Multi-operator Differential Evolution Algorithm for Solving Unconstrained Problems , 2020, 2020 IEEE Congress on Evolutionary Computation (CEC).

[31]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[32]  Bernhard Sendhoff,et al.  Theoretical comparisons of search dynamics of genetic algorithms and evolution strategies , 2005, 2005 IEEE Congress on Evolutionary Computation.

[33]  Anne Auger,et al.  Impacts of invariance in search: When CMA-ES and PSO face ill-conditioned and non-separable problems , 2011, Appl. Soft Comput..

[34]  Tobias Rodemann.  Industrial Portfolio Management for Many-Objective Optimization Algorithms , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[35]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[36]  C.A. Coello Coello,et al.  MOPSO: a proposal for multiple objective particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[37]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[38]  Thomas Stützle,et al.  Automatic Component-Wise Design of Multiobjective Evolutionary Algorithms , 2016, IEEE Transactions on Evolutionary Computation.

[39]  Anne Auger,et al.  Log-Linear Convergence and Divergence of the Scale-Invariant (1+1)-ES in Noisy Environments , 2011, Algorithmica.