Evaluating center-seeking and initialization bias: The case of particle swarm and gravitational search algorithms

Abstract Complex optimization problems that cannot be solved using exhaustive search require efficient search metaheuristics to find optimal solutions. In practice, metaheuristics suffer from various types of search bias, the understanding of which is of crucial importance, as it is directly pertinent to the problem of making the best possible selection of solvers. In this paper, two metrics are introduced: one for measuring center-seeking bias (CSB) and one for initialization region bias (IRB). The former is based on “ ξ -center offset”, an alternative to “center offset”, which is a common but inadequate approach to analyzing the center-seeking behavior of algorithms, as will be shown. The latter is proposed on the grounds of “region scaling”. The introduced metrics are used to evaluate the bias of three algorithms while running on a test bed of optimization problems having their optimal solution at, or near, the center of the search space. The most prominent finding of this paper is considerable CSB and IRB in the gravitational search algorithm (GSA). In addition, a partial solution to the center-seeking and initialization region bias of GSA is proposed by introducing a “mass-dispersed” version of GSA, mdGSA. mdGSA promotes the global search capability of GSA. Its performance is verified using the same mathematical optimization problem, next to a gene regulatory network parameter identification problem. The results of these experiments demonstrate the capabilities of mdGSA in solving real-world optimization problems.

[1]  G. K. Mahanti,et al.  Comparative Performance of Gravitational Search Algorithm and Modified Particle Swarm Optimization Algorithm for Synthesis of Thinned Scanned Concentric Ring Array Antenna , 2010 .

[2]  James Kennedy,et al.  Some Issues and Practices for Particle Swarms , 2007, 2007 IEEE Swarm Intelligence Symposium.

[3]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[4]  James M. Whitacre Recent trends indicate rapid growth of nature-inspired optimization in academia and industry , 2011, Computing.

[5]  Masahiro Okamoto,et al.  Nonlinear Numerical Optimization Technique Based on a Genetic Algorithm for Inverse Problems: Towards the Inference of Genetic Networks , 1999, German Conference on Bioinformatics.

[6]  Carlos A. Coello Coello,et al.  Evolutionary hidden information detection by granulation-based fitness approximation , 2010, Appl. Soft Comput..

[7]  Hitoshi Iba,et al.  Inference of gene regulatory networks using s-system and differential evolution , 2005, GECCO '05.

[8]  Humberto Bustince,et al.  A gravitational approach to edge detection based on triangular norms , 2010, Pattern Recognit..

[9]  Anton Crombach,et al.  Evolution of Evolvability in Gene Regulatory Networks , 2008, PLoS Comput. Biol..

[10]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[11]  Thomas Philip Runarsson,et al.  Constrained Evolutionary Optimization by Approximate Ranking and Surrogate Models , 2004, PPSN.

[12]  Zbigniew Michalewicz,et al.  Variants of Evolutionary Algorithms for Real-World Applications , 2011, Variants of Evolutionary Algorithms for Real-World Applications.

[13]  Naser Pariz,et al.  Adaptive Fuzzy Fitness Granulation in Structural Optimization Problems , 2007, 2007 IEEE 22nd International Symposium on Intelligent Control.

[14]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[15]  Jianzhong Zhou,et al.  Parameters identification of hydraulic turbine governing system using improved gravitational search algorithm , 2011 .

[16]  Hossein Nezamabadi-pour,et al.  Filter modeling using gravitational search algorithm , 2011, Eng. Appl. Artif. Intell..

[17]  Mohsen Davarynejad,et al.  Gene regulatory network model identification using artificial bee colony and swarm intelligence , 2012, 2012 IEEE Congress on Evolutionary Computation.

[18]  Efrn Mezura-Montes,et al.  Constraint-Handling in Evolutionary Optimization , 2009 .

[19]  WangFeng-Sheng,et al.  Evolutionary optimization with data collocation for reverse engineering of biological networks , 2005 .

[20]  Mohammad-R. Akbarzadeh-T,et al.  Perception-based heuristic granular search: Exploiting uncertainty for analysis of certain functions , 2011, Sci. Iran..

[21]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[22]  D. Goldberg,et al.  Don't evaluate, inherit , 2001 .

[23]  Sakti Prasad Ghoshal,et al.  A novel opposition-based gravitational search algorithm for combined economic and emission dispatch problems of power systems , 2012 .

[24]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[25]  Martin Pelikan,et al.  Initial-population bias in the univariate estimation of distribution algorithm , 2009, GECCO.

[26]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[27]  Peter A. Whigham,et al.  Search bias, language bias and genetic programming , 1996 .

[28]  Wei-Po Lee,et al.  Computational methods for discovering gene networks from expression data , 2009, Briefings Bioinform..

[29]  P. J. Angeline,et al.  Using selection to improve particle swarm optimization , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[30]  F. Glover,et al.  Handbook of Metaheuristics , 2019, International Series in Operations Research & Management Science.

[31]  Xin Yao,et al.  Stochastic ranking for constrained evolutionary optimization , 2000, IEEE Trans. Evol. Comput..

[32]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[33]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[34]  Hossein Nezamabadi-pour,et al.  GSA: A Gravitational Search Algorithm , 2009, Inf. Sci..

[35]  G. Di Caro,et al.  Ant colony optimization: a new meta-heuristic , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[36]  Yun Shang,et al.  A Note on the Extended Rosenbrock Function , 2006 .

[37]  Jrgen Branke,et al.  Evolutionary approaches to dynamic optimization problems , 2001 .

[38]  N. Hansen,et al.  Markov Chain Analysis of Cumulative Step-Size Adaptation on a Linear Constrained Problem , 2015, Evolutionary Computation.

[39]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[40]  Carlos A. Coello Coello,et al.  A Fitness Granulation Approach for Large-Scale Structural Design Optimization , 2012, Variants of Evolutionary Algorithms for Real-World Applications.

[41]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[42]  Kevin D. Seppi,et al.  Exposing origin-seeking bias in PSO , 2005, GECCO '05.

[43]  Serhat Duman,et al.  GRAVITATIONAL SEARCH ALGORITHM FOR ECONOMIC DISPATCH WITH VALVE-POINT EFFECTS , 2010 .

[44]  Alina Sîrbu,et al.  Comparison of evolutionary algorithms in gene regulatory network model inference , 2010, BMC Bioinformatics.

[45]  Z. Bar-Joseph,et al.  Algorithms in nature: the convergence of systems biology and computational thinking , 2011, Molecular systems biology.

[46]  Ugur Güvenc,et al.  Combined economic and emission dispatch solution using gravitational search algorithm , 2012, Sci. Iran..

[47]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[48]  Carlos A. Coello Coello,et al.  Accelerating convergence towards the optimal pareto front , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[49]  Carlos A. Coello Coello,et al.  THEORETICAL AND NUMERICAL CONSTRAINT-HANDLING TECHNIQUES USED WITH EVOLUTIONARY ALGORITHMS: A SURVEY OF THE STATE OF THE ART , 2002 .

[50]  Ajith Abraham,et al.  Particle Swarm Optimization: Performance Tuning and Empirical Analysis , 2009, Foundations of Computational Intelligence.

[51]  Shigenobu Kobayashi,et al.  A robust real-coded genetic algorithm using Unimodal Normal Distribution Crossover augmented by Uniform Crossover: effects of self-adaptation of crossover probabilities , 1999 .

[52]  Mohsen Davarynejad,et al.  Mass-Dispersed Gravitational Search Algorithm for Gene Regulatory Network Model Parameter Identification , 2012, SEAL.

[53]  Kalyanmoy Deb,et al.  A Computationally Efficient Evolutionary Algorithm for Real-Parameter Optimization , 2002, Evolutionary Computation.

[54]  Shuhei Kimura,et al.  Inference of S-system models of genetic networks using a cooperative coevolutionary algorithm , 2005, Bioinform..

[55]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[56]  P. N. Suganthan,et al.  A dynamic neighborhood learning based particle swarm optimizer for global numerical optimization , 2012, Inf. Sci..

[57]  Stefan Preitl,et al.  Gravitational search algorithm-based design of fuzzy control systems with a reduced parametric sensitivity , 2013, Inf. Sci..

[58]  Thomas Schlitt,et al.  Approaches to modeling gene regulatory networks: a gentle introduction. , 2013, Methods in molecular biology.

[59]  Yun-Wei Shang,et al.  A Note on the Extended Rosenbrock Function , 2006, Evolutionary Computation.

[60]  Andreas Hegyi,et al.  Motorway ramp-metering control with queuing consideration using Q-learning , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[61]  Leandro dos Santos Coelho,et al.  A hybrid shuffled complex evolution approach with pattern search for unconstrained optimization , 2011, Math. Comput. Simul..

[62]  Ajith Abraham,et al.  Inter-particle communication and search-dynamics of lbest particle swarm optimizers: An analysis , 2012, Inf. Sci..

[63]  Swagatam Das,et al.  Dynamic Constrained Optimization with offspring repair based Gravitational Search Algorithm , 2013, 2013 IEEE Congress on Evolutionary Computation.

[64]  Masaru Tomita,et al.  Dynamic modeling of genetic networks using genetic algorithm and S-system , 2003, Bioinform..