Numerical optimization and feed-forward neural networks training using an improved optimization algorithm: multiple leader salp swarm algorithm

Metaheuristics are one of the most promising techniques for solving optimization problems. Salp swarm algorithm (SSA) is a new swarm intelligence based metaheuristic. To improve the performance of SSA, this paper introduces multiple leader salp swarm algorithm (MLSSA), which has more exploratory power than SSA. The algorithm is tested on several mathematical optimization benchmark functions. Results are compared with some well known metaheuristics. The results represents the capability of MLSSA to converge towards the optimum. In recent studies many metaheuristic techniques are applied to train feed-forward neural networks. In this paper MLSSA is also applied for neural network training and is analysed for thirteen different datasets. Performance is compared with SSA, particle swarm optimization, differential evolution, genetic algorithm, ant colony optimization and evolution strategy.

[1]  Rasul Enayatifar,et al.  Efficient clustering in collaborative filtering recommender system: Hybrid method based on genetic algorithm and gravitational emulation local search algorithm. , 2019, Genomics.

[2]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms in Engineering Applications , 1997, Springer Berlin Heidelberg.

[3]  Hossam Faris,et al.  Optimizing connection weights in neural networks using the whale optimization algorithm , 2016, Soft Computing.

[4]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[5]  Ibrahim Eksin,et al.  A new optimization method: Big Bang-Big Crunch , 2006, Adv. Eng. Softw..

[6]  V.P. Plagianakos,et al.  Spiking neural network training using evolutionary algorithms , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[7]  Zhenxing Zhang,et al.  A novel atom search optimization for dispersion coefficient estimation in groundwater , 2019, Future Gener. Comput. Syst..

[8]  R. Venkata Rao,et al.  Teaching-Learning-Based Optimization: An optimization method for continuous non-linear large scale problems , 2012, Inf. Sci..

[9]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[10]  Andrew Lewis,et al.  The Whale Optimization Algorithm , 2016, Adv. Eng. Softw..

[11]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[12]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[13]  Ying Tan,et al.  Advances in Swarm Intelligence, First International Conference, ICSI 2010, Beijing, China, June 12-15, 2010, Proceedings, Part II , 2010, ICSI.

[14]  Yimin Zhang,et al.  Hybrid Lagrange interpolation differential evolution algorithm for path synthesis , 2019, Mechanism and Machine Theory.

[15]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[16]  Barry Webster,et al.  A Local Search Optimization Algorithm Based on Natural Principles of Gravitation , 2003, IKE.

[17]  M. M. Fahmy,et al.  Group counseling optimization , 2014, Appl. Soft Comput..

[18]  Dinesh Gopalani,et al.  Salp Swarm Algorithm (SSA) for Training Feed-Forward Neural Networks , 2018, SocProS.

[19]  Fazli Wahid,et al.  An Efficient Approach for Energy Consumption Optimization and Management in Residential Building Using Artificial Bee Colony and Fuzzy Logic , 2016 .

[20]  Zelda B. Zabinsky,et al.  A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems , 2005, J. Glob. Optim..

[21]  Andrew Lewis,et al.  Let a biogeography-based optimizer train your Multi-Layer Perceptron , 2014, Inf. Sci..

[22]  A. Kaveh,et al.  A novel heuristic optimization method: charged system search , 2010 .

[23]  Abdolreza Hatamlou,et al.  Black hole: A new heuristic optimization approach for data clustering , 2013, Inf. Sci..

[24]  S. Shadravan,et al.  The Sailfish Optimizer: A novel nature-inspired metaheuristic algorithm for solving constrained engineering optimization problems , 2019, Eng. Appl. Artif. Intell..

[25]  Q. Henry Wu,et al.  A Novel Group Search Optimizer Inspired by Animal Behavioural Ecology , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[26]  M. M. Fahmy,et al.  Group Counseling Optimization: A Novel Approach , 2009, SGAI Conf..

[27]  Andries Petrus Engelbrecht,et al.  Measuring exploration/exploitation in particle swarms using swarm diversity , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[28]  Nurettin Cetinkaya,et al.  A new meta-heuristic optimizer: Pathfinder algorithm , 2019, Appl. Soft Comput..

[29]  Caro Lucas,et al.  Imperialist competitive algorithm: An algorithm for optimization inspired by imperialistic competition , 2007, 2007 IEEE Congress on Evolutionary Computation.

[30]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[31]  Anupam Yadav,et al.  AEFA: Artificial electric field algorithm for global optimization , 2019, Swarm Evol. Comput..

[32]  Marjan Mernik,et al.  Exploration and exploitation in evolutionary algorithms: A survey , 2013, CSUR.

[33]  Rozaida Ghazali,et al.  An Enhanced Approach of Artificial Bee Colony for Energy Management in Energy Efficient Residential Building , 2018, Wirel. Pers. Commun..

[34]  Dervis Karaboga,et al.  Hybrid Artificial Bee Colony algorithm for neural network training , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[35]  Blum,et al.  [IEEE Fifth International Conference on Hybrid Intelligent Systems (HIS\'05) - Rio de Janeiro, Brazil (2005.11.6-2005.11.9)] Fifth International Conference on Hybrid Intelligent Systems (HIS\'05) - Training feed-forward neural networks with ant colony optimization: an application to pattern classifi , 2005 .

[36]  Singiresu S. Rao.  Engineering Optimization : Theory and Practice , 2010 .

[37]  Rozaida Ghazali,et al.  Hybrid of firefly algorithm and pattern search for solving optimization problems , 2018, Evol. Intell..

[38]  Tung-Kuan Liu,et al.  Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm , 2006, IEEE Trans. Neural Networks.

[39]  Harish Sharma,et al.  Spider Monkey Optimization algorithm for numerical optimization , 2014, Memetic Computing.

[40]  Enrique Alba,et al.  The exploration/exploitation tradeoff in dynamic cellular genetic algorithms , 2005, IEEE Transactions on Evolutionary Computation.

[41]  Vijander Singh,et al.  A novel nature-inspired algorithm for optimization: Squirrel search algorithm , 2019, Swarm Evol. Comput..

[42]  Seyed Mohammad Mirjalili.  How effective is the Grey Wolf optimizer in training multi-layer perceptrons , 2014, Applied Intelligence.

[43]  R. Venkata Rao,et al.  Teaching-learning-based optimization: A novel method for constrained mechanical design optimization problems , 2011, Comput. Aided Des..

[44]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[45]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[46]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[47]  Taher Niknam,et al.  Hybrid Optimization Algorithm to Solve the Nonconvex Multiarea Economic Dispatch Problem , 2019, IEEE Systems Journal.

[48]  Christian Blum,et al.  An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training , 2007, Neural Computing and Applications.

[49]  Seyed Mohammad Mirjalili,et al.  Multi-Verse Optimizer: a nature-inspired algorithm for global optimization , 2015, Neural Computing and Applications.

[50]  Hossam Faris,et al.  Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems , 2017, Adv. Eng. Softw..

[51]  Said Jadid Abdul Kadir,et al.  Binary Optimization Using Hybrid Grey Wolf Optimization for Feature Selection , 2019, IEEE Access.

[52]  Zhenxing Zhang,et al.  Atom search optimization and its application to solve a hydrogeologic parameter estimation problem , 2019, Knowl. Based Syst..

[53]  Xin-She Yang,et al.  Firefly algorithm, stochastic test functions and design optimisation , 2010, Int. J. Bio Inspired Comput..

[54]  Q. Henry Wu,et al.  Group Search Optimizer: An Optimization Algorithm Inspired by Animal Searching Behavior , 2009, IEEE Transactions on Evolutionary Computation.

[55]  Harish Sharma,et al.  Beer froth artificial bee colony algorithm for job-shop scheduling problem , 2018, Appl. Soft Comput..

[56]  Hua Chen,et al.  Periodic charging planning for a mobile WCE in wireless rechargeable sensor networks based on hybrid PSO and GA algorithm , 2019, Appl. Soft Comput..

[57]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[58]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[59]  Liang Gao,et al.  Queuing search algorithm: A novel metaheuristic algorithm for solving engineering optimization problems , 2018, Applied Mathematical Modelling.

[60]  Hossein Nezamabadi-pour,et al.  GSA: A Gravitational Search Algorithm , 2009, Inf. Sci..

[61]  Dinesh Gopalani,et al.  Opposition-Based Sine Cosine Algorithm (OSCA) for Training Feed-Forward Neural Networks , 2017, 2017 13th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[62]  Fred W. Glover,et al.  Tabu Search - Part I , 1989, INFORMS J. Comput..

[63]  Gisbert Schneider,et al.  Optimized Particle Swarm Optimization (OPSO) and its application to artificial neural network training , 2006, BMC Bioinformatics.

[64]  Christian Blum,et al.  Training feed-forward neural networks with ant colony optimization: an application to pattern classification , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[65]  Ying Tan,et al.  Fireworks Algorithm for Optimization , 2010, ICSI.

[66]  M. Bialko,et al.  Training of artificial neural networks using differential evolution algorithm , 2008, 2008 Conference on Human System Interactions.

[67]  Richard A. Formato,et al.  CENTRAL FORCE OPTIMIZATION: A NEW META-HEURISTIC WITH APPLICATIONS IN APPLIED ELECTROMAGNETICS , 2007 .

[68]  Dinesh Gopalani,et al.  A Novel Swarm Intelligence Based Optimization Method: Harris' Hawk Optimization , 2018, ISDA.

[69]  Jouni Lampinen,et al.  A Trigonometric Mutation Operation to Differential Evolution , 2003, J. Glob. Optim..

[70]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[71]  Thomas Stützle,et al.  Ant colony optimization: artificial ants as a computational intelligence technique , 2006 .

[72]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[73]  Dan Simon,et al.  Biogeography-Based Optimization , 2022 .

[74]  Chaohua Dai,et al.  Seeker Optimization Algorithm , 2006, 2006 International Conference on Computational Intelligence and Security.

[75]  Harish Sharma,et al.  Balanced artificial bee colony algorithm , 2013, Int. J. Artif. Intell. Soft Comput..

[76]  Paulo Cortez,et al.  Particle swarms for feedforward neural network training , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[77]  A. Gandomi.  Interior search algorithm (ISA): a novel approach for global optimization. , 2014, ISA transactions.