Training Feedforward Neural Networks Using Symbiotic Organisms Search Algorithm

Symbiotic organisms search (SOS) is a new robust and powerful metaheuristic algorithm, which stimulates the symbiotic interaction strategies adopted by organisms to survive and propagate in the ecosystem. In the supervised learning area, it is a challenging task to present a satisfactory and efficient training algorithm for feedforward neural networks (FNNs). In this paper, SOS is employed as a new method for training FNNs. To investigate the performance of the aforementioned method, eight different datasets selected from the UCI machine learning repository are employed for experiment and the results are compared among seven metaheuristic algorithms. The results show that SOS performs better than other algorithms for training FNNs in terms of converging speed. It is also proven that an FNN trained by the method of SOS has better accuracy than most algorithms compared.

[1]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[2]  De-Shuang Huang,et al.  A Constructive Hybrid Structure Optimization Methodology for Radial Basis Probabilistic Neural Networks , 2008, IEEE Transactions on Neural Networks.

[3]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[4]  Ugur Güvenc,et al.  Symbiotic organisms search optimization algorithm for economic/emission dispatch problem in power systems , 2016, Neural Computing and Applications.

[5]  De-Shuang Huang,et al.  Human face recognition based on multi-features using neural networks committee , 2004, Pattern Recognit. Lett..

[6]  Xiaofeng Wang,et al.  An efficient local Chan-Vese model for image segmentation , 2010, Pattern Recognit..

[7]  Naimin Zhang.  An online gradient method with momentum for two-layer feedforward neural networks , 2009, Appl. Math. Comput..

[8]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[9]  R. Siegler.  Three aspects of cognitive development , 1976, Cognitive Psychology.

[10]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[11]  Siti Zaiton Mohd Hashim,et al.  Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm , 2012, Appl. Math. Comput..

[12]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[13]  Nor Ashidi Mat Isa,et al.  Clustered-Hybrid Multilayer Perceptron network for pattern recognition application , 2011, Appl. Soft Comput..

[14]  Silke A.T. Weber,et al.  Social-Spider Optimization-Based Artificial Neural Networks Training and Its Applications for Parkinson's Disease Identification , 2014, 2014 IEEE 27th International Symposium on Computer-Based Medical Systems.

[15]  V. Mukherjee,et al.  A novel symbiotic organisms search algorithm for congestion management in deregulated environment , 2017, J. Exp. Theor. Artif. Intell..

[16]  O. Nelles.  Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models , 2000 .

[17]  Shafii Muhammad Abdulhamid,et al.  Symbiotic Organism Search optimization based task scheduling in cloud computing environment , 2016, Future Gener. Comput. Syst..

[18]  Guoqiang Peter Zhang,et al.  Neural network forecasting for seasonal and trend time series , 2005, Eur. J. Oper. Res..

[19]  Zita A. Vale,et al.  Multilayer perceptron neural networks training through charged system search and its Application for non-technical losses detection , 2013, 2013 IEEE PES Conference on Innovative Smart Grid Technologies (ISGT Latin America).

[20]  Min-Yuan Cheng,et al.  Optimizing Multiple-Resources Leveling in Multiple Projects Using Discrete Symbiotic Organisms Search , 2016, J. Comput. Civ. Eng..

[21]  Seyed Mohammad Mirjalili.  How effective is the Grey Wolf optimizer in training multi-layer perceptrons , 2014, Applied Intelligence.

[22]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[23]  Navid Razmjooy,et al.  A multi layer perceptron neural network trained by Invasive Weed Optimization for potato color image segmentation. , 2012 .

[24]  Hossein Nezamabadi-pour,et al.  GSA: A Gravitational Search Algorithm , 2009, Inf. Sci..

[25]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[26]  Christian Blum,et al.  Training feed-forward neural networks with ant colony optimization: an application to pattern classification , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[27]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[28]  D.-S. Huang,et al.  Radial Basis Probabilistic Neural Networks: Model and Application , 1999, Int. J. Pattern Recognit. Artif. Intell..

[29]  Xiaofeng Wang,et al.  A Novel Density-Based Clustering Framework by Using Level Set Method , 2009, IEEE Transactions on Knowledge and Data Engineering.

[30]  Vincent F. Yu,et al.  Symbiotic Organism Search (SOS) for Solving the Capacitated Vehicle Routing Problem , 2015 .

[31]  Min-Yuan Cheng,et al.  A novel Multiple Objective Symbiotic Organisms Search (MOSOS) for time-cost-labor utilization tradeoff problem , 2016, Knowl. Based Syst..

[32]  Himansu Sekhar Behera,et al.  A novel nature inspired firefly algorithm with higher order neural network: Performance analysis , 2016 .

[33]  Budi Santosa,et al.  Symbiotic organisms search and two solution representations for solving the capacitated vehicle routing problem , 2017, Appl. Soft Comput..

[34]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[35]  Cheng-Jian Lin,et al.  A self-adaptive quantum radial basis function network for classification applications , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[36]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[37]  Min-Yuan Cheng,et al.  Symbiotic Organisms Search: A new metaheuristic optimization algorithm , 2014 .

[38]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[39]  De-Shuang Huang,et al.  A constructive approach for finding arbitrary roots of polynomials by neural networks , 2004, IEEE Transactions on Neural Networks.

[40]  Siti Mariyam Hj. Shamsuddin,et al.  Enhancement of artificial neural network learning using centripetal accelerated particle swarm optimization for medical diseases diagnosis , 2014, Soft Comput..

[41]  Rory A. Fisher,et al.  Has Mendel's work been rediscovered? , 1936 .

[42]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[43]  Tayfun Dede,et al.  Estimates of energy consumption in Turkey using neural networks with the teaching–learning-based optimization algorithm , 2014 .

[44]  Hojjat Adeli,et al.  Spiking Neural Networks , 2009, Int. J. Neural Syst..

[45]  M. Georgiopoulos,et al.  Feed-forward neural networks , 1994, IEEE Potentials.

[46]  I-Cheng Yeh,et al.  Knowledge discovery on RFM model using Bernoulli sequence , 2009, Expert Syst. Appl..

[47]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[48]  Andrew Lewis,et al.  Let a biogeography-based optimizer train your Multi-Layer Perceptron , 2014, Inf. Sci..

[49]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[50]  Bikash Das,et al.  DG placement in radial distribution network by symbiotic organisms search algorithm for real power loss minimization , 2016, Appl. Soft Comput..

[51]  Piotr A. Kowalski,et al.  Training Neural Networks with Krill Herd Algorithm , 2015, Neural Processing Letters.

[52]  Dervis Karaboga,et al.  Artificial Bee Colony (ABC) Optimization Algorithm for Training Feed-Forward Neural Networks , 2007, MDAI.

[53]  Behnam Malakooti,et al.  Approximating polynomial functions by Feedforward Artificial Neural Networks: Capacity analysis and design , 1998 .

[54]  Arnapurna Panda,et al.  A Symbiotic Organisms Search algorithm with adaptive penalty function to solve multi-objective constrained optimization problems , 2016, Appl. Soft Comput..

[55]  Saeed Tavakoli,et al.  Improved Cuckoo Search Algorithm for Feed forward Neural Network Training , 2011 .

[56]  Vivekananda Mukherjee,et al.  A novel symbiotic organisms search algorithm for optimal power flow of power system with FACTS devices , 2016 .

[57]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[58]  Subhabrata Banerjee,et al.  Power Optimization of Three Dimensional Turbo Code Using a Novel Modified Symbiotic Organism Search (MSOS) Algorithm , 2017, Wirel. Pers. Commun..

[59]  Dervis Karaboga,et al.  Hybrid Artificial Bee Colony algorithm for neural network training , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[60]  W. Kinsner,et al.  Chaotic simulated annealing in multilayer feedforward networks , 1996, Proceedings of 1996 Canadian Conference on Electrical and Computer Engineering.

[61]  Christian Blum,et al.  An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training , 2007, Neural Computing and Applications.

[62]  Hossam Faris,et al.  Training feedforward neural networks using multi-verse optimizer for binary classification problems , 2016, Applied Intelligence.

[63]  Georg Dorffner,et al.  Neural Networks for Time Series Processing , 1996 .