Training radial basis function networks using biogeography-based optimizer

Training artificial neural networks is considered as one of the most challenging machine learning problems. This is mainly due to the presence of a large number of solutions and changes in the search space for different datasets. Conventional training techniques mostly suffer from local optima stagnation and degraded convergence, which make them impractical for datasets with many features. The literature shows that stochastic population-based optimization techniques suit this problem better and are reliably alternative because of high local optima avoidance and flexibility. For the first time, this work proposes a new learning mechanism for radial basis function networks based on biogeography-based optimizer as one of the most well-regarded optimizers in the literature. To prove the efficacy of the proposed methodology, it is employed to solve 12 well-known datasets and compared to 11 current training algorithms including gradient-based and stochastic approaches. The paper considers changing the number of neurons and investigating the performance of algorithms on radial basis function networks with different number of parameters as well. A statistical test is also conducted to judge about the significance of the results. The results show that the biogeography-based optimizer trainer is able to substantially outperform the current training algorithms on all datasets in terms of classification accuracy, speed of convergence, and entrapment in local optima. In addition, the comparison of trainers on radial basis function networks with different neurons size reveal that the biogeography-based optimizer trainer is able to train radial basis function networks with different number of structural parameters effectively.

[1]  Simon Fong,et al.  Accelerated Particle Swarm Optimization and Support Vector Machine for Business Optimization and Applications , 2011, NDT.

[2]  Chao-Ming Huang,et al.  An RBF Network With OLS and EPSO Algorithms for Real-Time Power Dispatch , 2007, IEEE Transactions on Power Systems.

[3]  Pedro Antonio Gutiérrez,et al.  Classification by Evolutionary Generalized Radial Basis Functions , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[4]  Goran Martinovic,et al.  Automatic Design of Radial Basis Function Networks Through Enhanced Differential Evolution , 2015, HAIS.

[5]  M.N.S. Swamy,et al.  Neural networks in a softcomputing framework , 2006 .

[6]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[7]  Apichat Heednacram,et al.  Implementation of Cuckoo Search in RBF Neural Network for Flood Forecasting , 2012, 2012 Fourth International Conference on Computational Intelligence, Communication Systems and Networks.

[8]  Leandro Nunes de Castro,et al.  BeeRBF: A bee-inspired data clustering approach to design RBF neural network classifiers , 2016, Neurocomputing.

[9]  Stephen A. Billings,et al.  Radial basis function network configuration using genetic algorithms , 1995, Neural Networks.

[10]  Ming-Huwi Horng,et al.  Firefly Meta-Heuristic Algorithm for Training the Radial Basis Function Network for Data Classification and Disease Diagnosis , 2012 .

[11]  D. Broomhead,et al.  Radial Basis Functions, Multi-Variable Functional Interpolation and Adaptive Networks , 1988 .

[12]  Ruba Talal.  Comparative Study between the (BA) Algorithm and (PSO) Algorithm to Train (RBF) Network at Data Classification , 2014 .

[13]  Yu-Chi Ho,et al.  Simple Explanation of the No Free Lunch Theorem of Optimization , 2001 .

[14]  Magnus Erik,et al.  Tuning Dierential Evolution For Articial Neural Networks , 2008 .

[15]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[16]  Carlos A. Coello Coello,et al.  A comparative study of differential evolution variants for global optimization , 2006, GECCO.

[17]  Chng Eng Siong,et al.  Gradient radial basis function networks for nonlinear and nonstationary time series prediction , 1996, IEEE Trans. Neural Networks.

[18]  Erkan Besdok,et al.  A Comparison of RBF Neural Network Training Algorithms for Inertial Sensor Based Terrain Classification , 2009, Sensors.

[19]  Min-Jung Lee,et al.  An adaptive neurocontroller using RBFN for robot manipulators , 2004, IEEE Transactions on Industrial Electronics.

[20]  Li Xiao-xia,et al.  Radial Basis Function Neural Network Based on Ant Colony Optimization , 2007, 2007 International Conference on Computational Intelligence and Security Workshops (CISW 2007).

[21]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[22]  Hao Yu,et al.  Selection of Proper Neural Network Sizes and Architectures—A Comparative Study , 2012, IEEE Transactions on Industrial Informatics.

[23]  Nikola Pavesic,et al.  Training RBF networks with selective backpropagation , 2004, Neurocomputing.

[24]  Guimin Chen,et al.  A Particle Swarm Optimizer with Multi-stage Linearly-Decreasing Inertia Weight , 2009, 2009 International Joint Conference on Computational Sciences and Optimization.

[25]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[26]  Li Cheng,et al.  A New Metaheuristic Bat-Inspired Algorithm , 2010 .

[27]  Li Xu,et al.  An optimizing method of RBF neural network based on genetic algorithm , 2011, Neural Computing and Applications.

[28]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[29]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[30]  Ibrahim Aljarah,et al.  Parallel particle swarm optimization clustering algorithm based on MapReduce methodology , 2012, 2012 Fourth World Congress on Nature and Biologically Inspired Computing (NaBIC).

[31]  Allan de M. Martins,et al.  Deterministic System Identification Using RBF Networks , 2014 .

[32]  Aboul Ella Hassanien,et al.  Hybrid Learning Enhancement of RBF Network with Particle Swarm Optimization , 2009, Foundations of Computational Intelligence.

[33]  Sultan Noman Qasem,et al.  Multi-objective hybrid evolutionary algorithms for radial basis function neural network design , 2012, Knowl. Based Syst..

[34]  Xin-She Yang,et al.  Firefly Algorithms for Multimodal Optimization , 2009, SAGA.

[35]  Christian W. Dawson,et al.  A review of genetic algorithms applied to training radial basis function networks , 2004, Neural Computing & Applications.

[36]  Xin-She Yang,et al.  Firefly Algorithm, Lévy Flights and Global Optimization , 2010, SGAI Conf..

[37]  Man Wai Mak,et al.  Genetic evolution of radial basis function centers for pattern classification , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[38]  Dan Simon,et al.  Biogeography-based optimization of neuro-fuzzy system parameters for diagnosis of cardiac disease , 2010, GECCO '10.

[39]  Andrew John Chipperfield,et al.  Tuning Differential Evolution For Artificial Neural Networks , 2008 .

[40]  Sultan Noman Qasem,et al.  Author's Personal Copy Applied Soft Computing Radial Basis Function Network Based on Time Variant Multi-objective Particle Swarm Optimization for Medical Diseases Diagnosis , 2022 .

[41]  Xin-She Yang,et al.  Firefly Algorithm: Recent Advances and Applications , 2013, ArXiv.

[42]  Bing Yu,et al.  Training radial basis function networks with differential evolution , 2006, 2006 IEEE International Conference on Granular Computing.

[43]  Ludmila I. Kuncheva,et al.  Initializing of an RBF network by a genetic algorithm , 1997, Neurocomputing.

[44]  Satchidananda Dehuri,et al.  Neural Networks Training Based on Differential Evolution in Radial Basis Function Networks for Classification of Web Logs , 2013, ICDCIT.

[45]  Alaa F. Sheta,et al.  Time-series forecasting using GA-tuned radial basis functions , 2001, Inf. Sci..

[46]  Xin-She Yang,et al.  A New Metaheuristic Bat-Inspired Algorithm , 2010, NICSO.

[47]  Friedhelm Schwenker,et al.  Three learning phases for radial-basis-function networks , 2001, Neural Networks.

[48]  Andrew Lewis,et al.  Let a biogeography-based optimizer train your Multi-Layer Perceptron , 2014, Inf. Sci..

[49]  John Tsimikas,et al.  On training RBF neural networks using input-output fuzzy clustering and particle swarm optimization , 2013, Fuzzy Sets Syst..

[50]  Hui Wang,et al.  Using Radial Basis Function Networks for Function Approximation and Classification , 2012 .

[51]  Patrick Siarry,et al.  A survey on optimization metaheuristics , 2013, Inf. Sci..

[52]  Helon Vicente Hultmann Ayala,et al.  Multiobjective Cuckoo Search Applied to Radial Basis Function Neural Networks Training for System Identification , 2014 .

[53]  David B. Fogel,et al.  The Advantages of Evolutionary Computation , 1997, BCEC.

[54]  De-Shuang Huang,et al.  A mended hybrid learning algorithm for radial basis function neural networks to improve generalization capability , 2007 .

[55]  J. A. Leonard,et al.  Radial basis function networks for classifying process faults , 1991, IEEE Control Systems.

[56]  Ajith Abraham,et al.  Inertia Weight strategies in Particle Swarm Optimization , 2011, 2011 Third World Congress on Nature and Biologically Inspired Computing.

[57]  Dan Simon,et al.  Biogeography-Based Optimization , 2022 .

[58]  Hui Peng,et al.  A hybrid algorithm to optimize RBF network architecture and parameters for nonlinear time series prediction , 2012 .

[59]  Xin-She Yang,et al.  Cuckoo Search via Lévy flights , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).