A hybrid simultaneous perturbation artificial bee colony and back-propagation algorithm for training a local linear radial basis neural network on ore grade estimation

In this paper, a local linear radial basis function (LLRBF) neural network that uses a skewed Gaussian activation function, called LLRBF-SG, is applied to the problem of ore grade estimation of highly skewed data from the Esfordi phosphate deposit. The network is trained using SPABC-BP, a method that combines a novel simultaneous perturbation artificial bee colony algorithm (SPABC) and back-propagation (BP) method. The SPABC algorithm is an extension of the standard artificial bee colony (ABC) algorithm that includes a tournament selection strategy, simultaneous perturbation stochastic approximation method and new search equations. The predictive accuracy of the network trained with the SPABC-BP algorithm is compared with hybrid versions of evolutionary and swarm intelligence algorithms, such as standard artificial bee colony, covariance matrix adaptation evolution strategy (CMAES) and particle swarm optimization (PSO) with BP. From the experimental results one concludes that networks trained with the SPABC-BP algorithm outperform networks trained with the alternative algorithms in the process of ore grade estimation. The predictive accuracy of the LLRBF-SG-SPABC-BP is compared with LLRBF-SPABC-BP and standard radial basis function (RBF) trained with SPABC-BP algorithm. An analysis of the results shows that the proposed LLRBF-SG network has higher generalization ability and is better suited to the problem of predicting ore grade values for highly skewed data.

[1]  Shigeo Abe.  Support Vector Machines for Pattern Classification , 2010, Advances in Pattern Recognition.

[2]  Lutz Prechelt,et al.  Automatic early stopping using cross validation: quantifying the criteria , 1998, Neural Networks.

[3]  Dervis Karaboga,et al.  Artificial Bee Colony (ABC) Optimization Algorithm for Training Feed-Forward Neural Networks , 2007, MDAI.

[4]  B. Samanta,et al.  Construction of a radial basis function network using an evolutionary algorithm for grade estimation in a placer gold deposit , 2009, Comput. Geosci..

[5]  Bernard Widrow,et al.  The basic ideas in neural networks , 1994, CACM.

[6]  OzturkCelal,et al.  A comprehensive survey , 2014 .

[7]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[8]  W. Nailon,et al.  Fast and accurate image registration using Tsallis entropy and simultaneous perturbation stochastic approximation , 2004 .

[9]  Gholam Ali Montazer,et al.  An improved radial basis function neural network for object image retrieval , 2015, Neurocomputing.

[10]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[11]  Bryan Denby,et al.  Neural Network Modelling of Ore Grade Spatial Variability , 1998 .

[12]  Dervis Karaboga,et al.  A comprehensive survey: artificial bee colony (ABC) algorithm and applications , 2012, Artificial Intelligence Review.

[13]  Aman Jantan,et al.  Swarm intelligence and neural network for data classification , 2014, 2014 IEEE International Conference on Control System, Computing and Engineering (ICCSCE 2014).

[14]  Omid Bozorg Haddad,et al.  Honey-Bees Mating Optimization (HBMO) Algorithm: A New Heuristic Approach for Water Resources Optimization , 2006 .

[15]  Ke Wang,et al.  A hybrid self-adaptive Particle Swarm Optimization–Genetic Algorithm–Radial Basis Function model for annual electricity demand prediction , 2015 .

[16]  Arta A. Jamshidi,et al.  Skew-Radial Basis Function Expansions for Empirical Modeling , 2009, SIAM J. Sci. Comput..

[17]  Yu Li,et al.  Particle swarm optimisation for evolving artificial neural network , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[18]  Arta A. Jamshidi,et al.  Modeling Multivariate Time Series on Manifolds with Skew Radial Basis Functions , 2011, Neural Computation.

[19]  Xiao-li Li,et al.  Hybrid self-adaptive learning based particle swarm optimization and support vector regression model for grade estimation , 2013, Neurocomputing.

[20]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[21]  Yanping Bai,et al.  Generalized radial basis function neural network based on an improved dynamic particle swarm optimization and AdaBoost algorithm , 2015, Neurocomputing.

[22]  Mary F. Wheeler,et al.  A Multiscale and Metamodel Simulation-Based Method for History Matching , 2006 .

[23]  Ke Jia,et al.  An Improved Hybrid Encoding Cuckoo Search Algorithm for 0-1 Knapsack Problems , 2014, Comput. Intell. Neurosci..

[24]  Mohammad Taghi Hamidi Beheshti,et al.  A local linear radial basis function neural network for financial time-series forecasting , 2010, Applied Intelligence.

[25]  Saeed Tavakoli,et al.  Improved Cuckoo Search Algorithm for Feed forward Neural Network Training , 2011 .

[26]  Liping Xie,et al.  Direct interval forecasting of wind speed using radial basis function neural networks in a multi-objective optimization framework , 2016, Neurocomputing.

[27]  Lothar Thiele,et al.  A Mathematical Analysis of Tournament Selection , 1995, ICGA.

[28]  M. Damodaran,et al.  Application of Simultaneous Perturbation Stochastic Approximation Method for Aerodynamic Shape Design Optimization , 2002 .

[29]  Bai Li,et al.  Research on WNN Modeling for Gold Price Forecasting Based on Improved Artificial Bee Colony Algorithm , 2014, Comput. Intell. Neurosci..

[30]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[31]  Ali Selamat,et al.  Hybrid Artificial Neural Network with Artificial Bee Colony Algorithm for Crime Classification , 2014, INNS-CIIS.

[32]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[33]  Jianchao Zeng,et al.  Comparison and Analysis of the Selection Mechanism in the Artificial Bee Colony Algorithm , 2009, 2009 Ninth International Conference on Hybrid Intelligent Systems.

[34]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[35]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[36]  Rozaida Ghazali,et al.  Using Artificial Bee Colony Algorithm for MLP Training on Earthquake Time Series Data Prediction , 2011, ArXiv.

[37]  Wang Ke,et al.  年間電力需要予測のためのハイブリッド自己適応粒子群最適化‐遺伝的アルゴリズム‐動径基底関数モデル , 2015 .

[38]  Alberto Tesi,et al.  On the Problem of Local Minima in Backpropagation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Xin-She Yang,et al.  Cuckoo Search via Lévy flights , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[40]  Magdalene Marinaki,et al.  Honey Bees Mating Optimization algorithm for financial classification problems , 2010, Appl. Soft Comput..

[41]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[42]  Dervis Karaboga,et al.  A survey: algorithms simulating bee swarm intelligence , 2009, Artificial Intelligence Review.

[43]  YehWei-Chang,et al.  Forecasting stock markets using wavelet transforms and recurrent neural networks , 2011 .

[44]  W. González-Manteiga,et al.  Comparison of Kriging and Neural Networks With Application to the Exploitation of a Slate Mine , 2004 .

[45]  Christian Igel,et al.  A Note on Generalization Loss When Evolving Adaptive Pattern Recognition Systems , 2013, IEEE Transactions on Evolutionary Computation.

[46]  James D. Keeler,et al.  Layered Neural Networks with Gaussian Hidden Units as Universal Approximations , 1990, Neural Computation.

[47]  Mingzhe Liu,et al.  Evolving RBF neural networks for rainfall prediction using hybrid particle swarm optimization and genetic algorithm , 2015, Neurocomputing.

[48]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[49]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[50]  Barry J. Adams,et al.  Honey-bee mating optimization (HBMO) algorithm for optimal reservoir operation , 2007, J. Frankl. Inst..

[51]  Zhang Xuegong,et al.  INTRODUCTION TO STATISTICAL LEARNING THEORY AND SUPPORT VECTOR MACHINES , 2000 .

[52]  Michael N. Vrahatis,et al.  On the alleviation of the problem of local minima in back-propagation , 1997 .

[53]  Wei-Chang Yeh,et al.  Forecasting stock markets using wavelet transforms and recurrent neural networks: An integrated system based on artificial bee colony algorithm , 2011, Appl. Soft Comput..

[54]  Snehamoy Chatterjee,et al.  Ore Grade Prediction Using a Genetic Algorithm and Clustering Based Ensemble Neural Network Model , 2010 .

[55]  Chunshien Li,et al.  Complex-Fuzzy Adaptive Image Restoration - An Artificial-Bee-Colony-Based Learning Approach , 2011, ACIIDS.