An Improved Radial Basis Function Networks in Networks Weights Adjustment for Training Real-World Nonlinear Datasets

In neural networks, the accuracies of its networks are mainly relying on two important factors which are the centers and the networks weight. The gradient descent algorithm is a widely used weight adjustment algorithm in most of neural networks training algorithm. However, the method is known for its weakness for easily trap in local minima. It suffers from a random weight generated for the networks during initial stage of training at input layer to hidden layer networks. The performance of radial basis function networks (RBFN) has been improved from different perspectives, including centroid initialization problem to weight correction stage over the years. Unfortunately, the solution does not provide a good trade-off between quality and efficiency of the weight produces by the algorithm. To solve this problem, an improved gradient descent algorithm for finding initial weight and improve the overall networks weight is proposed. This improved version algorithm is incorporated into RBFN training algorithm for updating weight. Hence, this paper presented an improved RBFN in term of algorithm for improving the weight adjustment in RBFN during training process. The proposed training algorithm, which uses improved gradient descent algorithm for weight adjustment for training RBFN, obtained significant improvement in predictions compared to the standard RBFN. The proposed training algorithm was implemented in MATLAB environment. The proposed improved network called IRBFN was tested against the standard RBFN in predictions. The experimental models were tested on four literatures nonlinear function and four real-world application problems, particularly in Air pollutant problem, Biochemical Oxygen Demand (BOD) problem, Phytoplankton problem, and forex pair EURUSD. The results are compared to IRBFN for root mean square error (RMSE) values with standard RBFN. The IRBFN yielded a promising result with an average improvement percentage more than 40 percent in RMSE.

[1]  Dilip Kumar Pratihar,et al.  Tuning of neural networks using particle swarm optimization to model MIG welding process , 2011, Swarm Evol. Comput..

[2]  Safwan El Assad,et al.  Efficient classification algorithm and a new training mode for the adaptive radial basis function neural network equaliser , 2012, IET Commun..

[3]  Hao Yu,et al.  Fast and Efficient Second-Order Method for Training Radial Basis Function Networks , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[4]  John Yearwood,et al.  Novel weighting in single hidden layer feedforward neural networks for data classification , 2012, Comput. Math. Appl..

[5]  Dean Zhao,et al.  A New Optimized GA-RBF Neural Network Algorithm , 2014, Comput. Intell. Neurosci..

[6]  Shaobo Lin.  Linear and nonlinear approximation of spherical radial basis function networks , 2016, J. Complex..

[7]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[8]  Christian Dias Cabacinha,et al.  Bee-inspired RBF network for volume estimation of individual trees , 2018, Comput. Electron. Agric..

[9]  Pedro Antonio Gutiérrez,et al.  Evolutionary q-Gaussian radial basis function neural networks for multiclassification , 2011, Neural Networks.

[10]  Liping Xie,et al.  Direct interval forecasting of wind speed using radial basis function neural networks in a multi-objective optimization framework , 2016, Neurocomputing.

[11]  J. Stephen Judd,et al.  Learning in neural networks , 1988, COLT '88.

[12]  Manuel Kindelan,et al.  Radial basis function interpolation in the limit of increasingly flat basis functions , 2016, J. Comput. Phys..

[13]  Chin-Wang Tao,et al.  Radial Basis Function Networks With Linear Interval Regression Weights for Symbolic Interval Data , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[14]  Reza Mohammadi,et al.  A new hybrid evolutionary based RBF networks method for forecasting time series: A case study of forecasting emergency supply demand time series , 2014, Eng. Appl. Artif. Intell..

[15]  Sung-Kwun Oh,et al.  Optimized face recognition algorithm using radial basis function neural networks and its practical applications , 2015, Neural Networks.

[16]  V. Vaidehi,et al.  Optimum steepest descent higher level learning radial basis function network , 2015, Expert Syst. Appl..

[17]  A. T. Akinwale,et al.  ARTIFICIAL NEURAL NETWORK MODEL FOR FORECASTING FOREIGN EXCHANGE RATE , 2011 .

[18]  Joel J. P. C. Rodrigues,et al.  Evolutionary radial basis function network for gestational diabetes data analytics , 2017, J. Comput. Sci..

[19]  Alex Graves,et al.  Practical Variational Inference for Neural Networks , 2011, NIPS.

[20]  Ahmad Kadri Junoh,et al.  An Improved Radial Basis Function Networks in Networks Weights Adjustment for Training Real-World Nonlinear Datasets , 2019 .

[21]  Kadir Kavaklioglu,et al.  Experimental investigation and radial basis function network modeling of direct evaporative cooling systems , 2018, International Journal of Heat and Mass Transfer.

[22]  Hossein Shafizadeh-Moghadam,et al.  Performance analysis of radial basis function networks and multi-layer perceptron networks in modeling urban change: a case study , 2015, Int. J. Geogr. Inf. Sci..

[23]  P. K. Dash,et al.  A comparative study of radial basis function network with different basis functions for stock trend prediction , 2015, 2015 IEEE Power, Communication and Information Technology Conference (PCITC).

[24]  Song Zhi-zhang.  The Application of RBF Neural Network Based on Ant Colony Clustering Algorithm to Pressure Sensor , 2013 .

[25]  Cheng-Han Tsai,et al.  A novel self-constructing Radial Basis Function Neural-Fuzzy System , 2013, Appl. Soft Comput..

[26]  Mario Cantú-Sifuentes,et al.  Multivariate statistical inference in a radial basis function neural network , 2018, Expert Syst. Appl..

[27]  S. K. Nandy,et al.  A Hardware Architecture for Radial Basis Function Neural Network Classifier , 2018, IEEE Transactions on Parallel and Distributed Systems.

[28]  Feilong Cao,et al.  A study on effectiveness of extreme learning machine , 2011, Neurocomputing.

[29]  Ka Fai Cedric Yiu,et al.  A new optimization algorithm for single hidden layer feedforward neural networks , 2013, Appl. Soft Comput..

[30]  Majid Bagheri,et al.  Modeling of a sequencing batch reactor treating municipal wastewater using multi-layer perceptron and radial basis function artificial neural networks , 2015 .

[31]  Ahmed Osmanović,et al.  Machine Learning Techniques for Classification of Breast Cancer , 2018, IFMBE Proceedings.

[32]  Soumitra Dutta,et al.  Fuzzy rough granular neural networks, fuzzy granules, and classification , 2011, Theor. Comput. Sci..

[33]  Junfei Qiao,et al.  An efficient self-organizing RBF neural network for water quality prediction , 2011, Neural Networks.

[34]  Zarita Zainuddin,et al.  An Improved Fast Training Algorithm for RBF Networks Using Symmetry-Based Fuzzy C-Means Clustering , 2008 .

[35]  R. Gadea,et al.  Multilayer perceptron neural networks and radial-basis function networks as tools to forecast accumulation of deoxynivalenol in barley seeds contaminated with Fusarium culmorum , 2011 .

[36]  Xizhao Wang,et al.  A review on neural networks with random weights , 2018, Neurocomputing.

[37]  Habib Rajabi Mashhadi,et al.  A fast static security assessment method based on radial basis function neural networks using enhanced clustering , 2013 .

[38]  Alexandr Andoni,et al.  Learning Polynomials with Neural Networks , 2014, ICML.

[39]  Wensheng Zhang,et al.  Generalization Performance of Radial Basis Function Networks , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Pei-Chann Chang,et al.  A novel model by evolving partially connected neural network for stock price trend forecasting , 2012, Expert Syst. Appl..

[41]  Haralambos Sarimveis,et al.  Cooperative learning for radial basis function networks using particle swarm optimization , 2016, Appl. Soft Comput..

[42]  Pedro Antonio Gutiérrez,et al.  MELM-GRBF: A modified version of the extreme learning machine for generalized radial basis function neural networks , 2011, Neurocomputing.

[43]  Seyedali Mirjalili.  Evolutionary Radial Basis Function Networks , 2019 .

[44]  Yang Li,et al.  Forecasting short-term subway passenger flow under special events scenarios using multiscale radial basis function networks ☆ , 2017 .

[45]  Pravin Chandra,et al.  A new weight initialization method for sigmoidal feedforward artificial neural networks , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[46]  W. J. Studden,et al.  Design and analysis of computer experiments when the output is highly correlated over the input space , 2002 .

[47]  Haralambos Sarimveis,et al.  A Fast and Efficient Method for Training Categorical Radial Basis Function Networks , 2017, IEEE Trans. Neural Networks Learn. Syst..

[48]  Chong Wu,et al.  Adaptive radial basis function networks with kernel shape parameters , 2010, Neural Computing and Applications.

[49]  Václav Skala,et al.  Radial Basis Function Approximations: Comparison and Applications , 2017, ArXiv.

[50]  Antonio J. Rivera,et al.  Training algorithms for Radial Basis Function Networks to tackle learning processes with imbalanced data-sets , 2014, Appl. Soft Comput..

[51]  Özgür Kisi,et al.  Performance of radial basis and LM-feed forward artificial neural networks for predicting daily watershed runoff , 2013, Appl. Soft Comput..

[52]  Ai Hui Tan,et al.  Optimization of Neural Networks Using Variable Structure Systems , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[53]  Wai Keung Wong,et al.  A hybrid particle swarm optimization and its application in neural networks , 2012, Expert Syst. Appl..

[54]  Hadi Memarian,et al.  Comparison between Multi-Layer Perceptron and Radial Basis Function Networks for Sediment Load Estimation in a Tropical Watershed , 2012 .

[55]  Quang Phuc Ha,et al.  Enhanced radial basis function neural networks for ozone level estimation , 2015, Neurocomputing.

[56]  Haralambos Sarimveis,et al.  A fast training algorithm for RBF networks based on subtractive clustering , 2003, Neurocomputing.

[57]  Mikhail Belkin,et al.  Back to the Future: Radial Basis Function Network Revisited , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Holger Dette,et al.  Generalized Latin Hypercube Design for Computer Experiments , 2010, Technometrics.

[59]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[60]  Cristian Arteaga,et al.  Universal approximation by radial basis function networks of Delsarte translates , 2013, Neural Networks.

[61]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[62]  Guang-Bin Huang,et al.  Extreme Learning Machine for Multilayer Perceptron , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[63]  James Nga-Kwok Liu,et al.  An eigenvector based center selection for fast training scheme of RBFNN , 2018, Inf. Sci..

[64]  Waleed A. Yousef,et al.  Learning algorithms may perform worse with increasing training set size: Algorithm-data incompatibility , 2014, Comput. Stat. Data Anal..

[65]  Ching-Ter Chang,et al.  Single-hidden-layer feed-forward quantum neural network based on Grover learning , 2013, Neural Networks.

[66]  Ke Wang,et al.  A hybrid self-adaptive Particle Swarm Optimization–Genetic Algorithm–Radial Basis Function model for annual electricity demand prediction , 2015 .

[67]  Konstantinos G. Margaritis,et al.  Topology and simulations of a Hierarchical Markovian Radial Basis Function Neural Network classifier , 2015, Inf. Sci..