MELM-GRBF: A modified version of the extreme learning machine for generalized radial basis function neural networks

Abstract In this paper, we propose a methodology for training a new model of artificial neural network called the generalized radial basis function (GRBF) neural network. This model is based on generalized Gaussian distribution, which parametrizes the Gaussian distribution by adding a new parameter τ . The generalized radial basis function allows different radial basis functions to be represented by updating the new parameter τ . For example, when GRBF takes a value of τ = 2 , it represents the standard Gaussian radial basis function. The model parameters are optimized through a modified version of the extreme learning machine (ELM) algorithm. In the methodology proposed (MELM-GRBF), the centers of each GRBF were taken randomly from the patterns of the training set and the radius and τ values were determined analytically, taking into account that the model must fulfil two constraints: locality and coverage. An thorough experimental study is presented to test its overall performance. Fifteen datasets were considered, including binary and multi-class problems, all of them taken from the UCI repository. The MELM-GRBF was compared to ELM with sigmoidal, hard-limit, triangular basis and radial basis functions in the hidden layer and to the ELM-RBF methodology proposed by Huang et al. (2004) [1] . The MELM-GRBF obtained better results in accuracy than the corresponding sigmoidal, hard-limit, triangular basis and radial basis functions for almost all datasets, producing the highest mean accuracy rank when compared with these other basis functions for all datasets.

[1]  Cuneyt Guzelis,et al.  Input-output clustering for determining centers of radial basis function network , 1997 .

[2]  Robert K. L. Gay,et al.  Error Minimized Extreme Learning Machine With Growth of Hidden Nodes and Incremental Learning , 2009, IEEE Transactions on Neural Networks.

[3]  Pedro Antonio Gutiérrez,et al.  Evolutionary q-Gaussian radial basis function neural networks for multiclassification , 2011, Neural Networks.

[4]  Chris Bishop,et al.  Improving the Generalization Properties of Radial Basis Function Neural Networks , 1991, Neural Computation.

[5]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[6]  Christian Igel,et al.  Empirical evaluation of the improved Rprop learning algorithms , 2003, Neurocomputing.

[7]  Simon Haykin,et al.  Intelligent Signal Processing , 2001 .

[8]  Witold Pedrycz,et al.  Improving RBF networks performance in regression tasks by means of a supervised fuzzy clustering , 2006, Neurocomputing.

[9]  Lei Xu,et al.  RBF nets, mixture experts, and Bayesian Ying-Yang learning , 1998, Neurocomputing.

[10]  Pedro Antonio Gutiérrez,et al.  Evolutionary q-Gaussian Radial Basis Functions for Improving Prediction Accuracy of Gene Classification Using Feature Selection , 2010, ICANN.

[11]  Michael Vogt Combination of radial basis function neural networks with optimized learning vector quantization , 1993, IEEE International Conference on Neural Networks.

[12]  Chee Kheong Siew,et al.  Extreme learning machine: RBF network case , 2004, ICARCV 2004 8th Control, Automation, Robotics and Vision Conference, 2004..

[13]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[14]  A. Tamhane,et al.  Multiple Comparison Procedures , 1989 .

[15]  O. J. Dunn Multiple Comparisons among Means , 1961 .

[16]  Alfonso Rodríguez-Patón,et al.  Evolutionary system for automatically constructing and adapting radial basis function networks , 2006, Neurocomputing.

[17]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[18]  A. Tamhane,et al.  Multiple Comparison Procedures , 2009 .

[19]  Guido Bugmann,et al.  Normalized Gaussian Radial Basis Function networks , 1998, Neurocomputing.

[20]  Ah Chung Tsoi,et al.  Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results , 1998, Neural Networks.

[21]  Pedro Antonio Gutiérrez,et al.  Development of a multi-classification neural network model to determine the microbial growth/no growth interface. , 2010, International journal of food microbiology.

[22]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[23]  Yves Chauvin,et al.  Backpropagation: theory, architectures, and applications , 1995 .

[24]  Pedro Antonio Gutiérrez,et al.  Evolutionary q-Gaussian Radial Basis Function Neural Network to determine the microbial growth/no growth interface of Staphylococcus aureus , 2011, Appl. Soft Comput..

[25]  César Hervás-Martínez,et al.  Evolutionary product unit based neural networks for regression , 2006, Neural Networks.

[26]  Pedro Antonio Gutiérrez,et al.  Combined projection and kernel basis functions for classification in evolutionary neural networks , 2009, Neurocomputing.

[27]  Zhiping Lin,et al.  Composite function wavelet neural networks with extreme learning machine , 2010, Neurocomputing.

[28]  Pedro Antonio Gutiérrez,et al.  Combined projection and kernel basis functions for classification in evolutionary neural networks , 2009, Neurocomputing.

[29]  Hongming Zhou,et al.  Optimization method based extreme learning machine for classification , 2010, Neurocomputing.

[30]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[31]  Yuan Lan,et al.  Ensemble of online sequential extreme learning machine , 2009, Neurocomputing.

[32]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[33]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[34]  Pedro Antonio Gutiérrez,et al.  Evolutionary product-unit neural networks classifiers , 2008, Neurocomputing.

[35]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[36]  Pedro Antonio Gutiérrez,et al.  A dynamic over-sampling procedure based on sensitivity for multi-class problems , 2011, Pattern Recognit..

[37]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[38]  Mario Ventresca,et al.  Improving gradient-based learning algorithms for large scale feedforward networks , 2009, 2009 International Joint Conference on Neural Networks.

[39]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[40]  M. Friedman A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[41]  I-Cheng Yeh,et al.  First and second order sensitivity analysis of MLP , 2010, Neurocomputing.