An Immunological Density-Preserving Approach to the Synthesis of RBF Neural Networks for classification

Radial basis function (RBF) neural networks are universal approximators and have been used for a wide range of applications. Aiming at reducing the number of neurons in the hidden layer, for regularization purposes, the center and dispersion of each RBF have to be properly defined by means of competitive learning. Only the output weights will be defined in a supervised manner. One of the drawbacks of such learning methodology, involving unsupervised and supervised learning, is that the centers will be defined so that regions in the input space with a high density of samples tend to be under-represented and those regions with a low density of samples tend to be over-represented. Additionally, few approaches provide a proper and individual indication of the dispersion of each RBF. This paper presents an immune density-preserving algorithm with adaptive radius, called ARIA, to determine the number of centers, their location and the dispersion of each RBF, based only on the available training data set. Considering classification problems, the algorithm to determine the hidden layer is compared to another immune-inspired algorithm called aiNet, K-means and the random choice of centers. The classification accuracy of the final network is compared to another density based approach and a decision tree classifier, C 5.0. The results are reported and analyzed.

[1]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[2]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[3]  Horst Bischof,et al.  An efficient MDL-based construction of RBF networks , 1998, Neural Networks.

[4]  James D. Keeler,et al.  Layered Neural Networks with Gaussian Hidden Units as Universal Approximations , 1990, Neural Computation.

[5]  Shie-Jue Lee,et al.  An ART-based construction of RBF networks , 2002, IEEE Trans. Neural Networks.

[6]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[7]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[8]  Jerne Nk.  Towards a network theory of the immune system. , 1974 .

[9]  Fernando José Von Zuben,et al.  Adaptive Radius Immune Algorithm for Data Clustering , 2005, ICARIS.

[10]  Fernando José Von Zuben,et al.  Handling Data Sparseness in Gene Network Reconstruction , 2005, 2005 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology.

[11]  Tomaso A. Poggio,et al.  Regularization Theory and Neural Networks Architectures , 1995, Neural Computation.

[12]  Ivo Düntsch,et al.  Density based approach to classification , 2003 .

[13]  Fernando José Von Zuben,et al.  Automatic Determination Of Radial Basis Functions: An Immunity-Based Approach , 2001, Int. J. Neural Syst..

[14]  M. Forster.  The new science of simplicity 1 , 2001 .

[15]  Mark J. L. Orr,et al.  Regularization in the Selection of Radial Basis Function Centers , 1995, Neural Computation.

[16]  G L Ada,et al.  The clonal-selection theory. , 1987, Scientific American.

[17]  Jonathan Timmis,et al.  Artificial Immune Systems: A New Computational Intelligence Approach , 2003 .

[18]  Sung Yang Bang,et al.  An Efficient Method to Construct a Radial Basis Function Neural Network Classifier , 1997, Neural Networks.

[19]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.