POFGEC: growing neural network of classifying potential function generators

In this paper, we propose an architecture and learning algorithm for a growing neural network. Drawing inspiration from the idea of electrical potentials, we develop a classifier based on a set of synthesised potential fields over the domain of input space using symmetrical functions (kernels). We propose a multilayer, multiclass potential function generators classifier (POFGEC) utilising growing architecture and a training algorithm to sequentially add potential functions created by the training patterns, if the addition improves the NN classification performance. We also present a pruning algorithm to achieve compact architecture. POFGEC incorporates the electrical potentials concept in the two main neural net building blocks: potential function generators (PFGs) and potential function entities (PFEs), which perform a non-linear transformation of the input data and create the decision rules by constructing the cumulative potential functions and adjusting the weights. The implementation of the presented method with several datasets demonstrates its capabilities in generating classification solutions for datasets of various shapes independent from the number of predefined classes. We also offer substantial comparative analysis with other known approaches in order to fully illustrate the capabilities of the proposed method and its relation with other existing techniques.

[1]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[2]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[3]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[4]  Hyung-Cheul Shin,et al.  Classification of BMI control commands from rat's neural signals using extreme learning machine , 2009, Biomedical engineering online.

[5]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[6]  Ke Chen,et al.  A self-generating modular neural network architecture for supervised learning , 1997, Neurocomputing.

[7]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[8]  Michael T. Manry,et al.  An integrated growing-pruning method for feedforward network training , 2008, Neurocomputing.

[9]  Iren Valova,et al.  Building RBF Neural Network Topology through Potential Functions , 2003, ICANN.

[10]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[11]  David A. Elizondo,et al.  The recursive deterministic perceptron neural network , 1998, Neural Networks.

[12]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[13]  Naftali Tishby,et al.  Data Clustering by Markovian Relaxation and the Information Bottleneck Method , 2000, NIPS.

[14]  Pawel Strumillo,et al.  Kernel orthonormalization in radial basis function neural networks , 1997, IEEE Trans. Neural Networks.

[15]  Visakan Kadirkamanathan,et al.  STATISTICAL CONTROL OF GROWING AND PRUNING IN RBF-LIKE NEURAL NETWORKS , 1997 .

[16]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[17]  Bernd Fritzke.  Supervised Learning with Growing Cell Structures , 1993, NIPS.

[18]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[19]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[20]  Julio Ortega Lopera,et al.  Improved RAN sequential prediction using orthogonal techniques , 2001, Neurocomputing.

[21]  Narasimhan Sundararajan,et al.  Radial Basis Function Neural Networks With Sequential Learning: Mran and Its Applications , 1999 .

[22]  Eytan Domany,et al.  Data Clustering Using a Model Granular Magnet , 1997, Neural Computation.

[23]  Iren Valova,et al.  DYPOF: Dynamically Adaptive RBF Neural Network with Potential Functions , 2003 .

[24]  B. Todorovic,et al.  Sequential growing and pruning of radial basis function network , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[25]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[26]  Bo Yang,et al.  Evolving Hierarchical RBF Neural Networks for Breast Cancer Detection , 2006, ICONIP.

[27]  Hava T. Siegelmann,et al.  Support Vector Clustering , 2002, J. Mach. Learn. Res..

[28]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[29]  Visakan Kadirkamanathan,et al.  A Function Estimation Approach to Sequential Learning with Neural Networks , 1993, Neural Computation.

[30]  Krishna R. Pattipati,et al.  Fault Diagnosis in Mixed-Signal Circuits via Neural-Network based classification Algorithms , 2001 .

[31]  Sheng Chen,et al.  Regularized orthogonal least squares algorithm for constructing radial basis function networks , 1996 .

[32]  Visakan Kadirkamanathan,et al.  Statistical Control of RBF-like Networks for Classification , 1997, ICANN.

[33]  Martin D. Buhmann,et al.  Radial Basis Functions , 2021, Encyclopedia of Mathematical Geosciences.

[34]  Kemal Polat,et al.  A cascade learning system for classification of diabetes disease: Generalized Discriminant Analysis and Least Square Support Vector Machine , 2008, Expert Syst. Appl..

[35]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[36]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[37]  Héctor Pomares,et al.  Time series analysis using normalized PG-RBF network with regression weights , 2002, Neurocomputing.

[38]  Mark J. L. Orr,et al.  Regularization in the Selection of Radial Basis Function Centers , 1995, Neural Computation.

[39]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[40]  Nicolaos B. Karayiannis,et al.  Growing radial basis neural networks: merging supervised and unsupervised learning with network growth techniques , 1997, IEEE Trans. Neural Networks.

[41]  Yonggwan Won,et al.  An Improvement of Extreme Learning Machine for Compact Single-Hidden-Layer Feedforward Neural Networks , 2008, Int. J. Neural Syst..

[42]  Guang-Bin Huang,et al.  Convex incremental extreme learning machine , 2007, Neurocomputing.