An incremental neural network with a reduced architecture

This paper proposes a technique, called Evolving Probabilistic Neural Network (ePNN), that presents many interesting features, including incremental learning, evolving architecture, the capacity to learn continually throughout its existence and requiring that each training sample be used only once in the training phase without reprocessing. A series of experiments was performed on data sets in the public domain; the results indicate that ePNN is superior or equal to the other incremental neural networks evaluated in this paper. These results also demonstrate the advantage of the small ePNN architecture and show that its architecture is more stable than the other incremental neural networks evaluated. ePNN thus appears to be a promising alternative for a quick learning system and a fast classifier with a low computational cost.

[1]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[2]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[3]  David A. Landgrebe,et al.  Analyzing high-dimensional multispectral data , 1993, IEEE Trans. Geosci. Remote. Sens..

[4]  Lei Chen,et al.  Enhanced random search based incremental extreme learning machine , 2008, Neurocomputing.

[5]  Vasant Honavar,et al.  Learn++: an incremental learning algorithm for supervised neural networks , 2001, IEEE Trans. Syst. Man Cybern. Part C.

[6]  Jun Tani,et al.  Incremental learning of sequence patterns with a modular network model , 2009, Neurocomputing.

[7]  Michael J. Watts,et al.  A Decade of Kasabov's Evolving Connectionist Systems: A Review , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[8]  Paulo Martins Engel,et al.  IPNN: An Incremental Probabilistic Neural Network for Function Approximation and Regression Tasks , 2010, 2010 Eleventh Brazilian Symposium on Neural Networks.

[9]  Ethem Alpaydin,et al.  GAL: Networks That Grow When They Learn and Shrink When They Forget , 1994, Int. J. Pattern Recognit. Artif. Intell..

[10]  Toshio Fukuda,et al.  A neural network architecture for incremental learning , 1995, Neurocomputing.

[11]  David G. Stork,et al.  Pattern Classification , 1973 .

[12]  B. Tudu,et al.  Incremental PNN classifier for a versatile electronic nose , 2008, 2008 3rd International Conference on Sensing Technology.

[13]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[14]  P. R. Bevington,et al.  Data Reduction and Error Analysis for the Physical Sciences , 1969 .

[15]  Christophe G. Giraud-Carrier,et al.  A Note on the Utility of Incremental Learning , 2000, AI Commun..

[16]  Jonathan Lee,et al.  A new ARTMAP-based neural network for incremental learning , 2006, Neurocomputing.

[17]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[18]  D. F. Specht,et al.  Probabilistic neural networks for classification, mapping, or associative memory , 1988, IEEE 1988 International Conference on Neural Networks.

[19]  George K. Papakonstantinou,et al.  Mixture Density Estimation Based on Maximum Likelihood and Sequential Test Statistics , 1999, Neural Processing Letters.

[20]  Yukio-Pegio Gunji,et al.  Recurrent neural network architecture with pre-synaptic inhibition for incremental learning , 2006, Neural Networks.

[21]  Nikola Kasabov,et al.  Evolving connectionist systems , 2002 .

[22]  Elias Oliveira,et al.  An Evolving System Based on Probabilistic Neural Network , 2010, 2010 Eleventh Brazilian Symposium on Neural Networks.

[23]  Fred Henrik Hamker,et al.  Life-long learning Cell Structures--continuously learning without catastrophic interference , 2001, Neural Networks.

[24]  J. B. Gomm,et al.  An adaptive neural network for on-line learning and diagnosis of process faults , 1995 .

[25]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[26]  Nikola Kasabov,et al.  Evolving Connectionist Systems: The Knowledge Engineering Approach , 2007 .

[27]  Zhihua Zhang,et al.  EM algorithms for Gaussian mixtures with split-and-merge operation , 2003, Pattern Recognition.

[28]  Nikola Kasabov,et al.  Evolving Connectionist Systems: Methods and Applications in Bioinformatics, Brain Study and Intelligent Machines , 2002, IEEE Transactions on Neural Networks.