Evolving Logic Networks With Real-Valued Inputs for Fast Incremental Learning

In this paper, we present a neural network structure and a fast incremental learning algorithm using this network. The proposed network structure, named evolving logic networks for real-valued inputs (ELN-R), is a data structure for storing and using the knowledge. A distinctive feature of ELN-R is that the previously learned knowledge stored in ELN-R can be used as a kind of building block in constructing new knowledge. Using this feature, the proposed learning algorithm can enhance the stability and plasticity at the same time, and as a result, the fast incremental learning can be realized. The performance of the proposed scheme is shown by a theoretical analysis and an experimental study on two benchmark problems.

[1]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[2]  Naohiro Ishii,et al.  Incremental learning methods with retrieving of interfered patterns , 1999, IEEE Trans. Neural Networks.

[3]  Richard Granger,et al.  Beyond Incremental Processing: Tracking Concept Drift , 1986, AAAI.

[4]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[5]  Vasant Honavar,et al.  Learn++: an incremental learning algorithm for supervised neural networks , 2001, IEEE Trans. Syst. Man Cybern. Part C.

[6]  Gregory L. Heileman,et al.  Rademacher penalization applied to fuzzy ARTMAP and boosted ARTMAP , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[7]  A. Kuh,et al.  A smart algorithm for incremental learning , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[8]  Paramasivan Saratchandran,et al.  Performance evaluation of a sequential minimal radial basis function (RBF) neural network learning algorithm , 1998, IEEE Trans. Neural Networks.

[9]  Fred Henrik Hamker,et al.  Life-long learning Cell Structures--continuously learning without catastrophic interference , 2001, Neural Networks.

[10]  LiMin Fu.  Incremental knowledge acquisition in supervised learning networks , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[11]  Simon Haykin,et al.  Neural networks , 1994 .

[12]  Y Lu,et al.  A Sequential Learning Scheme for Function Approximation Using Minimal Radial Basis Function Neural Networks , 1997, Neural Computation.

[13]  Rodney M. Goodman,et al.  Incremental learning with rule-based neural networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[14]  A. G. Constantinides,et al.  An heuristic pattern correction scheme for GRNNs and its application to speech recognition , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[15]  Narasimhan Sundararajan,et al.  A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation , 2005, IEEE Transactions on Neural Networks.

[16]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[17]  John A. Bullinaria,et al.  Evolving improved incremental learning schemes for neural network systems , 2005, 2005 IEEE Congress on Evolutionary Computation.

[18]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[19]  G. Tontini.  Robust learning and identification of patterns in statistical process control charts using a hybrid RBF fuzzy ARTMAP neural network , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[20]  James R. Williamson,et al.  Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps , 1996, Neural Networks.

[21]  R. Brits,et al.  A clustering approach to incremental learning for feedforward neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[22]  Visakan Kadirkamanathan,et al.  A Function Estimation Approach to Sequential Learning with Neural Networks , 1993, Neural Computation.

[23]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[24]  CHEE PENG LIM,et al.  An Incremental Adaptive Network for On-line Supervised Learning and Probability Estimation , 1997, Neural Networks.

[25]  Narasimhan Sundararajan,et al.  An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Minh Tue Vo,et al.  Incremental learning using the time delay neural network , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[27]  Fernando Santos Osório,et al.  INSS: A hybrid system for constructive machine learning , 1999, Neurocomputing.

[28]  Georgios C. Anagnostopoulos,et al.  Ellipsoid ART and ARTMAP for incremental clustering and classification , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[29]  José Carlos Príncipe,et al.  Incremental backpropagation learning networks , 1996, IEEE Trans. Neural Networks.

[30]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.