Constructing and training feed-forward neural networks for pattern classification

A new approach of constructing and training neural networks for pattern classification is proposed. Data clusters are generated and trained sequentially based on distinct local subsets of the training data. Obtained clusters are then used to construct a feed-forward network, which is further trained using standard algorithms operating on the global training set. The network obtained using this approach effectively inherits the knowledge from the local training procedure before improving on its generalization ability through the subsequent global training. Various experiments demonstrate the superiority of this approach over competing methods.

[1]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[2]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[3]  D. F. Specht,et al.  Enhancements to probabilistic neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[4]  Witold Pedrycz,et al.  Conditional fuzzy clustering in the design of radial basis function neural networks , 1998, IEEE Trans. Neural Networks.

[5]  Dietmar Heinke,et al.  Comparing neural networks: a benchmark on growing neural gas, growing cell structures, and fuzzy ARTMAP , 1998, IEEE Trans. Neural Networks.

[6]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[7]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[8]  Rajeev Kumar,et al.  Multiobjective genetic algorithm partitioning for hierarchical learning of high-dimensional pattern spaces: a learning-follows-decomposition strategy , 1998, IEEE Trans. Neural Networks.

[9]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[10]  C. L. Scofield,et al.  Into silicon: real time learning in a high density RBF neural network , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[11]  Tom Downs,et al.  CARVE-a constructive algorithm for real-valued examples , 1998, IEEE Trans. Neural Networks.

[12]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[13]  Eric B. Baum,et al.  Constructing Hidden Units Using Examples and Queries , 1990, NIPS.

[14]  Stephen Grossberg,et al.  Studies of mind and brain , 1982 .

[15]  Sven Behnke,et al.  Competitive neural trees for pattern classification , 1998, IEEE Trans. Neural Networks.

[16]  Pietro Burrascano,et al.  Learning vector quantization for the probabilistic neural network , 1991, IEEE Trans. Neural Networks.

[17]  D. L. Reilly,et al.  A neural model for category learning , 1982, Biological Cybernetics.

[18]  Robert F. Ling,et al.  Cluster analysis algorithms for data reduction and classification of objects , 1981 .

[19]  Mohamad T. Musavi,et al.  On the training of radial basis function classifiers , 1992, Neural Networks.

[20]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[21]  Noel E. Sharkey,et al.  An Analysis of Catastrophic Interference , 1995, Connect. Sci..

[22]  K. Lang,et al.  Learning to tell two spirals apart , 1988 .

[23]  Thierry Denoeux,et al.  Initializing back propagation networks with prototypes , 1993, Neural Networks.

[24]  Bernard Widrow,et al.  Improving the learning speed of 2-layer neural networks by choosing initial values of the adaptive weights , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[25]  Andrew R. Webb,et al.  Shape-adaptive radial basis functions , 1998, IEEE Trans. Neural Networks.

[26]  Scott E. Fahlman,et al.  An empirical study of learning speed in back-propagation networks , 1988 .

[27]  Richard O. Duda,et al.  Pattern Classification by Iteratively Determined Linear and Piecewise Linear Discriminant Functions , 1966, IEEE Trans. Electron. Comput..

[28]  A. S. Noetzel,et al.  The piecewise linear neural network: training and recognition , 1990, 1990 IJCNN International Joint Conference on Neural Networks.