Adaptive task decomposition and modular multilayer perceptrons for letter recognition

This paper proposes a task decomposition method, which divides a large-scale learning problem into multiple limited-scale pairs of training subsets and cross validation (CV) subsets. Correspondingly, modular multilayer perceptrons are set up. At first, one training subset only consists of its own class and several most neighboring categories, and then some classes in the CV subset are moved into it according to the generalization error of the module. This work presents an empirical formula for selecting the initial number of hidden nodes, and a method for determining the optimal number of hidden units with the help of singular value decomposition. The result for letter recognition shows that the above methods are quite effective.

[1]  Sankar K. Pal,et al.  Data mining in soft computing framework: a survey , 2002, IEEE Trans. Neural Networks.

[2]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[3]  J. R. Parker,et al.  Rank and response combination from confusion matrix data , 2001, Inf. Fusion.

[4]  Masami Ito,et al.  Task decomposition and module combination based on class relations: a modular neural network for pattern classification , 1999, IEEE Trans. Neural Networks.

[5]  Sergio Bermejo,et al.  Oriented principal component analysis for large margin classifiers , 2001, Neural Networks.

[6]  Thierry Denoeux,et al.  Resample and combine: an approach to improving uncertainty representation in evidential pattern classification , 2003, Inf. Fusion.

[7]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[8]  Robert E. Jenkins,et al.  A Simplified Neural-Network Solution through Problem Decomposition: The Case of the Truck Backer-Upper , 1992, Neural Computation.

[9]  R.P. Lippmann,et al.  Pattern classification using neural networks , 1989, IEEE Communications Magazine.

[10]  Rudy Setiono,et al.  Feedforward Neural Network Construction Using Cross Validation , 2001, Neural Computation.

[11]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[12]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[13]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[14]  Kazuyuki Murase,et al.  A new algorithm to design compact two-hidden-layer artificial neural networks , 2001, Neural Networks.

[15]  Chong-Ho Choi,et al.  Input feature selection for classification problems , 2002, IEEE Trans. Neural Networks.

[16]  Franco Scarselli,et al.  Are Multilayer Perceptrons Adequate for Pattern Recognition and Verification? , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Jordi Vitrià,et al.  Using an ICA representation of high dimensional data for object recognition and classification , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[18]  Sandro Ridella,et al.  Circular backpropagation networks for classification , 1997, IEEE Trans. Neural Networks.

[19]  Terence C. Fogarty,et al.  First nearest neighbor classification on Frey and Slate's letter recognition problem , 1992, Machine Learning.

[20]  Mark D. Plumbley.  Algorithms for nonnegative independent component analysis , 2003, IEEE Trans. Neural Networks.

[21]  Sung-Bae Cho,et al.  Neural-network classifiers for recognizing totally unconstrained handwritten numerals , 1997, IEEE Trans. Neural Networks.

[22]  Pascal Vasseur,et al.  Introduction to Multisensor Data Fusion , 2005, The Industrial Information Technology Handbook.

[23]  David J. Slate,et al.  Letter Recognition Using Holland-Style Adaptive Classifiers , 1991, Machine Learning.

[24]  Marilyn Lougher Vaughn.  Derivation of the multilayer perceptron weight constraints for direct network interpretation and knowledge discovery , 1999, Neural Networks.

[25]  James R. Williamson,et al.  Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps , 1996, Neural Networks.

[26]  D. Roverso,et al.  ARTD: Autonomous Recursive Task Decomposition for Many-Class Learning , 2002 .

[27]  P. Gallinari,et al.  Cooperation of neural nets and task decomposition , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[28]  Kishan G. Mehrotra,et al.  Efficient classification for multiclass problems using modular neural networks , 1995, IEEE Trans. Neural Networks.

[29]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..