Modular Neural Network Design For The Problem Of Alphabetic Character Recognition

This paper reports on an experimental approach to nd a modularized articial neural network solution for the UCI letters recognition problem. Our experiments have been carried out in two parts. We investigate directed task decomposition using expert knowledge and clustering approaches to nd the subtasks for the modules of the network. We next investigate processes to combine the modules eectiv ely in a single decision process. After having found suitable modules through task decomposition we have found through further experimentation that when the modules are combined with decision tree supervision, their functional error is reduced signican tly to improve their combination through the decision process that has been implemented as a small multilayered perceptron. The experiments conclude with a modularized neural network design for this classication problem that has increased learning and generalization characteristics. The test results for this network are markedly better than a single or stand alone network that has a fully connected topology.

[1]  J. R. Quinlan,et al.  Comparing connectionist and symbolic learning methods , 1994, COLT 1994.

[2]  Yoshua Bengio,et al.  Gradient-Based Optimization of Hyperparameters , 2000, Neural Computation.

[3]  Haiying Wang,et al.  Feature Decomposition Architectures for Neural Networks: Algorithms, Error Bounds, and Applications , 2002, Int. J. Neural Syst..

[4]  Philip A. Collier,et al.  CHARACTERISTICS OF DATA SUITABLE FOR LEARNING WITH CONNECTIONIST AND SYMBOLIC METHODS , 1994 .

[5]  Yoshua Bengio,et al.  Boosting Neural Networks , 2000, Neural Computation.

[6]  Amanda J. C. Sharkey,et al.  Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems , 1999 .

[7]  Michael I. Jordan,et al.  Task Decomposition Through Competition in a Modular Connectionist Architecture: The What and Where Vision Tasks , 1990, Cogn. Sci..

[8]  Anthony V. Robins,et al.  Catastrophic Forgetting, Rehearsal and Pseudorehearsal , 1995, Connect. Sci..

[9]  Sankar K. Pal,et al.  Rough-Fuzzy MLP: Modular Evolution, Rule Generation, and Evaluation , 2003, IEEE Trans. Knowl. Data Eng..

[10]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[11]  Trevor J. M. Bench-Capon Neural networks and open texture , 1993, ICAIL '93.

[12]  Bart L. M. Happel,et al.  Design and evolution of modular neural network architectures , 1994, Neural Networks.

[13]  Mohamed S. Kamel,et al.  Modular neural networks: a survey. , 1999, International journal of neural systems.

[14]  Steven Guan,et al.  Parallel growing and training of neural networks using output parallelism , 2002, IEEE Trans. Neural Networks.

[15]  Russell Greiner,et al.  Computational learning theory and natural learning systems , 1997 .

[16]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.