Multilayer cluster neural network for totally unconstrained handwritten numeral recognition

Abstract In this paper, we propose a simple multilayer cluster neural network with five independent subnetworks for off-line recognition of totally unconstrained handwritten numerals. We also show that the use of genetic algorithms for avoiding the problem of finding local minima in training the multilayer cluster neural network with gradient descent technique reduces error rates. In order to verify the performance of the proposed multilayer cluster neural network, experiments with unconstrained handwritten numeral database of Concordia University of Canada, that of Electro-Technical Laboratory of Japan, and that of Electronics and Telecommunications Research Institute of Korea were performed. For the case of determining the initial weights of the multilayer cluster neural network randomly, the error rates were 2.90%, 1.50%, and 0.80%, respectively. And, for the case of determining the initial weights using a genetic algorithm, the error rates were 2.20%, 0.87%, and 0.60%, respectively.

[1]  Hiroaki Kitano,et al.  Empirical Studies on the Speed of Convergence of Neural Network Training Using Genetic Algorithms , 1990, AAAI.

[2]  Yann LeCun,et al.  Generalization and network design strategies , 1989 .

[3]  Ching Y. Suen,et al.  A generalized knowledge-based system for the recognition of unconstrained handwritten numerals , 1990, IEEE Trans. Syst. Man Cybern..

[4]  Bernard Lemarié,et al.  Practical Realization of a Radial Basis Function Network for Handwritten Digit Recognition , 1993, IWANN.

[5]  Adam Krzyzak,et al.  Computer Vision and Shape Recognition , 1989, World Scientific Series in Computer Science.

[6]  Charles E. Taylor,et al.  Artificial Life II , 1991 .

[7]  Gérard Dreyfus,et al.  Handwritten digit recognition by neural networks with single-layer training , 1992, IEEE Trans. Neural Networks.

[8]  Theodosios Pavlidis,et al.  A Shape Analysis Model with Applications to a Character Recognition System , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[10]  Oded Maimon,et al.  A Distributed Genetic Algorithm for Neural Network Design and Training , 1992, Complex Syst..

[11]  Harris Drucker,et al.  Boosting Performance in Neural Networks , 1993, Int. J. Pattern Recognit. Artif. Intell..

[12]  Sargur N. Srihari,et al.  Understanding Handwritten Text in a Structured Environment: Determining ZIP Codes from Addresses , 1991, Int. J. Pattern Recognit. Artif. Intell..

[13]  Luigi Stringa,et al.  A New Set of Constraint-Free Character Recognition Grammars , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[15]  Ching Y. Suen,et al.  Contour Tracing and parametric Approximations for Digitized Patterns , 1989, Computer Vision and Shape Recognition.

[16]  Lawrence D. Jackel,et al.  Constrained neural network for unconstrained handwritten digit recognition , 1990 .

[17]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[18]  R. Pfeifer,et al.  Connectionism in Perspective , 1989 .

[19]  G. Winkler,et al.  A combination of statistical and syntactical pattern recognition applied to classification of unconstrained handwritten numerals , 1980, Pattern Recognit..

[20]  Ching Y. Suen,et al.  Structural classification and relaxation matching of totally unconstrained handwritten zip-code numbers , 1988, Pattern Recognit..

[21]  Sandro Ridella,et al.  Global Optimization of Functions with the Interval Genetic Algorithm , 1992, Complex Syst..

[22]  Ching Y. Suen,et al.  Computer recognition of unconstrained handwritten numerals , 1992, Proc. IEEE.