Handwritten digit recognition by neural networks with single-layer training

It is shown that neural network classifiers with single-layer training can be applied efficiently to complex real-world classification problems such as the recognition of handwritten digits. The STEPNET procedure, which decomposes the problem into simpler subproblems which can be solved by linear separators, is introduced. Provided appropriate data representations and learning rules are used, performance comparable to that obtained by more complex networks can be achieved. Results from two different databases are presented: an European database comprising 8700 isolated digits and a zip code database from the US Postal Service comprising 9000 segmented digits. A hardware implementation of the classifier is briefly described.

[1]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[2]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[3]  Guo-Zheng Sun,et al.  A Novel Net that Learns Sequential Decision Process , 1987, NIPS.

[4]  John S. Denker,et al.  Strategies for Teaching Layered Networks Classification Tasks , 1987, NIPS.

[5]  Paul E. Utgoff,et al.  Perceptron Trees : A Case Study in ybrid Concept epresentations , 1999 .

[6]  Etienne Barnard,et al.  A comparison between criterion functions for linear classifiers, with an application to neural nets , 1989, IEEE Trans. Syst. Man Cybern..

[7]  Gérard Dreyfus,et al.  Single-layer learning revisited: a stepwise procedure for building and training a neural network , 1989, NATO Neurocomputing.

[8]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[9]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[10]  Isabelle Guyon,et al.  Handwritten Digit Recognition: Applications of Neural Net Chips and Automatic Learning , 1989, NATO Neurocomputing.

[11]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[12]  Yves Chauvin.  Generalization Dynamics in LMS Trained Linear Networks , 1990, NIPS.

[13]  R. Lee,et al.  Reconfigurable Neural Net Chip with 32K Connections , 1990, NIPS.

[14]  J. Ouali,et al.  Design and implementation of a dedicated neural network for handwritten digit recognition , 1991, Euro ASIC '91.

[15]  L.D. Jackel,et al.  An analog neural network processor and its application to high-speed character recognition , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[16]  James A. Pittman,et al.  Recognizing Hand-Printed Letters and Digits Using Backpropagation Learning , 1991, Neural Computation.

[17]  Terence D. Sanger,et al.  A Tree-Structured Algorithm for Reducing Computation in Networks with Separable Basis Functions , 1991, Neural Computation.

[18]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.