The Diabolo Classifier

We present a new classification architecture based on autoassociative neural networks that are used to learn discriminant models of each class. The proposed architecture has several interesting properties with respect to other model-based classifiers like nearest-neighbors or radial basis functions: it has a low computational complexity and uses a compact distributed representation of the models. The classifier is also well suited for the incorporation of a priori knowledge by means of a problem-specific distance measure. In particular, we will show that tangent distance (Simard, Le Cun, & Denker, 1993) can be used to achieve transformation invariance during learning and recognition. We demonstrate the application of this classifier to optical character recognition, where it has achieved state-of-the-art results on several reference databases. Relations to other models, in particular those based on principal component analysis, are also discussed.

[1]  Bernard Widrow,et al.  The "rubber-mask" technique - I. Pattern measurement and analysis , 1973, Pattern Recognit..

[2]  D. Burr.  A dynamic model for image registration , 1981 .

[3]  D. J. Burr,et al.  Matching Elastic Templates , 1983 .

[4]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[5]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[6]  Yann LeCun,et al.  Efficient Pattern Recognition Using a New Transformation Distance , 1992, NIPS.

[7]  Patrice Y. Simard,et al.  An efficient algorithm for learning invariance in adaptive classifiers , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[8]  Dean Pomerleau,et al.  Input Reconstruction Reliability Estimation , 1992, NIPS.

[9]  Gilles Burel,et al.  Recognition of handwritten digits by image processing and neural network , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[10]  Garrison W. Cottrell,et al.  Non-Linear Dimensionality Reduction , 1992, NIPS.

[11]  Emmanuel Viennet.  Architectures connexionnistes multi-modulaires. Application a l'analyse de scenes , 1993 .

[12]  Stephen M. Omohundro,et al.  Surface Learning with Applications to Lipreading , 1993, NIPS.

[13]  Harris Drucker,et al.  Boosting Performance in Neural Networks , 1993, Int. J. Pattern Recognit. Artif. Intell..

[14]  Nanda Kambhatla,et al.  Fast Non-Linear Dimension Reduction , 1993, NIPS.

[15]  Patrice Y. Simard.  Efficient Computation of Complex Distance Metrics Using Hierarchical Filtering , 1993, NIPS.

[16]  Françoise Fogelman-Soulié,et al.  Multi-Modular Neural Network Architectures: Applications in Optical Character and Human Face Recognition , 1993, Int. J. Pattern Recognit. Artif. Intell..

[17]  Maurice Milgram,et al.  Transformation Invariant Autoassociation with Application to Handwritten Character Recognition , 1994, NIPS.

[18]  Satoshi Suzuki,et al.  Unsupervised Classification of 3D Objects from 2D Views , 1994, NIPS.

[19]  Geoffrey E. Hinton,et al.  To appear in : Advances in Neural Information Processing Systems , 2007 .

[20]  Alessandro Sperduti,et al.  A Rapid Graph-based Method for Arbitrary Transformation-Invariant Pattern Classification , 1994, NIPS.

[21]  Patrice Y. Simard,et al.  Learning Prototype Models for Tangent Distance , 1994, NIPS.

[22]  Geoffrey E. Hinton,et al.  Recognizing Handwritten Digits Using Mixtures of Linear Models , 1994, NIPS.

[23]  Nathalie Japkowicz,et al.  A Novelty Detection Approach to Classification , 1995, IJCAI.

[24]  B. Lamy.  Reconnaissance de caracteres manuscrits par combinaison de modeles connexionnistes , 1995 .

[25]  Holger Schwenk,et al.  Learning Discriminant Tangent Models for Handwritten Character Recognition , 1995 .

[26]  Harris Drucker,et al.  Comparison of learning algorithms for handwritten digit recognition , 1995 .

[27]  Paolo Frasconi,et al.  Learning in multilayered networks used as autoassociators , 1995, IEEE Trans. Neural Networks.

[28]  Geoffrey E. Hinton,et al.  Using Generative Models for Handwritten Digit Recognition , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Marco Gori,et al.  A neural network-based model for paper currency recognition and verification , 1996, IEEE Trans. Neural Networks.

[30]  Maurice Milgram,et al.  Constraint tangent distance for on-line character recognition , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[31]  Yoav Freund,et al.  Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[32]  Geoffrey E. Hinton,et al.  Modeling the manifolds of images of handwritten digits , 1997, IEEE Trans. Neural Networks.

[33]  Geoffrey E. Hinton,et al.  Instantiating Deformable Models with a Neural Net , 1997, Comput. Vis. Image Underst..

[34]  Michael E. Tipping,et al.  Mixtures of Principal Component Analysers , 1997 .

[35]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.