Clustered class-dependant training method for digit recognition classifiers

This paper presents a convolutional neural network clustering approach for handwritten digits recognition. Neural networks were trained individually, using the same training set and combined into clusters, depending on the training method used. These clusters formed a layered architecture, where each layer attempted to recognize the given digit, when the previous layers were not able to do so with sufficient certainty. We examine various ways of combining such clusters and training their constituent networks.

[1]  Bernhard Schölkopf,et al.  Training Invariant Support Vector Machines , 2002, Machine Learning.

[2]  Hermann Ney,et al.  Local context in non-linear deformation models for handwritten character recognition , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[3]  Harry Wechsler,et al.  Face recognition using hybrid classifier systems , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[4]  Jude W. Shavlik,et al.  Combining the Predictions of Multiple Classifiers: Using Competitive Learning to Initialize Neural Networks , 1995, IJCAI.

[5]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[6]  David H. Wolpert,et al.  Stacked generalization , 1992, Neural Networks.

[7]  Luca Maria Gambardella,et al.  Deep, Big, Simple Neural Nets for Handwritten Digit Recognition , 2010, Neural Computation.

[8]  Lars Kai Hansen,et al.  Ensemble methods for handwritten digit recognition , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[9]  Anders Krogh,et al.  Learning with ensembles: How overfitting can be useful , 1995, NIPS.

[10]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  R. Clemen.  Combining forecasts: A review and annotated bibliography , 1989 .

[12]  Nathan Intrator,et al.  Classification of seismic signals by integrating ensembles of neural networks , 1998, IEEE Trans. Signal Process..

[13]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .