A multi-net local learning framework for pattern recognition

This paper proposes a general local learning framework to effectively alleviate the complexities of classifier design by means of "divide and conquer" principle and ensemble method. The learning framework consists of quantization layer and ensemble layer. After GLVQ and MLP are applied to the framework, the proposed method is tested on MNIST handwritten digit database. The obtained performance is very promising, an error rate with 0.99%, which is comparable to that of LeNet5, one of the best classifiers on this database. Further, in contrast to LeNet5, our method is especially suitable for a large-scale real-world classification problem.

[1]  Jian-xiong Dong,et al.  A Fast SVM Training Algorithm , 2003, Int. J. Pattern Recognit. Artif. Intell..

[2]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[3]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[4]  Atsushi Sato,et al.  A formulation of learning vector quantization using a new misclassification measure , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[5]  A. Sato,et al.  An Analysis of Convergence in Generalized LVQ , 1998 .

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  Amanda J. C. Sharkey,et al.  Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems , 1999 .

[8]  Adam Krzyżak,et al.  Local learning framework for handwritten character recognition , 2002 .

[9]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[10]  Amanda J. C. Sharkey,et al.  Multi-Net Systems , 1999 .

[11]  Biing-Hwang Juang,et al.  Discriminative learning for minimum error classification [pattern recognition] , 1992, IEEE Trans. Signal Process..

[12]  Jian-xiong Dong,et al.  Local Learning Framework for Recognition of Lowercase Handwritten Characters , 2001, MLDM.

[13]  Kagan Tumer,et al.  Linear and Order Statistics Combiners for Pattern Classification , 1999, ArXiv.

[14]  Meng Shi,et al.  Handwritten numeral recognition using gradient and curvature of gray scale image , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[15]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[16]  Adam Krzyzak,et al.  A Local Learning Framework for Pattern Classification , 2022 .

[17]  L. Bottou,et al.  COMPARISON OF LEARNING ALGORITHMS FOR , 1995 .

[18]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[19]  Paul S. Bradley,et al.  Refining Initial Points for K-Means Clustering , 1998, ICML.

[20]  Arnaldo Spalvieri,et al.  Quantizing for minimum average misclassification risk , 1998, IEEE Trans. Neural Networks.

[21]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[22]  Arnaldo Spalvieri,et al.  Certain facts about Kohonen's LVQ1 algorithm , 1996 .

[23]  Jian-xiong Dong,et al.  A Fast Parallel Optimization for Training Support Vector Machine , 2003, MLDM.

[24]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[25]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .