Bangla handwritten digit recognition using autoencoder and deep convolutional neural network

Handwritten digit recognition is a typical image classification problem. Convolutional neural networks, also known as ConvNets, are powerful classification models for such tasks. As different languages have different styles and shapes of their numeral digits, accuracy rates of the models vary from each other and from language to language. However, unsupervised pre-training in such situation has shown improved accuracy for classification tasks, though no such work has been found for Bangla digit recognition. This paper presents the use of unsupervised pre-training using autoencoder with deep ConvNet in order to recognize handwritten Bangla digits, i.e., 0–9. The datasets that are used in this paper are CMATERDB 3.1.1 and a dataset published by the Indian Statistical Institute (ISI). This paper studies four different combinations of these two datasets-two experiments are done against their own training and testing images, other two experiments are done cross validating the datasets. In one of these four experiments, the proposed approach achieves 99.50% accuracy, which is so far the best for recognizing handwritten Bangla digits. The ConvNet model is trained with 19,313 images of ISI handwritten character dataset and tested with images of CMATERDB dataset.

[1]  Tasnuva Hassan,et al.  Handwritten Bangla numeral recognition using Local Binary Pattern , 2015, 2015 International Conference on Electrical Engineering and Information Communication Technology (ICEEICT).

[2]  M. M. Hafizur Rahman,et al.  Bangla Handwritten Character Recognition using Convolutional Neural Network , 2015 .

[3]  Ying Wen,et al.  A classifier for Bangla handwritten numeral recognition , 2012, Expert Syst. Appl..

[4]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[5]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[6]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[7]  Yue Lu,et al.  Handwritten Bangla digit recognition using hierarchical Bayesian network , 2008, 2008 3rd International Conference on Intelligent System and Knowledge Engineering.

[8]  Bidyut Baran Chaudhuri,et al.  Handwritten Numeral Databases of Indian Scripts and Multistage Recognition of Mixed Numerals , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Subhadip Basu,et al.  A novel framework for automatic sorting of postal documents with multi-script address blocks , 2010, Pattern Recognit..

[10]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[11]  Khawza I. Ahmed,et al.  Handwritten Bangla digit recognition using Sparse Representation Classifier , 2014, 2014 International Conference on Informatics, Electronics & Vision (ICIEV).

[12]  M. Nasipuri,et al.  A new quad tree based feature set for recognition of handwritten bangla numerals , 2012, 2012 IEEE International Conference on Engineering Education: Innovative Practices and Future Trends (AICERA).

[13]  Kazuyuki Murase,et al.  Bangla handwritten character recognition using deep belief network , 2014, 2013 International Conference on Electrical Information and Communication Technology (EICT).

[14]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  D. George,et al.  A hierarchical Bayesian model of invariant pattern recognition in the visual cortex , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[16]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[17]  Mahantapas Kundu,et al.  A statistical-topological feature combination for recognition of handwritten numerals , 2012, Appl. Soft Comput..

[18]  Geoffrey E. Hinton,et al.  Using very deep autoencoders for content-based image retrieval , 2011, ESANN.

[19]  M. A. H. Akhand,et al.  Convolutional neural network training with artificial pattern for Bangla handwritten numeral recognition , 2016, 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV).

[20]  Pooja Kamavisdar,et al.  A Survey on Image Classification Approaches and Techniques , 2013 .

[21]  S. Chatterji,et al.  The Origin and Development of the Bengali Language , 2024 .

[22]  Hong Yan,et al.  Rapid feature extraction for Bangla handwritten digit recognition , 2011, 2011 International Conference on Machine Learning and Cybernetics.

[23]  Mahantapas Kundu,et al.  Handwritten Bangla Digit Recognition Using Classifier Combination Through DS Technique , 2005, PReMI.

[24]  Ching Y. Suen,et al.  A new benchmark on the recognition of handwritten Bangla and Farsi numeral characters , 2009, Pattern Recognit..

[25]  Mahantapas Kundu,et al.  A genetic algorithm based region sampling for selection of local features in handwritten digit recognition application , 2012, Appl. Soft Comput..

[26]  Peter Glöckner,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2013 .

[27]  Mita Nasipuri,et al.  A multi-objective approach towards cost effective isolated handwritten Bangla character and digit recognition , 2016, Pattern Recognit..

[28]  Mohammad Shorif Uddin,et al.  Hand Written Bangla Numerals Recognition for Automated Postal System , 2013 .

[29]  Bidyut Baran Chaudhuri,et al.  OCR in Bangla: an Indo-Bangladeshi language , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[30]  Isabelle Guyon,et al.  Comparison of classifier methods: a case study in handwritten digit recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).