Handwritten Bangla Digit Recognition Using Deep Learning

In spite of the advances in pattern recognition technology, Handwritten Bangla Character Recognition (HBCR) (such as alpha-numeric and special characters) remains largely unsolved due to the presence of many perplexing characters and excessive cursive in Bangla handwriting. Even the best existing recognizers do not lead to satisfactory performance for practical applications. To improve the performance of Handwritten Bangla Digit Recognition (HBDR), we herein present a new approach based on deep neural networks which have recently shown excellent performance in many pattern recognition and machine learning applications, but has not been throughly attempted for HBDR. We introduce Bangla digit recognition techniques based on Deep Belief Network (DBN), Convolutional Neural Networks (CNN), CNN with dropout, CNN with dropout and Gaussian filters, and CNN with dropout and Gabor filters. These networks have the advantage of extracting and using feature information, improving the recognition of two dimensional shapes with a high degree of invariance to translation, scaling and other pattern distortions. We systematically evaluated the performance of our method on publicly available Bangla numeral image database named CMATERdb 3.1.1. From experiments, we achieved 98.78% recognition rate using the proposed method: CNN with Gabor features and dropout, which outperforms the state-of-the-art algorithms for HDBR.

[1]  Bidyut Baran Chaudhuri,et al.  Automatic Recognition of Unconstrained Off-Line Bangla Handwritten Numerals , 2000, ICMI.

[2]  Luca Maria Gambardella,et al.  Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition , 2010, ArXiv.

[3]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[4]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[5]  Yue Lu,et al.  Handwritten Bangla digit recognition using hierarchical Bayesian network , 2008, 2008 3rd International Conference on Intelligent System and Knowledge Engineering.

[6]  Dan Ciresan,et al.  Multi-Column Deep Neural Networks for offline handwritten Chinese character classification , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[7]  Bidyut Baran Chaudhuri,et al.  Indian script character recognition: a survey , 2004, Pattern Recognit..

[8]  Khawza I. Ahmed,et al.  Handwritten Bangla digit recognition using Sparse Representation Classifier , 2014, 2014 International Conference on Informatics, Electronics & Vision (ICIEV).

[9]  Bidyut B. Chaudhuri A Complete Handwritten Numeral Database of Bangla – A Major Indic Script , 2006 .

[10]  M. M. Hafizur Rahman,et al.  Bangla Handwritten Character Recognition using Convolutional Neural Network , 2015 .

[11]  Subhadip Basu,et al.  Handwritten Bangla Basic and Compound character recognition using MLP and SVM classifier , 2010, ArXiv.

[12]  Yehuda Koren,et al.  Lessons from the Netflix prize challenge , 2007, SKDD.

[13]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[14]  Yoshua Bengio,et al.  Classification using discriminative restricted Boltzmann machines , 2008, ICML '08.

[15]  Ching Y. Suen,et al.  A new benchmark on the recognition of handwritten Bangla and Farsi numeral characters , 2009, Pattern Recognit..

[16]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[17]  Mahantapas Kundu,et al.  A genetic algorithm based region sampling for selection of local features in handwritten digit recognition application , 2012, Appl. Soft Comput..

[18]  Geoffrey E. Hinton,et al.  The "wake-sleep" algorithm for unsupervised neural networks. , 1995, Science.

[19]  Mahantapas Kundu,et al.  A statistical-topological feature combination for recognition of handwritten numerals , 2012, Appl. Soft Comput..

[20]  Bidyut Baran Chaudhuri,et al.  A system towards Indian postal automation , 2004, Ninth International Workshop on Frontiers in Handwriting Recognition.

[21]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Ben Kröse,et al.  Deep Belief Networks for dimensionality reduction , 2008 .

[23]  Coskun Özkan,et al.  The comparison of activation functions for multispectral Landsat TM image classification , 2003 .

[24]  Yoshua Bengio,et al.  Exploring Strategies for Training Deep Neural Networks , 2009, J. Mach. Learn. Res..

[25]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[26]  Geoffrey E. Hinton,et al.  Acoustic Modeling Using Deep Belief Networks , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[27]  Yichuan Tang,et al.  Deep Learning using Linear Support Vector Machines , 2013, 1306.0239.

[28]  Luca Maria Gambardella,et al.  Better Digit Recognition with a Committee of Simple Neural Nets , 2011, 2011 International Conference on Document Analysis and Recognition.

[29]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[30]  Xiaohui Xie,et al.  Handwritten Hangul recognition using deep convolutional neural networks , 2014, International Journal on Document Analysis and Recognition (IJDAR).

[31]  Bidyut Baran Chaudhuri,et al.  A complete printed Bangla OCR system , 1998, Pattern Recognit..

[32]  Vijayan K. Asari,et al.  State Preserving Extreme Learning Machine for face recognition , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[33]  Geoffrey E. Hinton,et al.  Exponential Family Harmoniums with an Application to Information Retrieval , 2004, NIPS.

[34]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[35]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[36]  Lambert Schomaker,et al.  A Comparison of Feature and Pixel-Based Methods for Recognizing Handwritten Bangla Digits , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[37]  Marcus Liwicki,et al.  Comparative Study of Part-Based Handwritten Character Recognition Methods , 2011, 2011 International Conference on Document Analysis and Recognition.

[38]  Umapada Pal,et al.  Touching numeral segmentation using water reservoir concept , 2003, Pattern Recognit. Lett..