Image augmentation by blocky artifact in Deep Convolutional Neural Network for handwritten digit recognition

Deep Convolutional Neural Networks - also known as DCNN - are powerful models for different visual pattern classification problems. Many works in this field use image augmentation at the training phase to achieve better accuracy. This paper presents blocky artifact as an augmentation technique to increase the accuracy of DCNN for handwritten digit recognition, both English and Bangla digits, i.e., 0–9. This paper conducts a number of experiments on three different datasets: MNIST Dataset, CMATERDB 3.1.1 Dataset and Indian Statistical Institute (ISI) Dataset. For each dataset, DCNNs with the proposed augmentation technique give better results than those without such augmentation. Unsupervised pre-training with the blocky artifact achieves 99.56%, 99.83% and 99.35% accuracy respectively on MNIST, CMATERDDB and ISI datasets producing, in the process, so far the best accuracy rate for CMATERDB and ISI datasets.

[1]  M. A. H. Akhand,et al.  Convolutional neural network training with artificial pattern for Bangla handwritten numeral recognition , 2016, 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV).

[2]  Prabhat,et al.  Artificial Neural Network , 2018, Encyclopedia of GIS.

[3]  Mahantapas Kundu,et al.  A genetic algorithm based region sampling for selection of local features in handwritten digit recognition application , 2012, Appl. Soft Comput..

[4]  Shengen Yan,et al.  Deep Image: Scaling up Image Recognition , 2015, ArXiv.

[5]  Xiaolin Hu,et al.  Recurrent convolutional neural network for object recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Khawza I. Ahmed,et al.  Handwritten Bangla digit recognition using Sparse Representation Classifier , 2014, 2014 International Conference on Informatics, Electronics & Vision (ICIEV).

[7]  Md Shopon,et al.  Bangla handwritten digit recognition using autoencoder and deep convolutional neural network , 2016, 2016 International Workshop on Computational Intelligence (IWCI).

[8]  Yann LeCun,et al.  The mnist database of handwritten digits , 2005 .

[9]  Zhuowen Tu,et al.  Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree , 2015, AISTATS.

[10]  Subhadip Basu,et al.  A novel framework for automatic sorting of postal documents with multi-script address blocks , 2010, Pattern Recognit..

[11]  Bidyut Baran Chaudhuri,et al.  Handwritten Numeral Databases of Indian Scripts and Multistage Recognition of Mixed Numerals , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Mohammad Shorif Uddin,et al.  Hand Written Bangla Numerals Recognition for Automated Postal System , 2013 .

[13]  Tasnuva Hassan,et al.  Handwritten Bangla numeral recognition using Local Binary Pattern , 2015, 2015 International Conference on Electrical Engineering and Information Communication Technology (ICEEICT).

[14]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[15]  Ying Wen,et al.  A classifier for Bangla handwritten numeral recognition , 2012, Expert Syst. Appl..

[16]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  D. George,et al.  A hierarchical Bayesian model of invariant pattern recognition in the visual cortex , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[19]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[20]  Isabelle Guyon,et al.  Comparison of classifier methods: a case study in handwritten digit recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[21]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[22]  Mahantapas Kundu,et al.  A statistical-topological feature combination for recognition of handwritten numerals , 2012, Appl. Soft Comput..

[23]  Geoffrey E. Hinton,et al.  Using very deep autoencoders for content-based image retrieval , 2011, ESANN.

[24]  Luca Maria Gambardella,et al.  Deep, Big, Simple Neural Nets for Handwritten Digit Recognition , 2010, Neural Computation.

[25]  Luca Maria Gambardella,et al.  Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition , 2010, ArXiv.

[26]  Charu C. Aggarwal,et al.  Neural Networks and Deep Learning , 2018, Springer International Publishing.

[27]  Mita Nasipuri,et al.  A multi-objective approach towards cost effective isolated handwritten Bangla character and digit recognition , 2016, Pattern Recognit..

[28]  Yong-Sheng Chen,et al.  Batch-normalized Maxout Network in Network , 2015, ArXiv.

[29]  Yue Lu,et al.  Handwritten Bangla digit recognition using hierarchical Bayesian network , 2008, 2008 3rd International Conference on Intelligent System and Knowledge Engineering.

[30]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[31]  Mahantapas Kundu,et al.  Handwritten Bangla Digit Recognition Using Classifier Combination Through DS Technique , 2005, PReMI.

[32]  M. M. Hafizur Rahman,et al.  Bangla Handwritten Character Recognition using Convolutional Neural Network , 2015 .

[33]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Kensuke Yokoi,et al.  APAC: Augmented PAttern Classification with Neural Networks , 2015, ArXiv.

[35]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  M. Nasipuri,et al.  A new quad tree based feature set for recognition of handwritten bangla numerals , 2012, 2012 IEEE International Conference on Engineering Education: Innovative Practices and Future Trends (AICERA).