Arabic Handwritten Digit Recognition Based on Restricted Boltzmann Machine and Convolutional Neural Networks

Handwritten digit recognition is an open problem in computer vision and pattern recognition, and solving this problem has elicited increasing interest. The main challenge of this problem is the design of an efficient method that can recognize the handwritten digits that are submitted by the user via digital devices. Numerous studies have been proposed in the past and in recent years to improve handwritten digit recognition in various languages. Research on handwritten digit recognition in Arabic is limited. At present, deep learning algorithms are extremely popular in computer vision and are used to solve and address important problems, such as image classification, natural language processing, and speech recognition, to provide computers with sensory capabilities that reach the ability of humans. In this study, we propose a new approach for Arabic handwritten digit recognition by use of restricted Boltzmann machine (RBM) and convolutional neural network (CNN) deep learning algorithms. In particular, we propose an Arabic handwritten digit recognition approach that works in two phases. First, we use the RBM, which is a deep learning technique that can extract highly useful features from raw data, and which has been utilized in several classification problems as a feature extraction technique in the feature extraction phase. Then, the extracted features are fed to an efficient CNN architecture with a deep supervised learning architecture for the training and testing process. In the experiment, we used the CMATERDB 3.3.1 Arabic handwritten digit dataset for training and testing the proposed method. Experimental results show that the proposed method significantly improves the accuracy rate, with accuracy reaching 98.59%. Finally, comparison of our results with those of other studies on the CMATERDB 3.3.1 Arabic handwritten digit dataset shows that our approach achieves the highest accuracy rate.

[1]  Ganapati Panda,et al.  Unconstrained handwritten digit recognition using perceptual shape primitives , 2016, Pattern Analysis and Applications.

[2]  X. Li,et al.  Application of a new Restricted Boltzmann Machine to Radar Target Recognition , 2016, 2016 Progress in Electromagnetic Research Symposium (PIERS).

[3]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[4]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[5]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[6]  Ezzat El-Sherif,et al.  Arabic handwritten digit recognition , 2008, International Journal of Document Analysis and Recognition (IJDAR).

[7]  Abdul Kawsar Tushar,et al.  Handwritten Arabic numeral recognition using deep learning neural networks , 2017, 2017 IEEE International Conference on Imaging, Vision & Pattern Recognition (icIVPR).

[8]  Xiaola Lin,et al.  Feature extraction using Restricted Boltzmann Machine for stock price prediction , 2012, 2012 IEEE International Conference on Computer Science and Automation Engineering (CSAE).

[9]  Xiuping Jia,et al.  Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[10]  João Paulo Papa,et al.  Model selection for Discriminative Restricted Boltzmann Machines through meta-heuristic techniques , 2015, J. Comput. Sci..

[11]  Sameh M. Awaidah,et al.  A multiple feature/resolution scheme to Arabic (Indian) numerals recognition using hidden Markov models , 2009, Signal Process..

[12]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[13]  Sabri A. Mahmoud,et al.  Recognition of writer-independent off-line handwritten Arabic (Indian) numerals using hidden Markov models , 2008, Signal Process..

[14]  M. Szarvas,et al.  Pedestrian detection with convolutional neural networks , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[15]  Haiying Huang,et al.  Comparison of different variants of Restricted Boltzmann Machines , 2014, Proceedings of 2nd International Conference on Information Technology and Electronic Commerce.

[16]  Ching Y. Suen,et al.  A novel hybrid CNN-SVM classifier for recognizing handwritten digits , 2012, Pattern Recognit..

[17]  Sebastiano Impedovo,et al.  A novel prototype generation technique for handwriting digit recognition , 2014, Pattern Recognit..

[18]  S. V. Rajashekararadhya,et al.  Isolated Handwritten Kannada and Tamil Numeral Recognition: A Novel Approach , 2008, 2008 First International Conference on Emerging Trends in Engineering and Technology.

[19]  Fairouz Lekhal,et al.  Arabic Numerals Recognition based on an Improved Version of the Loci Characteristic , 2011 .

[20]  Venu Govindaraju,et al.  Segmentation of Arabic Handwriting Based on both Contour and Skeleton Segmentation , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[21]  Faruq A. Al-Omari,et al.  Handwritten Indian numerals recognition system using probabilistic neural networks , 2004, Adv. Eng. Informatics.

[22]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[23]  Abdelhak Boukharouba,et al.  Novel feature extraction technique for the recognition of handwritten digits , 2017 .

[24]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[25]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[26]  Sudip Saha,et al.  Handwritten Arabic Numeral Recognition using a Multi Layer Perceptron , 2010, ArXiv.

[27]  S LewMichael,et al.  Deep learning for visual understanding , 2016 .

[28]  Ching Y. Suen,et al.  A trainable feature extractor for handwritten digit recognition , 2007, Pattern Recognit..

[29]  Soo-Young Lee,et al.  Color image processing based on Nonnegative Matrix Factorization with Convolutional Neural Network , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).