Handwriting recognition on form document using convolutional neural network and support vector machines (CNN-SVM)

In this paper, we propose a workflow and a machine learning model for recognizing handwritten characters on form document. The learning model is based on Convolutional Neural Network (CNN) as a powerful feature extraction and Support Vector Machines (SVM) as a high-end classifier. The proposed method is more efficient than modifying the CNN with complex architecture. We evaluated some SVM and found that the linear SVM using L1 loss function and L2 regularization giving the best performance both of the accuracy rate and the computation time. Based on the experiment results using data from NIST SD 192nd edition both for training and testing, the proposed method which combines CNN and linear SVM using L1 loss function and L2 regularization achieved a recognition rate better than only CNN. The recognition rate achieved by the proposed method are 98.85% on numeral characters, 93.05% on uppercase characters, 86.21% on lowercase characters, and 91.37% on the merger of numeral and uppercase characters. While the original CNN achieves an accuracy rate of 98.30% on numeral characters, 92.33% on uppercase characters, 83.54% on lowercase characters, and 88.32% on the merger of numeral and uppercase characters. The proposed method was also validated by using ten folds cross-validation, and it shows that the proposed method still can improve the accuracy rate. The learning model was used to construct a handwriting recognition system to recognize a more challenging data on form document automatically. The pre-processing, segmentation and character recognition are integrated into one system. The output of the system is converted into an editable text. The system gives an accuracy rate of 83.37% on ten different test form document.

[1]  D. I. Sensuse,et al.  Cattle Race Classification Using Gray Level Co-occurrence Matrix Convolutional Neural Networks , 2015 .

[2]  Yann LeCun,et al.  Convolutional neural networks applied to house numbers digit classification , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[3]  Habibollah Haron,et al.  Support Vector Machine (SVM) for English Handwritten Character Recognition , 2010, 2010 Second International Conference on Computer Engineering and Applications.

[4]  Chih-Jen Lin,et al.  Linear and Kernel Classification: When to Use Which? , 2016, SDM.

[5]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[6]  Shruthi,et al.  Offline Handwritten Word Recognition using Multiple Features with SVM Classifier for Holistic Approach , 2015 .

[7]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[8]  Eko Prasetyo,et al.  Data Mining : Mengolah Data Menjadi Informasi Menggunakan Matlab , 2015 .

[9]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[10]  Monji Kherallah,et al.  A New Design Based-SVM of the CNN Classifier Architecture with Dropout for Offline Arabic Handwritten Recognition , 2016, ICCS.

[11]  Ching Y. Suen,et al.  Character Recognition Systems: A Guide for Students and Practitioners , 2007 .

[12]  Benjamin Graham,et al.  Sparse arrays of signatures for online character recognition , 2013, ArXiv.

[13]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[14]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[15]  Satoshi Naoi,et al.  Reconstruction combined training for convolutional neural networks on character recognition , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[16]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[17]  Marcus Liwicki,et al.  DeXpression: Deep Convolutional Neural Network for Expression Recognition , 2015, ArXiv.

[18]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[19]  Chih-Jen Lin,et al.  A dual coordinate descent method for large-scale linear SVM , 2008, ICML '08.

[20]  Bogdan Kwolek,et al.  Face Detection Using Convolutional Neural Networks and Gabor Filters , 2005, ICANN.

[21]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[22]  Mohd Amir,et al.  Automatic classification of handwritten and printed text in ICR boxes , 2014, 2014 IEEE International Advance Computing Conference (IACC).

[23]  Ettyc Juharwidyningsih PENGENALAN KARAKTER TULISAN TANGAN ANGKA DAN OPERATOR MATEMATIKA BERDASARKAN ZERNIKE MOMENTS MENGGUNAKAN PENGGABUNGAN FUZZY C-MEANS – K-NEAREST NEIGHBOR DAN SUPPORT VECTOR MACHINE , 2014 .

[24]  Sumedha B. Hallale,et al.  Twelve Directional Feature Extraction for Handwritten English Character Recognition , 2013 .

[25]  Liu Xiao-hua,et al.  Short-term Load Forecasting Based on Support Vector Machine , 2005 .

[26]  Tushar Patnaik,et al.  Recognition for Handwritten English Letters : A Review , 2013 .

[27]  Sergios Theodoridis,et al.  Machine Learning: A Bayesian and Optimization Perspective , 2015 .

[28]  Li Yao,et al.  Comparative Study of SVM Methods Combined with Voxel Selection for Object Category Classification on fMRI Data , 2011, PloS one.

[29]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[30]  Marti A. Hearst Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..