Farsi/Arabic handwritten digit recognition based on ensemble of SVD classifiers and reliable multi-phase PSO combination rule

The problem of handwritten digit recognition has long been an open problem in the field of pattern classification and of great importance in industry. The heart of the problem lies within the ability to design an efficient algorithm that can recognize digits written and submitted by users via a tablet, scanner, and other digital devices. From an engineering point of view, it is desirable to achieve a good performance within limited resources. To this end, we have developed a new approach for handwritten digit recognition that uses a small number of patterns for training phase. To improve the overall performance achieved in classification task, the literature suggests combining the decision of multiple classifiers rather than using the output of the best classifier in the ensemble; so, in this new approach, an ensemble of classifiers is used for the recognition of handwritten digit. The classifiers used in proposed system are based on singular value decomposition (SVD) algorithm. The experimental results and the literature show that the SVD algorithm is suitable for solving sparse matrices such as handwritten digit. The decisions obtained by SVD classifiers are combined by a novel proposed combination rule which we named reliable multi-phase particle swarm optimization. We call the method “Reliable” because we have introduced a novel reliability parameter which is applied to tackle the problem of PSO being trapped in local minima. In comparison with previous methods, one of the significant advantages of the proposed method is that it is not sensitive to the size of training set. Unlike other methods, the proposed method uses just 15 % of the dataset as a training set, while other methods usually use (60–75) % of the whole dataset as the training set. To evaluate the proposed method, we tested our algorithm on Farsi/Arabic handwritten digit dataset. What makes the recognition of the handwritten Farsi/Arabic digits more challenging is that some of the digits can be legally written in different shapes. Therefore, 6000 hard samples (600 samples per class) are chosen by K-nearest neighbor algorithm from the HODA dataset which is a standard Farsi/Arabic digit dataset. Experimental results have shown that the proposed method is fast, accurate, and robust against the local minima of PSO. Finally, the proposed method is compared with state of the art methods and some ensemble classifier based on MLP, RBF, and ANFIS with various combination rules.

[1]  Meng Shi,et al.  Handwritten numeral recognition using gradient and curvature of gray scale image , 2002, Pattern Recognit..

[2]  Tieniu Tan,et al.  Fusion of global and local features for face verification , 2002, Object recognition supported by user interaction for service robots.

[3]  Madasu Hanmandlu,et al.  Input Fuzzy Modeling for the Recognition of Handwritten Hindi Numerals , 2007, Fourth International Conference on Information Technology (ITNG'07).

[4]  Ehsanollah Kabir,et al.  A PSO-based weighting method for linear combination of neural networks , 2010, Comput. Electr. Eng..

[5]  Karim Faez,et al.  Language-Based Feature Extraction Using Template-Matching in Farsi/Arabic Handwritten Numeral Recognition , 2007 .

[6]  Reza Ebrahimpour,et al.  Farsi handwritten digit recognition based on mixture of RBF experts , 2010, IEICE Electron. Express.

[7]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[8]  Jian Yang,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Chong-Ho Choi,et al.  Combined subspace method using global and local features for face recognition , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[10]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Xin Yao,et al.  DIVACE: Diverse and Accurate Ensemble Learning Algorithm , 2004, IDEAL.

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  Ching Y. Suen,et al.  A novel cascade ensemble classifier system with a high recognition performance on handwritten digits , 2007, Pattern Recognit..

[14]  Ching Y. Suen,et al.  Historical review of OCR research and development , 1992, Proc. IEEE.

[15]  Parinya Sanguansat,et al.  Two-Dimensional Principal Component Analysis and Its Extensions , 2012 .

[16]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[17]  Yoshua Bengio,et al.  Reading checks with multilayer graph transformer networks , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[18]  David G. Stork,et al.  Pattern Classification , 1973 .

[19]  Dian Tjondronegoro,et al.  Selecting, Optimizing and Fusing 'Salient' Gabor Features for Facial Expression Recognition , 2009, ICONIP.

[20]  Ching Y. Suen,et al.  A new benchmark on the recognition of handwritten Bangla and Farsi numeral characters , 2009, Pattern Recognit..

[21]  M. Dehghan,et al.  Farsi handwritten character recognition with moment invariants , 1997, Proceedings of 13th International Conference on Digital Signal Processing.

[22]  Karim Faez,et al.  Recognition of handwritten Persian/Arabic numerals by shadow coding and an edited probabilistic neural network , 1995, Proceedings., International Conference on Image Processing.

[23]  Hiroshi Sako,et al.  Handwritten digit recognition: investigation of normalization and feature extraction techniques , 2004, Pattern Recognit..

[24]  C. H. Chen,et al.  Handbook of Pattern Recognition and Computer Vision , 1993 .

[25]  Ching Y. Suen,et al.  A trainable feature extractor for handwritten digit recognition , 2007, Pattern Recognit..

[26]  B. V. K. Vijaya Kumar,et al.  Representational oriented component analysis (ROCA) for face recognition with one sample image per training class , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Ching Y. Suen,et al.  Isolated Handwritten Farsi Numerals Recognition Using Sparse and Over-Complete Representations , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[28]  Ching Y. Suen,et al.  Application of Support Vector Machines for Recognition of Handwritten Arabic/Persian Digits , 2003 .

[29]  Sebastiano Impedovo,et al.  Automatic Bankcheck Processing: A New Engineered System , 1997, Int. J. Pattern Recognit. Artif. Intell..

[30]  Richard Bellman,et al.  Introduction to matrix analysis (2nd ed.) , 1997 .

[31]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Mohammad Rahmati,et al.  Recognition of Persian handwritten digits using image profiles of multiple orientations , 2004, Pattern Recognit. Lett..

[33]  Alireza Alaei,et al.  Using Modified Contour Features and SVM Based Classifier for the Recognition of Persian/Arabic Handwritten Numerals , 2009, 2009 Seventh International Conference on Advances in Pattern Recognition.

[34]  Zhi-Hua Zhou,et al.  Face recognition from a single image per person: A survey , 2006, Pattern Recognit..

[35]  Lars Elden,et al.  Matrix methods in data mining and pattern recognition , 2007, Fundamentals of algorithms.

[36]  S. V. Rajashekararadhya,et al.  Isolated Handwritten Kannada and Tamil Numeral Recognition: A Novel Approach , 2008, 2008 First International Conference on Emerging Trends in Engineering and Technology.

[37]  H. Heinrich R. Bellman, Introduction to Matrix Analysis. XX + 328 S. London 1960. McGraw-Hill. Preis geb. 77s. 6d , 1961 .

[38]  Gavin Brown,et al.  Diversity in neural network ensembles , 2004 .

[39]  H. Devi,et al.  Thresholding: A Pixel-Level Image Processing Methodology Preprocessing Technique for an OCR System for the Brahmi Script , 2006 .

[40]  Sarunas Raudys,et al.  On Dimensionality, Sample Size, Classification Error, and Complexity of Classification Algorithm in Pattern Recognition , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Saeed Mozaffari,et al.  Structural decomposition and statistical description of Farsi/Arabic handwritten numeric characters , 2005, Eighth International Conference on Document Analysis and Recognition (ICDAR'05).

[42]  Hermann Ney,et al.  Deformation Models for Image Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Karim Faez,et al.  Recognition of isolated handwritten Persian/Arabic characters and numerals using support vector machines , 2003, 2003 IEEE XIII Workshop on Neural Networks for Signal Processing (IEEE Cat. No.03TH8718).

[44]  Donald H. Foley Considerations of sample and feature size , 1972, IEEE Trans. Inf. Theory.

[45]  Ching Y. Suen,et al.  Sorting and Recognizing Cheques and Financial Documents , 1998, Document Analysis Systems.

[46]  Daoqiang Zhang,et al.  ( 2 D ) 2 PCA : 2-Directional 2-Dimensional PCA for Efficient Face Representation and Recognition , 2005 .

[47]  James C. Bezdek,et al.  Decision templates for multiple classifier fusion: an experimental comparison , 2001, Pattern Recognit..

[48]  Hiromitsu Yamada,et al.  Optical Character Recognition , 1999 .

[49]  Cheng-Lin Liu,et al.  Handwritten digit recognition: benchmarking of state-of-the-art techniques , 2003, Pattern Recognit..

[50]  Abdesselam Bouzerdoum,et al.  A combined method for Persian and Arabic handwritten digit recognition , 1996, 1996 Australian New Zealand Conference on Intelligent Information Systems. Proceedings. ANZIIS 96.

[51]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[52]  Daoqiang Zhang,et al.  (2D)2PCA: Two-directional two-dimensional PCA for efficient face representation and recognition , 2005, Neurocomputing.

[53]  Hiroshi Sako,et al.  Class-specific feature polynomial classifier for pattern classification and its application to handwritten numeral recognition , 2006, Pattern Recognit..

[54]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[55]  Richard Bellman,et al.  Introduction to Matrix Analysis , 1972 .

[56]  Venu Govindaraju,et al.  Segmentation of Arabic Handwriting Based on both Contour and Skeleton Segmentation , 2009, 2009 10th International Conference on Document Analysis and Recognition.

[57]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[58]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[59]  Mario Vento,et al.  Reliability Parameters to Improve Combination Strategies in Multi-Expert Systems , 1999, Pattern Analysis & Applications.

[60]  Alice J. O'Toole,et al.  Low-dimensional representation of faces in higher dimensions of the face space , 1993 .

[61]  Berkant Savas,et al.  Handwritten digit classification using higher order singular value decomposition , 2007, Pattern Recognit..

[62]  Ehsanollah Kabir,et al.  Introducing a very large dataset of handwritten Farsi digits and a study on their varieties , 2007, Pattern Recognit. Lett..

[63]  Ali Aghagolzadeh,et al.  A New Pattern for Handwritten Persian/Arabic Digit Recognition , 2007 .

[64]  Jamshid Shanbehzadeh,et al.  Persian/arabic handwritten word recognition using M-band packet wavelet transform , 2008, Image Vis. Comput..

[65]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[66]  Sargur N. Srihari,et al.  Integration of hand-written address interpretation technology into the United States Postal Service Remote Computer Reader system , 1997, Proceedings of the Fourth International Conference on Document Analysis and Recognition.

[67]  Yann LeCun,et al.  Transformation Invariance in Pattern Recognition - Tangent Distance and Tangent Propagation , 2012, Neural Networks: Tricks of the Trade.

[68]  Karim Faez,et al.  Feature extraction with wavelet transform for recognition of isolated handwritten Farsi/Arabic characters and numerals , 2002, 2002 14th International Conference on Digital Signal Processing Proceedings. DSP 2002 (Cat. No.02TH8628).

[69]  Farhad Faradji,et al.  Language-Based Feature Extraction Using Template-Matching in Farsi/Arabic Handwritten Numeral Recognition , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[70]  Patrice Y. Simard,et al.  Best practices for convolutional neural networks applied to visual document analysis , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[71]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[72]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.