Application of image retrieval for aesthetic evaluation and improvement suggestion of isolated Bangla handwritten characters

Bangla is one of the most widely used languages worldwide. This paper presents an application of image retrieval techniques to automatically judge the aesthetic quality of handwritten Bangla isolated characters. Retrieval techniques are also adapted to give improvement suggestions, with a plan to incorporate the methods in applications which can assist in learning/teaching handwriting. The proposed method borrows key concepts from content-based image retrieval. Our method was tested on the BanglaLekha-Isolated data set, which contains images of 84 Bangla characters, with nearly 2000 samples per character. The data set contains evaluation of the aesthetic quality of the handwriting judged on a scale of 1–5. For this work, the dataset was partitioned into a test set of 400 images and a database-set of ≈ 1600 images, per Bangla character. Assuming that a scoring difference of 1 is acceptable, the proposed method achieves an accuracy of 77.39% when using features extracted by a convolutional neural network based autoencoder. Experiments were also done with the popular HOG feature. However, the autoencoder-based results showed clear superiority compared the HOG-based results. Our proposed method for improvement suggestions also shows that it is possible to shows samples from the dataset which will help users improve their handwriting while requiring small changes to their own handwriting.

[1]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[2]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[3]  L. Deng,et al.  The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web] , 2012, IEEE Signal Processing Magazine.

[4]  Rafiqul Islam,et al.  BanglaLekha-Isolated: A multi-purpose comprehensive dataset of Handwritten Bangla Isolated characters , 2017, Data in brief.

[5]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[6]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[7]  Joanne Phelps,et al.  The Children's Handwriting Scale: A New Diagnostic Tool , 1985 .

[8]  M. Ediger.  Assessing Handwriting Achievement. , 2002 .

[9]  Bidyut Baran Chaudhuri,et al.  Handwritten Numeral Databases of Indian Scripts and Multistage Recognition of Mixed Numerals , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[11]  Thierry Pun,et al.  Performance evaluation in content-based image retrieval: overview and proposals , 2001, Pattern Recognit. Lett..

[12]  R. A. Young,et al.  Teaching Fluent Handwriting Remediates Many Reading-Related Learning Disabilities , 2015 .

[13]  B. Tuck,et al.  Effects of Achievement Expectations and Handwriting Quality on Scoring Essays. , 1983 .

[14]  Hans-Peter Kriegel,et al.  State-of-the-Art in Content-Based Image and Video Retrieval , 2001, Computational Imaging and Vision.

[15]  Oscar Déniz-Suárez,et al.  Face recognition using Histograms of Oriented Gradients , 2011, Pattern Recognit. Lett..

[16]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[17]  Lewis D. Griffin,et al.  Multiscale Histogram of Oriented Gradient Descriptors for Robust Character Recognition , 2011, 2011 International Conference on Document Analysis and Recognition.