Occluded offline handwritten Chinese character recognition using deep convolutional generative adversarial network and improved GoogLeNet

In this paper, we propose a novel method for recognizing occluded offline handwritten Chinese characters based on deep convolutional generative adversarial network (DCGAN) and improved GoogLeNet. Different from previous methods, our proposed method is capable of inpainting and recognizing occluded characters without needing to know the concrete positions of corrupted regions. First, the generator and discriminator of DCGAN are combined to generate realistic Chinese characters from corrupted images, and the contextual loss and the content loss are further used to inpaint generated images. Finally, we use the improved GoogLeNet with traditional feature extraction methods to recognize the recovered handwritten Chinese characters. The proposed method is evaluated on the extended CASIA-HWDB1.1 dataset for two challenging inpainting tasks with different portions of blocks or random missing pixels. Experimental results show that our method can achieve higher repair rates and higher recognition accuracies than most of existing methods.

[1]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[3]  Jie Zhan,et al.  Comparison of two deep learning methods for ship target recognition with optical remotely sensed data , 2020, Neural Computing and Applications.

[4]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[5]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[6]  Qiang Huo,et al.  Offline recognition of handwritten Chinese characters using Gabor features, CDHMM modeling and MCE training , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  Diego Klabjan,et al.  Generative Adversarial Nets for Multiple Text Corpora , 2017, 2021 International Joint Conference on Neural Networks (IJCNN).

[8]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[9]  Lianwen Jin,et al.  High performance offline handwritten Chinese character recognition using GoogLeNet and directional feature maps , 2015, 2015 13th International Conference on Document Analysis and Recognition (ICDAR).

[10]  Fei Yin,et al.  ICDAR 2013 Chinese Handwriting Recognition Competition , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[11]  Shuicheng Yan,et al.  Generalized Nonconvex Nonsmooth Low-Rank Minimization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[13]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2008, Commun. ACM.

[14]  Andrew Zisserman,et al.  Get Out of my Picture! Internet-based Inpainting , 2009, BMVC.

[15]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[17]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[18]  Narendra Ahuja,et al.  Image completion using planar structure guidance , 2014, ACM Trans. Graph..

[19]  Dan Ciresan,et al.  Multi-Column Deep Neural Networks for offline handwritten Chinese character classification , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[20]  W. Y. Liu,et al.  A new Chinese character recognition approach based on the fuzzy clustering analysis , 2013, Neural Computing and Applications.

[21]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[22]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  Q. M. Jonathan Wu,et al.  Saliency detection via conditional adversarial image-to-image network , 2018, Neurocomputing.

[25]  Yoshua Bengio,et al.  Online and offline handwritten Chinese character recognition: A comprehensive study and new benchmark , 2016, Pattern Recognit..

[26]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[28]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[29]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[30]  Linlin Liu,et al.  ClothingOut: a category-supervised GAN model for clothing segmentation and retrieval , 2018, Neural Computing and Applications.

[31]  Jun Sun,et al.  Handwritten Character Recognition by Alternately Trained Relaxation Convolutional Neural Network , 2014, 2014 14th International Conference on Frontiers in Handwriting Recognition.

[32]  Jürgen Schmidhuber,et al.  LSTM recurrent networks learn simple context-free and context-sensitive languages , 2001, IEEE Trans. Neural Networks.

[33]  Xuelong Li,et al.  Fast and Accurate Matrix Completion via Truncated Nuclear Norm Regularization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[35]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[36]  Minh N. Do,et al.  Semantic Image Inpainting with Perceptual and Contextual Losses , 2016, ArXiv.

[37]  José M. Bioucas-Dias,et al.  An augmented Lagrangian approach to linear inverse problems with compound regularization , 2010, 2010 IEEE International Conference on Image Processing.

[38]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[39]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[40]  Daniel S. Yeung,et al.  Handwritten Chinese character recognition by rule-embedded Neocognitron , 1994, Neural Computing & Applications.