Occluded offline handwritten Chinese character inpainting via generative adversarial network and self-attention mechanism

Abstract Occluded offline handwritten Chinese characters inpainting is a critical step for handwritten Chinese characters recognition. We propose to apply generative adversarial network and self-attention mechanism to inpaint occluded offline handwritten Chinese characters. First, cyclic loss is used to guarantee the cyclic consistency of the uncorrupted area between corrupted images and original real images instead of masks. Second, self-attention mechanism is combined with generative adversarial network to increase receptive field and explore more Chinese character features. Then an improved character-VGG-19 that is pre-trained with handwritten Chinese character dataset is used to calculate content loss to extract character features more effectively and assist generator to generate realistic characters. Finally, adversarial classification loss is used to make our discriminator classify input images instead of just distinguishing real images from fake images in order to learn the distribution of Chinese characters more effectively. The proposed method is evaluated on an occluded CASIA-HWDB1.1 dataset for three challenging inpainting tasks with different portions of blocks, or pixels randomly missing, or pixels randomly adding. Experimental results show that our method is more effective, compared with several state-of-the-art handwritten Chinese character inpainting methods.