PathosisGAN: Sick Face Image Synthesis with Generative Adversarial Network

The image-to-image translation method based on Generative Adversarial Networks (GAN) realizes the conversion of the image from the source data domain to the target data domain by learning the joint distribution of the two data domains. However, there are still some challenges in applying GAN directly to sick face synthesis. Firstly, most existing image-to-image translation methods realize global style feature transfer, which is difficult to extract the subtle sick features (e.g., dark circles) in the local area of human face. Secondly, the number of images generated by image translation is limited, which is not conducive to a large-scale expansion of training data. In order to solve these problems, we build a novel Generative Adversarial Network model called PathosisGAN based on the CycleGAN framework. The model uses a mask control module to transfer the sick features in the local area of the human face to the source images and retain the source images subject information. Meanwhile, we add a feature extraction module to the GAN model to synthesize face images with different degrees of sickness, enhancing the data augmentation effect. Experimental results show that PathosisGAN achieves the synthesis of sick face images under unpaired data. Compared with other methods, the synthesized face images have clear sick features and natural visual effects, which provide enough sample data for medical analysis tasks based on human face images.