Infrared and visible image fusion using dual discriminators generative adversarial networks with Wasserstein distance

Abstract Generative adversarial network (GAN) has shown great potential in infrared and visible image fusion. The existing GAN-based methods establish an adversarial game between generative image and source images to train the generator until the generative image contains enough meaningful information from source images. However, they only design one discriminator to force the fused result to complement gradient information from visible image, which may lose some detail information that existing in infrared image and omit some texture information that existing in visible image. To this end, we propose an end-to-end dual discriminators Wasserstein generative adversarial network, termed as D2WGAN, a framework that extends GAN to dual discriminators. In D2WGAN, the fused image can keep pixel intensity and details of infrared image by the first discriminator, and capture rich texture information of visible image by the second discriminator. In addition, to improve the performance of D2WGAN, we employ the GAN with Wasserstein distance. Moreover, in order to make the fused image keep more details from visible image in texture feature domain, we define a novel LBP (local binary pattern) loss. The extensive qualitative and quantitative experiments on public datasets demonstrate that D2WGAN can generate better results compared with the other state-of-the-art methods.