A GAN-based visible and infrared image fusion algorithm

In this paper,we propose a new GAN-based end-to-end image fusion network (VIFGAN) for fusion of visible and infrared images. VIFGAN contains a generator and a discriminator. We added the DenseNet module to the generator,and this module can extract deeper features and details. We also propose a Two-way regulation loss function(TWR-Loss). The loss function considers both the radiation information and texture information in the image, which can make the network suitable for image fusion tasks of different spectrum combinations. The experimental results show that in the fusion task of visible light and infrared images, the proposed network has better fusion performance than the existing fusion algorithm, the visual effect of the fused image is better, and the extracted details are more abundant.

[1]  Tianshuang Qiu,et al.  Medical image fusion based on sparse representation of classified image patches , 2017, Biomed. Signal Process. Control..

[2]  Walaa Hassan Elashmawi,et al.  A Robust Pan-Sharpening Scheme for Improving Resolution of Satellite Images in the Domain of the Nonsubsampled Shearlet Transform , 2019 .

[3]  Hui Li,et al.  MDLatLRR: A Novel Decomposition Method for Infrared and Visible Image Fusion , 2018, IEEE Transactions on Image Processing.

[4]  Junjun Jiang,et al.  FusionDN: A Unified Densely Connected Network for Image Fusion , 2020, AAAI.

[5]  Bin Wang,et al.  Multi-band images synchronous fusion based on NSST and fuzzy logical inference , 2019, Infrared Physics & Technology.

[6]  Yu Liu,et al.  Multi-focus image fusion with a deep convolutional neural network , 2017, Inf. Fusion.

[7]  Hui Li,et al.  DenseFuse: A Fusion Approach to Infrared and Visible Images , 2018, IEEE Transactions on Image Processing.

[8]  V. Aslantaş,et al.  A new image quality metric for image fusion: The sum of the correlations of differences , 2015 .

[9]  Hong Zhao,et al.  Image Fusion With Cosparse Analysis Operator , 2017, IEEE Signal Processing Letters.

[10]  Xin Zhang,et al.  Infrared and Visible Image Fusion with a Generative Adversarial Network and a Residual Network , 2020, Applied Sciences.

[11]  Rabab Kreidieh Ward,et al.  Image Fusion With Convolutional Sparse Representation , 2016, IEEE Signal Processing Letters.

[12]  Junjun Jiang,et al.  FusionGAN: A generative adversarial network for infrared and visible image fusion , 2019, Inf. Fusion.

[13]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[14]  Minh N. Do,et al.  The Nonsubsampled Contourlet Transform: Theory, Design, and Applications , 2006, IEEE Transactions on Image Processing.

[15]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Haifeng Li,et al.  Dictionary learning method for joint sparse representation-based image fusion , 2013 .

[17]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[18]  Zhizhong Fu,et al.  Infrared and visible images fusion based on RPCA and NSCT , 2016 .