Object-based multispectral image fusion method using deep learning

The goal of multispectral image fusion is to integrate complementary information from multispectral sensors to enhance human visual perception and object detection. Additionally, there are also cases when only the object needs to be emphasized with minimal background interference. This paper presents an object-based fusion method using deep learning to accomplish this objective. The proposed method uses information regarding the region of an object to perform fusion on the object. As we cannot provide labels for fusion results at the learning stage, we propose an unsupervised learning method. The proposed method simultaneously provides appropriate image information from the background and target for surveillance and reconnaissance.

[1]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[2]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[4]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Graham D. Finlayson,et al.  POP Image Fusion -- Derivative Domain Image Fusion without Reintegration , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[6]  Jian Sun,et al.  ExFuse: Enhancing Feature Fusion for Semantic Segmentation , 2018, ECCV.

[7]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[8]  Lawrence B. Wolff,et al.  Multispectral image visualization through first-order fusion , 2002, IEEE Trans. Image Process..

[9]  David Connah,et al.  Spectral Edge Image Fusion: Theory and Applications , 2014, ECCV.

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Toet Alexander,et al.  TNO Image Fusion Dataset , 2014 .

[12]  Xiangyu Zhang,et al.  Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).