Research on hybrid fusion algorithm for multi-feature among heterogeneous image

Abstract There are many types of features in heterogeneous images, and the differences between these features are significant; thus, ensuring that different features can be fused into a single image is difficult. To address this problem, a hybrid fusion algorithm for multi-feature fusion between heterogeneous images is proposed in this paper. First, the contour features of the heterogeneous images are extracted by using the local pixel meaning value contrast, and the fusion weight of the contour feature is obtained using a guider filter with the aim of fusing the contour features between images. Second, the edge features of the heterogeneous images are extracted by using the local standard deviation, and then fused by using the local standard deviation similarity between images as the threshold, yielding the edge features of the fused image. Third, the contour-feature fusion and edge-feature fusion images are combined to obtain the low-frequency fusion image. Fourth, the source images are transformed by using two-dimensional variation mode decomposition (2D-VMD), and a detailed-feature fusion image is obtained by using the skewness and an impulse neural network (PCNN). Finally, the spatial domain fusion image is injected into 2D-VMD as the low-frequency fusion image of 2D-VMD, and the fusion result is obtained via inverse transformation of 2D-VMD. The experimental results demonstrate that the proposed fusion algorithm can clearly improve the fusion performance of heterogeneous images, and generate a more informative fusion image.

[1]  V. Aslantaş,et al.  A new image quality metric for image fusion: The sum of the correlations of differences , 2015 .

[2]  Zhuowen Tu,et al.  Robust $L_{2}E$ Estimation of Transformation for Non-Rigid Registration , 2015, IEEE Transactions on Signal Processing.

[3]  Yan Huang,et al.  Infrared and Visible Image Fusion Based on Different Constraints in the Non-Subsampled Shearlet Transform Domain , 2018, Sensors.

[4]  Xiangzhi Bai,et al.  Infrared and visual image fusion through feature extraction by morphological sequential toggle operator , 2015 .

[5]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[6]  Dominique Zosso,et al.  Two-Dimensional Variational Mode Decomposition , 2015, EMMCVPR.

[7]  Zhang Baohua,et al.  A multi-focus image fusion algorithm based on an improved dual-channel PCNN in NSCT domain , 2013 .

[8]  Sun Li,et al.  Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters , 2016, Inf. Fusion.

[9]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[10]  Shaowen Yao,et al.  Multi-Sensor Image Fusion Based on Interval Type-2 Fuzzy Sets and Regional Features in Nonsubsampled Shearlet Transform Domain , 2018, IEEE Sensors Journal.

[11]  Hong Wei,et al.  Fusion of infrared polarization and intensity images using support value transform and fuzzy combination rules , 2013 .

[12]  Huimin Lu,et al.  Multisensor Image Fusion and Enhancement in Spectral Total Variation Domain , 2018, IEEE Transactions on Multimedia.

[13]  Ravindra Dhuli,et al.  Two-scale image fusion of visible and infrared images using saliency detection , 2016 .

[14]  Yu Liu,et al.  Infrared and visible image fusion with convolutional neural networks , 2017, Int. J. Wavelets Multiresolution Inf. Process..

[15]  Bo Li,et al.  Multifocus image fusion via fixed window technique of multiscale images and non-local means filtering , 2017, Signal Process..

[16]  Haixu Wang,et al.  Multimodal medical image fusion based on IHS and PCA , 2010 .

[17]  Yang Song,et al.  Fusion of infrared and visible images based on nonsubsampled contourlet transform and sparse K-SVD dictionary learning , 2017 .

[18]  Jun Sun,et al.  Multi-Focus Image Fusion via Clustering PCA Based Joint Dictionary Learning , 2017, IEEE Access.

[19]  Yafei Zhang,et al.  Infrared and visible image fusion scheme based on NSCT and low-level visual features , 2016 .

[20]  Li Zhao,et al.  Guided filter-based images fusion algorithm for CT and MRI medical images , 2018, IET Image Process..

[21]  Yuming Fang,et al.  A Hybrid Method for Multi-Focus Image Fusion Based on Fast Discrete Curvelet Transform , 2017, IEEE Access.

[22]  Shukai Duan,et al.  A general memristor-based pulse coupled neural network with variable linking coefficient for multi-focus image fusion , 2018, Neurocomputing.

[23]  Mecheri Kious,et al.  A novel hybrid image fusion method based on integer lifting wavelet and discrete cosine transformer for visual sensor networks , 2018, Multimedia Tools and Applications.

[24]  Li Yan,et al.  A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain , 2015 .

[25]  Ying Zhu,et al.  Multi-focus Image Fusion Based on the Improved PCNN and Guided Filter , 2017, Neural Processing Letters.

[26]  Dominique Zosso,et al.  Two-Dimensional Compact Variational Mode Decomposition , 2017, Journal of Mathematical Imaging and Vision.

[27]  Li Li,et al.  Multiscale infrared and visible image fusion using gradient domain guided image filtering , 2018 .

[28]  Dominique Zosso,et al.  Variational Mode Decomposition , 2014, IEEE Transactions on Signal Processing.

[29]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[30]  Su-zhen Lin,et al.  Fusion of infrared intensity and polarization images using embedded multi-scale transform , 2015 .

[31]  Jinde Cao,et al.  Fully Convolutional Network-Based Multifocus Image Fusion , 2018, Neural Computation.

[32]  Ji Zhao,et al.  Non-rigid visible and infrared face registration via regularized Gaussian fields criterion , 2015, Pattern Recognit..

[33]  Junjun Jiang,et al.  FusionGAN: A generative adversarial network for infrared and visible image fusion , 2019, Inf. Fusion.

[34]  Wei Liu,et al.  A novel infrared and visible image fusion algorithm based on shift-invariant dual-tree complex shearlet transform and sparse representation , 2017, Neurocomputing.

[35]  C. Bennila Thangammal,et al.  Visible and infrared image fusion using DTCWT and adaptive combined clustered dictionary , 2018, Infrared Physics & Technology.

[36]  Pan Zhu,et al.  A fusion method for infrared–visible image and infrared-polarization image based on multi-scale center-surround top-hat transform , 2017 .