Infrared and visible image fusion based on convolutional sparse representation and guided filtering

Abstract. Infrared and visible image fusion is a hot research direction in the fields of computer vision and image processing, and it is a common multimodal image fusion. An effective image fusion algorithm via convolutional sparse representation (CSR) and guided filtering is proposed for fusing infrared and visible images in this paper. First, a series of dictionary filters are trained by the CSR strategy, and the smooth image component and detailed image component are obtained by classifying those filters into high-pass filters and low-pass filters. Then two rules are designed to fuse the smooth image component and detailed image component, respectively. For the detailed image component, a weight construction method based on guided filtering is designed to get the weight maps, and the smooth image component is fused by applying the “choose-max” strategy to the corresponding sparse coefficients. Finally, the fused image is obtained by combining the fused smooth image component and detailed image component. Experimental results show that the proposed algorithm achieves good fusion results and exhibits advantages over comparison image fusion methods.

[1]  Ashish Khare,et al.  DTCWT Based Multimodal Medical Image Fusion , 2011 .

[2]  K. Ramar,et al.  Spatial frequency discrete wavelet transform image fusion technique for remote sensing applications , 2019, Engineering Science and Technology, an International Journal.

[3]  Shutao Li,et al.  Multifocus Image Fusion and Restoration With Sparse Representation , 2010, IEEE Transactions on Instrumentation and Measurement.

[4]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[5]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[6]  B. K. Shreyamsha Kumar,et al.  Image fusion based on pixel significance using cross bilateral filter , 2013, Signal, Image and Video Processing.

[7]  Ashish Khare,et al.  Local energy-based multimodal medical image fusion in curvelet domain , 2016, IET Comput. Vis..

[8]  Yi Liu,et al.  Sparse Representation based Multi-sensor Image Fusion: A Review , 2017, ArXiv.

[9]  Brendt Wohlberg,et al.  Efficient Algorithms for Convolutional Sparse Representations , 2016, IEEE Transactions on Image Processing.

[10]  Yan Wang,et al.  Infrared and multi-type images fusion algorithm based on contrast pyramid transform , 2016 .

[11]  Rabab Kreidieh Ward,et al.  Image Fusion With Convolutional Sparse Representation , 2016, IEEE Signal Processing Letters.

[12]  Maruturi Haribabu,et al.  Visibility based multi modal medical image fusion with DWT , 2017, 2017 IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI).

[13]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[14]  Veysel Aslantas,et al.  Extending depth-of-field of a digital camera using particle swarm optimization based image fusion , 2010, IEEE International Symposium on Consumer Electronics (ISCE 2010).

[15]  Jun Chen,et al.  Infrared and visible image fusion based on target-enhanced multiscale transform decomposition , 2020, Inf. Sci..

[16]  Jinyi Long,et al.  Fusion of Medical Sensors Using Adaptive Cloud Model in Local Laplacian Pyramid Domain , 2019, IEEE Transactions on Biomedical Engineering.

[17]  Awais Ahmad,et al.  Medical image fusion method by using Laplacian pyramid and convolutional sparse representation , 2020, Concurr. Comput. Pract. Exp..

[18]  Zheng Liu,et al.  PERFORMANCE ASSESSMENT OF COMBINATIVE PIXEL-LEVEL IMAGE FUSION BASED ON AN ABSOLUTE FEATURE MEASUREMENT , 2007 .

[19]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[20]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[21]  Haitao Yin Tensor Sparse Representation for 3-D Medical Image Fusion Using Weighted Average Rule , 2018, IEEE Transactions on Biomedical Engineering.

[22]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[23]  Rabab K. Ward,et al.  Medical Image Fusion via Convolutional Sparsity Based Morphological Component Analysis , 2019, IEEE Signal Processing Letters.