Non-subsampled shearlet transform-based image fusion using modified weighted saliency and local difference

Existing image fusion methods can not efficiently capture significant edges, texture and fine details of the source images due to inefficient fusion framework. In addition, for objective evaluation of fusion algorithms, not much attention is given to simultaneously measure both texture and structural information of the source images which are preserved in the fused image. To address these issues, non-subsampled shearlet transform (NSST) is used to decompose pre-registered source images into low- and high-frequency components. These low- and high-frequency coefficients are fused by using our proposed modified weighted salience and local difference fusion rules, respectively. To enrich edge information in the fused image, Canny edge detector with scale multiplication is employed. Moreover, a metric QTS is proposed to jointly measure both texture and structural information present in the fused image. The proposed metric is formulated on the basis of local standard deviation filtering, local information entropy, and local difference filtering. Both subjective and objective results validate the proposed fusion framework and the metric QTS.

[1]  G. Easley,et al.  Sparse directional image representations using the discrete shearlet transform , 2008 .

[2]  Shuhe Zhao,et al.  A new method of remote sensing image decision-level fusion based on Support Vector Machine , 2003, International Conference on Recent Advances in Space Technologies, 2003. RAST '03. Proceedings of.

[3]  Gong Zhang,et al.  Image fusion method of SAR and infrared image based on Curvelet transform with adaptive weighting , 2015, Multimedia Tools and Applications.

[4]  Li Ping,et al.  Face recognition based on the fusion of wavelet packet sub-images and fisher linear discriminant , 2017 .

[5]  T. Stathaki,et al.  Optimal Contrast Correction for ICA-Based Fusion of Multimodal Images , 2008, IEEE Sensors Journal.

[6]  Rong Wang,et al.  A Feature-Level Image Fusion Algorithm Based on Neural Networks , 2007, 2007 1st International Conference on Bioinformatics and Biomedical Engineering.

[7]  Vladimir S. Petrovic,et al.  Gradient-based multiresolution image fusion , 2004, IEEE Transactions on Image Processing.

[8]  Guy Marchal,et al.  Automated multi-moda lity image registration based on information theory , 1995 .

[9]  Ping Li,et al.  Face recognition based on the fusion of wavelet packet sub-images and fisher linear discriminant , 2017, Multim. Tools Appl..

[10]  Vikrant Bhateja,et al.  Multimodal Medical Image Sensor Fusion Framework Using Cascade of Wavelet and Contourlet Transform Domains , 2015, IEEE Sensors Journal.

[11]  Xiongfei Li,et al.  Image fusion based on simultaneous empirical wavelet transform , 2017, Multimedia Tools and Applications.

[12]  Mei Yang,et al.  A novel algorithm of image fusion using shearlets , 2011 .

[13]  Mashruha Raquib Mitashe,et al.  An adaptive digital image watermarking scheme with PSO, DWT and XFCM , 2017, 2017 IEEE International Conference on Imaging, Vision & Pattern Recognition (icIVPR).

[14]  Min Huang,et al.  Multifocus image fusion method of Ripplet transform based on cycle spinning , 2014, Multimedia Tools and Applications.

[15]  Lei Zhang,et al.  Canny edge detection enhancement by scale multiplication , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Yu Guo,et al.  A novel image fusion algorithm based on nonsubsampled shearlet transform , 2014 .

[17]  Madheswari Kanmani,et al.  An optimal weighted averaging fusion strategy for thermal and visible images using dual tree discrete wavelet transform and self tunning particle swarm optimization , 2016, Multimedia Tools and Applications.

[18]  Peter J. Burt,et al.  Enhanced image capture through fusion , 1993, 1993 (4th) International Conference on Computer Vision.

[19]  Guy Marchal,et al.  Automated multi-modality image registration based on information theory , 1995 .

[20]  Myungjin Choi,et al.  THE CURVELET TRANSFORM FOR IMAGE FUSION , 2004 .

[21]  Jianping Liu,et al.  Technique for image fusion based on nonsubsampled shearlet transform and improved pulse-coupled neural network , 2013 .

[22]  Pan Lin,et al.  Multimodal Sensor Medical Image Fusion Based on Type-2 Fuzzy Logic in NSCT Domain , 2016, IEEE Sensors Journal.

[23]  Durga Prasad Bavirisetti,et al.  Fusion of Infrared and Visible Sensor Images Based on Anisotropic Diffusion and Karhunen-Loeve Transform , 2016, IEEE Sensors Journal.

[24]  Anil K. Jain,et al.  Decision-Level Fusion in Fingerprint Verification , 2001, Multiple Classifier Systems.

[25]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[26]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[27]  Wang-Q Lim,et al.  The Discrete Shearlet Transform: A New Directional Transform and Compactly Supported Shearlet Frames , 2010, IEEE Transactions on Image Processing.

[28]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[29]  Wei Li,et al.  Multi-level fusion of graph based discriminant analysis for hyperspectral image classification , 2016, Multimedia Tools and Applications.

[30]  Liu Cao,et al.  Multi-Focus Image Fusion Based on Spatial Frequency in Discrete Cosine Transform Domain , 2015, IEEE Signal Processing Letters.

[31]  N. Aishwarya,et al.  An image fusion framework using novel dictionary based sparse representation , 2017, Multimedia Tools and Applications.

[32]  Laurent Demanet,et al.  Fast Discrete Curvelet Transforms , 2006, Multiscale Model. Simul..

[33]  Rong Huang,et al.  Some inequalities for the Hadamard product and the Fan product of matrices , 2008 .

[34]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[35]  Yu Liu,et al.  Multi-focus image fusion with a deep convolutional neural network , 2017, Inf. Fusion.

[36]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[38]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[39]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[40]  Xin Liu,et al.  A novel similarity based quality metric for image fusion , 2008, Inf. Fusion.

[41]  Yaonan Wang,et al.  Multifocus image fusion using artificial neural networks , 2002, Pattern Recognit. Lett..

[42]  Yu-Chiang Frank Wang,et al.  Exploring Visual and Motion Saliency for Automatic Video Object Extraction , 2013, IEEE Transactions on Image Processing.

[43]  David Summers,et al.  Harvard Whole Brain Atlas: www.med.harvard.edu/AANLIB/home.html , 2003 .

[44]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[45]  Yansheng Li,et al.  Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration , 2017, Inf. Sci..

[46]  Ji Zhao,et al.  Non-rigid visible and infrared face registration via regularized Gaussian fields criterion , 2015, Pattern Recognit..

[47]  Zheng Liu,et al.  Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain , 2013, IEEE Transactions on Multimedia.