Structure-aware image fusion

Abstract Most existing multi-modal image fusion methods require multi-scale transforms. However, this requirement does not necessarily lead to the fusion result containing the original intensity of source images, and multi-scale transforms need a high computational complexity. In this paper, we tackle the problem of multi-modal image fusion in the spatial domain with a low computational complexity. A salient structure extraction method and a structure-preserving filter are developed to fuse medical images. The developed structure-preserving filter has a property that it recovers small-scale details of the guidance image in the neighborhood of large-scale structures of the input image. Based on the property of the structure-preserving filter, the fusion result is constructed by combining the output of the structure-preserving filter and the source images. Experiments are conducted to demonstrate the effectiveness of the proposed method in comparison with the state-of-the-art approaches in terms of three performance metrics.

[1]  Shutao Li,et al.  Performance comparison of different multi-resolution transforms for image fusion , 2011, Inf. Fusion.

[2]  Shuicheng Yan,et al.  Scale-Aware Fast R-CNN for Pedestrian Detection , 2015, IEEE Transactions on Multimedia.

[3]  Graham D. Finlayson,et al.  POP Image Fusion -- Derivative Domain Image Fusion without Reintegration , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[4]  Consuelo Gonzalo-Martín,et al.  Scale-Aware Pansharpening Algorithm for Agricultural Fragmented Landscapes , 2016, Remote. Sens..

[5]  Nikolaos Mitianoudis,et al.  Pixel-based and region-based image fusion schemes using ICA bases , 2007, Inf. Fusion.

[6]  Gemma Piella,et al.  A general framework for multiresolution image fusion: from pixels to regions , 2003, Inf. Fusion.

[7]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[8]  Manuel M. Oliveira,et al.  Domain transform for edge-aware image and video processing , 2011, SIGGRAPH 2011.

[9]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[10]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[11]  Qi Zhang,et al.  Rolling Guidance Filter , 2014, ECCV.

[12]  Vladimir S. Petrovic,et al.  Subjective tests for image fusion evaluation and objective metric validation , 2007, Inf. Fusion.

[13]  Zheng Liu,et al.  Image fusion by using steerable pyramid , 2001, Pattern Recognit. Lett..

[14]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Qiaoqiao Li,et al.  Linking synaptic computation for image enhancement , 2017, Neurocomputing.

[16]  G. Easley,et al.  Sparse directional image representations using the discrete shearlet transform , 2008 .

[17]  George K. Matsopoulos,et al.  Application of Morphological Pyramids: Fusion of MR and CT Phantoms , 1995, J. Vis. Commun. Image Represent..

[18]  Xiaojun Chang,et al.  Feature Interaction Augmented Sparse Learning for Fast Kinect Motion Detection , 2017, IEEE Transactions on Image Processing.

[19]  Cedric Nishan Canagarajah,et al.  Segmentation-Driven Image Fusion Based on Alpha-Stable Modeling of Wavelet Coefficients , 2009, IEEE Transactions on Multimedia.

[20]  Emmanuel J. Candès,et al.  The curvelet transform for image denoising , 2002, IEEE Trans. Image Process..

[21]  Stavri G. Nikolov,et al.  Image fusion: Advances in the state of the art , 2007, Inf. Fusion.

[22]  Xiaojun Chang,et al.  Semisupervised Feature Analysis by Mining Correlations Among Multiple Tasks , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Yi Yang,et al.  Bi-Level Semantic Representation Analysis for Multimedia Event Detection , 2017, IEEE Transactions on Cybernetics.

[24]  Yi Yang,et al.  Semantic Pooling for Complex Event Analysis in Untrimmed Videos , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Zhihui Li,et al.  Beyond Trace Ratio: Weighted Harmonic Mean of Trace Ratios for Multiclass Discriminant Analysis , 2017, IEEE Transactions on Knowledge and Data Engineering.

[26]  Haibo Wang,et al.  Fast filtering image fusion , 2017, J. Electronic Imaging.

[27]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[29]  Ashish Khare,et al.  Fusion of multimodal medical images using Daubechies complex wavelet transform - A multiresolution approach , 2014, Inf. Fusion.

[30]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[31]  Alexander A. Sawchuk,et al.  Adaptive Noise Smoothing Filter for Images with Signal-Dependent Noise , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Xin Liu,et al.  A novel similarity based quality metric for image fusion , 2008, Inf. Fusion.

[33]  Minh N. Do,et al.  The Nonsubsampled Contourlet Transform: Theory, Design, and Applications , 2006, IEEE Transactions on Image Processing.

[34]  Aimin Hao,et al.  Robust multi-modal medical image fusion via anisotropic heat diffusion guided low-rank structural analysis , 2015, Inf. Fusion.

[35]  Bin Li,et al.  Multimodal Medical Volumetric Data Fusion Using 3-D Discrete Shearlet Transform and Global-to-Local Rule , 2014, IEEE Transactions on Biomedical Engineering.

[36]  Wai-kuen Cham,et al.  Gradient-directed composition of multi-exposure images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[37]  L. Yang,et al.  Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform , 2008, Neurocomputing.

[38]  Malay Kumar Kundu,et al.  Corrections to "A Neuro-Fuzzy Approach for Medical Image Fusion" , 2015, IEEE Trans. Biomed. Eng..

[39]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[40]  Rick S. Blum,et al.  A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application , 1999, Proc. IEEE.

[41]  Yong Jiang,et al.  P–M equation based multiscale decomposition and its application to image fusion , 2013, Pattern Analysis and Applications.

[42]  Qiaoqiao Li,et al.  Feature-Linking Model for Image Enhancement , 2016, Neural Computation.

[43]  Peter J. Burt,et al.  Enhanced image capture through fusion , 1993, 1993 (4th) International Conference on Computer Vision.

[44]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Zheng Liu,et al.  Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain , 2013, IEEE Transactions on Multimedia.

[46]  Shutao Li,et al.  Pixel-level image fusion: A survey of the state of the art , 2017, Inf. Fusion.

[47]  Gonzalo Pajares,et al.  A wavelet-based image fusion tutorial , 2004, Pattern Recognit..

[48]  G. Qu,et al.  Information measure for performance of image fusion , 2002 .

[49]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[50]  Qiaoqiao Li,et al.  Multifocus image fusion using phase congruency , 2015, J. Electronic Imaging.

[51]  Richard Szeliski,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[52]  Zheng Liu,et al.  Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision: A Comparative Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  David Connah,et al.  Spectral Edge Image Fusion: Theory and Applications , 2014, ECCV.