Medical Image Fusion via Convolutional Sparsity Based Morphological Component Analysis

In this letter, a sparse representation (SR) model named convolutional sparsity based morphological component analysis (CS-MCA) is introduced for pixel-level medical image fusion. Unlike the standard SR model, which is based on single image component and overlapping patches, the CS-MCA model can simultaneously achieve multi-component and global SRs of source images, by integrating MCA and convolutional sparse representation (CSR) into a unified optimization framework. For each source image, in the proposed fusion method, the CSRs of its cartoon and texture components are first obtained by the CS-MCA model using pre-learned dictionaries. Then, for each image component, the sparse coefficients of all the source images are merged and the fused component is accordingly reconstructed using the corresponding dictionary. Finally, the fused image is calculated as the superposition of the fused cartoon and texture components. Experimental results demonstrate that the proposed method can outperform some benchmarking and state-of-the-art SR-based fusion methods in terms of both visual perception and objective assessment.

[1]  Pan Lin,et al.  Multiple Visual Features Measurement With Gradient Domain Guided Filtering for Multisensor Image Fusion , 2017, IEEE Transactions on Instrumentation and Measurement.

[2]  Yu Liu,et al.  Simultaneous image fusion and denoising with adaptive sparse representation , 2015, IET Image Process..

[3]  Vishal M. Patel,et al.  Convolutional Sparse and Low-Rank Coding-Based Image Decomposition , 2017, IEEE Transactions on Image Processing.

[4]  Qiang Zhang,et al.  Robust Multi-Focus Image Fusion Using Multi-Task Sparse Representation and Spatial Context , 2016, IEEE Transactions on Image Processing.

[5]  Liu Cao,et al.  Multi-Focus Image Fusion Based on Spatial Frequency in Discrete Cosine Transform Domain , 2015, IEEE Signal Processing Letters.

[6]  D. Donoho,et al.  Redundant Multiscale Transforms and Their Application for Morphological Component Separation , 2004 .

[7]  David R. Bull,et al.  Perceptual Image Fusion Using Wavelets , 2017, IEEE Transactions on Image Processing.

[8]  Dapeng Tao,et al.  Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning , 2018, Pattern Recognit..

[9]  Rick S. Blum,et al.  A new automated quality assessment algorithm for image fusion , 2009, Image Vis. Comput..

[10]  Hanseok Ko,et al.  Joint patch clustering-based dictionary learning for multimodal image fusion , 2016, Inf. Fusion.

[11]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[12]  Zheng Liu,et al.  Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain , 2013, IEEE Transactions on Multimedia.

[13]  Yaonan Wang,et al.  Combination of images with diverse focuses using the spatial frequency , 2001, Inf. Fusion.

[14]  Yi Chai,et al.  A novel multi-modality image fusion method based on image decomposition and sparse representation , 2017, Inf. Sci..

[15]  Shutao Li,et al.  Pixel-level image fusion with simultaneous orthogonal matching pursuit , 2012, Inf. Fusion.

[16]  Shutao Li,et al.  Pixel-level image fusion: A survey of the state of the art , 2017, Inf. Fusion.

[17]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[18]  Shutao Li,et al.  Group-Sparse Representation With Dictionary Learning for Medical Image Denoising and Fusion , 2012, IEEE Transactions on Biomedical Engineering.

[19]  Vladimir Petrovic,et al.  Objective image fusion performance measure , 2000 .

[20]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[21]  Henk J. A. M. Heijmans,et al.  A new quality metric for image fusion , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[22]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[23]  Brendt Wohlberg,et al.  Efficient Algorithms for Convolutional Sparse Representations , 2016, IEEE Transactions on Image Processing.

[24]  Shutao Li,et al.  Hybrid Multiresolution Method for Multisensor Multimodal Image Fusion , 2010, IEEE Sensors Journal.

[25]  Sabalan Daneshvar,et al.  MRI and PET image fusion by combining IHS and retina-inspired models , 2010, Inf. Fusion.

[26]  B. S. Manjunath,et al.  Multisensor Image Fusion Using the Wavelet Transform , 1995, CVGIP Graph. Model. Image Process..

[27]  L. Yang,et al.  Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform , 2008, Neurocomputing.

[28]  Rick S. Blum,et al.  Robust sparse representation based multi-focus image fusion with dictionary construction and local spatial consistency , 2018, Pattern Recognit..

[29]  Yi Liu,et al.  Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: A review , 2018, Inf. Fusion.

[30]  Yong Jiang,et al.  Image fusion with morphological component analysis , 2014, Inf. Fusion.

[31]  Zheng Liu,et al.  Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision: A Comparative Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Sun Li,et al.  Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters , 2016, Inf. Fusion.

[33]  Yu Liu,et al.  Multi-focus image fusion with a deep convolutional neural network , 2017, Inf. Fusion.

[34]  Belur V. Dasarathy,et al.  Medical Image Fusion: A survey of the state of the art , 2013, Inf. Fusion.

[35]  Malay Kumar Kundu,et al.  Corrections to "A Neuro-Fuzzy Approach for Medical Image Fusion" , 2015, IEEE Trans. Biomed. Eng..

[36]  Anup Basu,et al.  Cross-Scale Coefficient Selection for Volumetric Medical Image Fusion , 2013, IEEE Transactions on Biomedical Engineering.

[37]  Huchuan Lu,et al.  Medical Image Fusion and Denoising with Alternating Sequential Filter and Adaptive Fractional Order Total Variation , 2017, IEEE Transactions on Instrumentation and Measurement.

[38]  Shutao Li,et al.  Multifocus Image Fusion and Restoration With Sparse Representation , 2010, IEEE Transactions on Instrumentation and Measurement.

[39]  Weisheng Li,et al.  Anatomical-Functional Image Fusion by Information of Interest in Local Laplacian Filtering Domain , 2017, IEEE Transactions on Image Processing.

[40]  Rabab Kreidieh Ward,et al.  Image Fusion With Convolutional Sparse Representation , 2016, IEEE Signal Processing Letters.

[41]  Yibo Chen,et al.  Robust Multi-Focus Image Fusion Using Edge Model and Multi-Matting. , 2018, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[42]  Yu Zhang,et al.  Boundary finding based multi-focus image fusion through multi-scale morphological focus-measure , 2017, Inf. Fusion.

[43]  Haitao Yin Tensor Sparse Representation for 3-D Medical Image Fusion Using Weighted Average Rule. , 2018, IEEE transactions on bio-medical engineering.

[44]  M. Hossny,et al.  Image fusion performance metric based on mutual information and entropy driven quadtree decomposition , 2010 .

[45]  Nannan Yu,et al.  Image Features Extraction and Fusion Based on Joint Sparse Representation , 2011, IEEE Journal of Selected Topics in Signal Processing.

[46]  Shuai Ding,et al.  Image Fusion Algorithm Based on Nonsubsampled Contourlet Transform , 2013 .