Pulse Coupled Neural Network-Based Multimodal Medical Image Fusion via Guided Filtering and WSEML in NSCT Domain

Multimodal medical image fusion aims to fuse images with complementary multisource information. In this paper, we propose a novel multimodal medical image fusion method using pulse coupled neural network (PCNN) and a weighted sum of eight-neighborhood-based modified Laplacian (WSEML) integrating guided image filtering (GIF) in non-subsampled contourlet transform (NSCT) domain. Firstly, the source images are decomposed by NSCT, several low- and high-frequency sub-bands are generated. Secondly, the PCNN-based fusion rule is used to process the low-frequency components, and the GIF-WSEML fusion model is used to process the high-frequency components. Finally, the fused image is obtained by integrating the fused low- and high-frequency sub-bands. The experimental results demonstrate that the proposed method can achieve better performance in terms of multimodal medical image fusion. The proposed algorithm also has obvious advantages in objective evaluation indexes VIFF, QW, API, SD, EN and time consumption.

[1]  Weisheng Li,et al.  Two‐scale image decomposition based image fusion using structure tensor , 2020, Int. J. Imaging Syst. Technol..

[2]  Wei Yu,et al.  Infrared and visible image fusion via detail preserving adversarial learning , 2020, Inf. Fusion.

[3]  Hongbing Ma,et al.  Brain Image Enhancement Approach Based on Singular Value Decomposition in Nonsubsampled Shearlet Transform Domain , 2020 .

[4]  Ying Zhu,et al.  Multi-modal medical image fusion by Laplacian pyramid and adaptive sparse representation , 2020, Comput. Biol. Medicine.

[5]  Rabab Kreidieh Ward,et al.  Deep learning for pixel-level image fusion: Recent advances and future prospects , 2018, Inf. Fusion.

[6]  B. K. Shreyamsha Kumar.  Image fusion based on pixel significance using cross bilateral filter , 2015 .

[7]  Yang Lei,et al.  Multimodal Sensor Medical Image Fusion Based on Local Difference in Non-Subsampled Domain , 2019, IEEE Transactions on Instrumentation and Measurement.

[8]  Jiayi Ma,et al.  MEF-GAN: Multi-Exposure Image Fusion via Generative Adversarial Networks , 2020, IEEE Transactions on Image Processing.

[9]  Qinyong Lin,et al.  Improving medical image fusion method using fuzzy entropy and nonsubsampling contourlet transform , 2020, Int. J. Imaging Syst. Technol..

[10]  Yi Zhou,et al.  Infrared and visible image fusion via gradientlet filter , 2020, Comput. Vis. Image Underst..

[11]  Xiongfei Li,et al.  Multifocus image fusion using convolutional neural networks in the discrete wavelet transform domain , 2019, Multimedia Tools and Applications.

[12]  Xianbiao Qi,et al.  Fast infrared and visible image fusion with structural decomposition , 2020, Knowl. Based Syst..

[13]  Lei Wang,et al.  Multi-focus image fusion: A Survey of the state of the art , 2020, Inf. Fusion.

[14]  Yu Liu,et al.  Simultaneous image fusion and denoising with adaptive sparse representation , 2015, IET Image Process..

[15]  Chunming Wu,et al.  Infrared and visible image fusion method of dual NSCT and PCNN , 2020, PloS one.

[16]  Ashish Khare,et al.  Multimodal Medical Image Fusion in Dual Tree Complex Wavelet Transform Domain Using Maximum and Average Fusion Rules , 2012 .

[17]  Demetrio Labate,et al.  Optimally Sparse Multidimensional Representation Using Shearlets , 2007, SIAM J. Math. Anal..

[18]  Naima Iltaf,et al.  A multifocus image fusion using highlevel DWT components and guided filter , 2020, Multimedia Tools and Applications.

[19]  Susanto Rahardja,et al.  A New Multi-Focus Image Fusion Algorithm and Its Efficient Implementation , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Lixia Zhang,et al.  Multi-Modality Image Fusion in Adaptive-Parameters SPCNN Based on Inherent Characteristics of Image , 2020, IEEE Sensors Journal.

[21]  C. Bennila Thangammal,et al.  Visible and infrared image fusion using DTCWT and adaptive combined clustered dictionary , 2018, Infrared Physics & Technology.

[22]  R. S. Anand,et al.  Multimodal Medical Image Sensor Fusion Model Using Sparse K-SVD Dictionary Learning in Nonsubsampled Shearlet Domain , 2020, IEEE Transactions on Instrumentation and Measurement.

[23]  Xin Zhao,et al.  A pixel-level fusion method for multi-source optical remote sensing image combining the principal component analysis and curvelet transform , 2020, Earth Science Informatics.

[24]  Zhenhong Jia,et al.  A novel multiscale transform decomposition based multi-focus image fusion framework , 2021, Multimedia Tools and Applications.

[25]  Yu Han,et al.  A new image fusion performance metric based on visual information fidelity , 2013, Inf. Fusion.

[26]  Qi Mao,et al.  Image fusion based on multiscale transform and sparse representation to enhance terahertz images. , 2020, Optics express.

[27]  Hongbing Ma,et al.  A novel approach for multi-focus image fusion based on SF-PAPCNN and ISML in NSST domain , 2020, Multimedia Tools and Applications.

[28]  Meiling Wang,et al.  Using Taylor Expansion and Convolutional Sparse Representation for Image Fusion , 2020, Neurocomputing.

[29]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[30]  Rabab K. Ward,et al.  Medical Image Fusion via Convolutional Sparsity Based Morphological Component Analysis , 2019, IEEE Signal Processing Letters.

[31]  Manas Kamal Bhuyan,et al.  Image Fusion Using Adjustable Non-subsampled Shearlet Transform , 2019, IEEE Transactions on Instrumentation and Measurement.

[32]  Lei Wang,et al.  EGGDD: An explicit dependency model for multi-modal medical image fusion in shift-invariant shearlet transform domain , 2014, Inf. Fusion.

[33]  Liangliang Li,et al.  Enhancement of hyperspectral remote sensing images based on improved fuzzy contrast in nonsubsampled shearlet transform domain , 2019, Multimedia Tools and Applications.

[34]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[35]  Ruichao Hou,et al.  Robust spiking cortical model and total-variational decomposition for multimodal medical image fusion , 2020, Biomed. Signal Process. Control..

[36]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[37]  Zheng Liu,et al.  Objective Assessment of Multiresolution Image Fusion Algorithms for Context Enhancement in Night Vision: A Comparative Study , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Minh N. Do,et al.  The Nonsubsampled Contourlet Transform: Theory, Design, and Applications , 2006, IEEE Transactions on Image Processing.

[39]  Qu Xiao,et al.  Image Fusion Algorithm Based on Spatial Frequency-Motivated Pulse Coupled Neural Networks in Nonsubsampled Contourlet Transform Domain , 2008 .

[40]  Han Xu,et al.  MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion , 2021, Inf. Fusion.

[41]  Jun Chen,et al.  Infrared and visible image fusion based on target-enhanced multiscale transform decomposition , 2020, Inf. Sci..

[42]  Yu Liu,et al.  Multi-focus image fusion with a deep convolutional neural network , 2017, Inf. Fusion.

[43]  Zhenhong Jia,et al.  A Novel Medical Image Fusion Approach Based on Nonsubsampled Shearlet Transform , 2019, J. Medical Imaging Health Informatics.

[44]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Gamal Attiya,et al.  Satellite image fusion based on modified central force optimization , 2020, Multimedia Tools and Applications.

[46]  Xun Chen,et al.  Medical Image Fusion With Parameter-Adaptive Pulse Coupled Neural Network in Nonsubsampled Shearlet Transform Domain , 2019, IEEE Transactions on Instrumentation and Measurement.

[47]  Hong Peng,et al.  Multi-focus image fusion based on dynamic threshold neural P systems and surfacelet transform , 2020, Knowl. Based Syst..

[48]  Yu Liu,et al.  IFCNN: A general image fusion framework based on convolutional neural network , 2020, Inf. Fusion.

[49]  B. K. Shreyamsha Kumar,et al.  Image fusion based on pixel significance using cross bilateral filter , 2013, Signal, Image and Video Processing.