Multimodal sensor medical image fusion based on nonsubsampled shearlet transform and S-PCNNs in HSV space

Abstract Computational imaging plays an important role in medical treatment for providing more comprehensive medical images. This work proposes a new scheme to fuse computed tomography (CT), magnetic resonance (MRI), and positron emission tomography (PET) images into a single image. A novel two-stage medical image fusion scheme, which is based on non-subsampled shearlet transform (NSST) and simplified pulse coupled neural networks (S-PCNNs), is proposed in the hue-saturation-value (HSV) color space. Firstly, CT and MRI images are decomposed into a set of low and high frequency coefficients by NSST, PET images are transformed into the HSV color space, and then the V component of PET image in the HSV color space. Secondly, intersecting cortical models (ICMs) are utilized to extract the edges and outlines in a larger area from the high frequency coefficients, and S-PCNNs are employed to describe the subtly detailed information in a smaller area. Thirdly, different fusion rules are designed to fuse the corresponding low and high frequency coefficients. At last, the fused medical image is obtained by the inverse HSV and NSST transformation, successively. The experimental results show that the proposed scheme is effective, and it can fuse more information into the final images than conventional methods.

[1]  Ming Deng,et al.  Parametric active contour based on sparse decomposition for multi-objects extraction , 2018, Signal Process..

[2]  G. Easley,et al.  Sparse directional image representations using the discrete shearlet transform , 2008 .

[3]  Minh N. Do,et al.  The Nonsubsampled Contourlet Transform: Theory, Design, and Applications , 2006, IEEE Transactions on Image Processing.

[4]  Xin Jin,et al.  Remote sensing image fusion method in CIELab color space using nonsubsampled shearlet transform and pulse coupled neural networks , 2016 .

[5]  Shutao Li,et al.  Pixel-level image fusion: A survey of the state of the art , 2017, Inf. Fusion.

[6]  Yu-Jin Zhang,et al.  Query Adaptive Fusion for Graph-Based Visual Reranking , 2017, IEEE Journal of Selected Topics in Signal Processing.

[7]  Tian Le.  Image Processing Using Intersecting Cortical Model , 2009 .

[8]  Hamid Reza Shahdoosti,et al.  MRI and PET image fusion using structure tensor and dual ripplet-II transform , 2017, Multimedia Tools and Applications.

[9]  Cuntai Guan,et al.  A New Variational Method for Bias Correction and Its Applications to Rodent Brain Extraction , 2017, IEEE Transactions on Medical Imaging.

[10]  Shaowen Yao,et al.  A novel DNA sequence similarity calculation based on simplified pulse-coupled neural network and Huffman coding , 2016 .

[11]  Yu Liu,et al.  A medical image fusion method based on convolutional neural networks , 2017, 2017 20th International Conference on Information Fusion (Fusion).

[12]  Durga Prasad Bavirisetti,et al.  Fusion of Infrared and Visible Sensor Images Based on Anisotropic Diffusion and Karhunen-Loeve Transform , 2016, IEEE Sensors Journal.

[13]  Xin Jin,et al.  Block medical image fusion based on adaptive PCNN , 2015, 2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS).

[14]  Yan Feng,et al.  A fusion algorithm for infrared and visible based on guided filtering and phase congruency in NSST domain , 2017 .

[15]  W. Kong,et al.  Adaptive fusion method of visible light and infrared images based on non-subsampled shearlet transform and fast non-negative matrix factorization , 2014 .

[16]  Belur V. Dasarathy,et al.  Medical Image Fusion: A survey of the state of the art , 2013, Inf. Fusion.

[17]  Shutao Li,et al.  Multifocus Image Fusion and Restoration With Sparse Representation , 2010, IEEE Transactions on Instrumentation and Measurement.

[18]  Haibo Wang,et al.  Fast filtering image fusion , 2017, J. Electronic Imaging.

[19]  Rabab Kreidieh Ward,et al.  Image Fusion With Convolutional Sparse Representation , 2016, IEEE Signal Processing Letters.

[20]  Weitong Li,et al.  A composite objective metric and its application to multi-focus image fusion , 2017 .

[21]  Hao-quan Wang,et al.  Multi-Mode Medical Image Fusion Algorithm Based on Principal Component Analysis , 2009, 2009 International Symposium on Computer Network and Multimedia Technology.

[22]  Wang-Q Lim,et al.  Sparse multidimensional representation using shearlets , 2005, SPIE Optics + Photonics.

[23]  Vinod Kumar,et al.  Nonsubsampled shearlet based CT and MR medical image fusion using biologically inspired spiking neural network , 2015, Biomed. Signal Process. Control..

[24]  L. Yang,et al.  Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform , 2008, Neurocomputing.

[25]  Hamid Reza Shahdoosti,et al.  Multimodal image fusion using sparse representation classification in tetrolet domain , 2018, Digit. Signal Process..

[26]  Kup-Sze Choi,et al.  Fast feature-preserving speckle reduction for ultrasound images via phase congruency , 2017, Signal Process..

[27]  Diego Renza,et al.  Quality assessment by region in spot images fused by means dual-tree complex wavelet transform , 2011 .

[28]  Quan Wang,et al.  Multi-focus image fusion method using S-PCNN optimized by particle swarm optimization , 2017, Soft Computing.

[29]  Bin Li,et al.  A novel multi-modal medical image fusion method based on shift-invariant shearlet transform , 2013 .

[30]  Xin Guo-jiang.  Image identification method based on local time series of intersecting cortical model , 2011 .

[31]  Sarat Kumar Sahoo,et al.  Pulse coupled neural networks and its applications , 2014, Expert Syst. Appl..

[32]  Xiaoqing Luo,et al.  Image Fusion Using Quaternion Wavelet Transform and Multiple Features , 2017, IEEE Access.

[33]  Jason M. Kinser,et al.  Theoretical foundation of the intersecting cortical model and its use for change detection of aircraft, cars, and nuclear explosion tests , 2004, Signal Process..

[34]  Vikrant Bhateja,et al.  Medical image fusion with Laplacian Pyramids , 2014, 2014 International Conference on Medical Imaging, m-Health and Emerging Communication Systems (MedCom).

[35]  Omid Khayat,et al.  Image denoising using sparse representation classification and non-subsampled shearlet transform , 2016, Signal, Image and Video Processing.

[36]  Quan Wang,et al.  Multifocus Color Image Fusion Based on NSST and PCNN , 2016, J. Sensors.

[37]  Yu Liu,et al.  Simultaneous image fusion and denoising with adaptive sparse representation , 2015, IET Image Process..

[38]  Guanying Wang,et al.  Multimodal medical image fusion using PCNN optimized by the QPSO algorithm , 2016, Appl. Soft Comput..

[39]  Biplab Kanti Sen,et al.  Medical image fusion technique based on type-2 near fuzzy set , 2015, 2015 IEEE International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN).

[40]  Chunhui Zhao,et al.  A fast fusion scheme for infrared and visible light images in NSCT domain , 2015 .

[41]  Biao Hou,et al.  SAR Image Despeckling Based on Nonsubsampled Shearlet Transform , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[42]  Siqi Li,et al.  A Novel Fusion Framework Based on Adaptive PCNN in NSCT Domain for Whole-Body PET and CT Images , 2017, Comput. Math. Methods Medicine.

[43]  Yang Lei,et al.  Novel fusion method for visible light and infrared images based on NSST–SF–PCNN , 2014 .

[44]  Nikou Farhangi,et al.  Using BayesShrink, BiShrink, Weighted BayesShrink, and Weighted BiShrink in NSST and SWT for Despeckling SAR Images , 2018, EURASIP J. Image Video Process..

[45]  Padma Ganasala,et al.  Multimodality medical image fusion based on new features in NSST domain , 2014 .

[46]  T. S. Udhaya Suriya,et al.  An Improved Fusion Technique for Medical Images Using Discrete Wavelet Transform , 2016 .

[47]  Jun Sun,et al.  Multifocus Image Fusion Based on Extreme Learning Machine and Human Visual System , 2017, IEEE Access.

[48]  Aimin Hao,et al.  Robust multi-modal medical image fusion via anisotropic heat diffusion guided low-rank structural analysis , 2015, Inf. Fusion.

[49]  Pan Lin,et al.  Multimodal Sensor Medical Image Fusion Based on Type-2 Fuzzy Logic in NSCT Domain , 2016, IEEE Sensors Journal.

[50]  Jason M. Kinser,et al.  The intersecting cortical model in image processing , 2004 .