Fusion of multimodal medical images using nonsubsampled shearlet transform and particle swarm optimization

Medical imaging has been an indispensable tool in modern medicine in last decades. Various types of imaging systems provide structural and functional information about tissues. But most of the time both kinds of information are necessary to make proper decision. Image fusion aims at gathering complementary information of different sources in one image to be more informative. This paper proposes a new method for this purpose. In proposed method, source images are first decomposed using nonsubsampled shearlet transform. Extracting most of relevant information and merging them to achieve the best weights for fusion task is done by principal component analysis and particle swarm optimization. Fused image is provided by merging source images according to weights achieved from previous steps. Quantitative and qualitative analysis prove outperformance of our methods compared to well-known fusion methods. The experimental results show improvement compared to subsequent best method, in terms of peak-signal-to-noise-ratio (+ 8.85%), entropy (+ 3.48%), standard deviation (+ 16.3%), and quality index (+ 14.84%).

[1]  S. Muttan,et al.  Discrete wavelet transform based principal component averaging fusion for medical images , 2015 .

[2]  G. Easley,et al.  Sparse directional image representations using the discrete shearlet transform , 2008 .

[3]  Manfred Ehlers,et al.  Multisensor image fusion techniques in remote sensing , 1991 .

[4]  K. Indira,et al.  Pixel based Medical Image Fusion Techniques using Discrete Wavelet Transform and Stationary Wavelet Transform , 2015 .

[5]  Yu-Chiang Frank Wang,et al.  Exploring Visual and Motion Saliency for Automatic Video Object Extraction , 2013, IEEE Transactions on Image Processing.

[6]  Marimuthu Karuppiah,et al.  CT and MRI Image Fusion Based on Discrete Wavelet Transform and Type-2 Fuzzy Logic , 2017 .

[7]  Vinod Kumar,et al.  CT and MR Image Fusion Scheme in Nonsubsampled Contourlet Transform Domain , 2014, Journal of Digital Imaging.

[8]  Chandan Chakraborty,et al.  Multispectral MRI Image Fusion for Enhanced Visualization of Meningioma Brain Tumors and Edema Using Contourlet Transform and Fuzzy Statistics , 2016 .

[9]  Belur V. Dasarathy,et al.  Medical Image Fusion: A survey of the state of the art , 2013, Inf. Fusion.

[10]  Vikrant Bhateja,et al.  Medical image fusion using combination of PCA and wavelet analysis , 2014, 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[11]  Vikrant Bhateja,et al.  Multimodal Medical Image Sensor Fusion Framework Using Cascade of Wavelet and Contourlet Transform Domains , 2015, IEEE Sensors Journal.

[12]  Pan Lin,et al.  Log-Gabor Energy Based Multimodal Medical Image Fusion in NSCT Domain , 2014, Comput. Math. Methods Medicine.

[13]  Yufeng Zheng,et al.  An advanced image fusion algorithm based on wavelet transform: incorporation with PCA and morphological processing , 2004, IS&T/SPIE Electronic Imaging.

[14]  Andrew P. Bradley.  Shift-invariance in the Discrete Wavelet Transform , 2003, DICTA.

[15]  Yuanyuan Wang,et al.  Biological image fusion using a NSCT based variable-weight method , 2011, Inf. Fusion.

[16]  Minh N. Do,et al.  Ieee Transactions on Image Processing the Contourlet Transform: an Efficient Directional Multiresolution Image Representation , 2022 .

[17]  Yong Jiang,et al.  Image fusion with morphological component analysis , 2014, Inf. Fusion.

[18]  Liu Shuaiqia,et al.  MEDICAL IMAGE FUSION BASED ON NONSUBSAMPLED DIRECTION COMPLEX WAVELET TRANSFORM , 2014 .

[19]  Cedric Nishan Canagarajah,et al.  Image Fusion Using Complex Wavelets , 2002, BMVC.

[20]  Pan Lin,et al.  Multimodal Sensor Medical Image Fusion Based on Type-2 Fuzzy Logic in NSCT Domain , 2016, IEEE Sensors Journal.

[21]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[22]  Zhiming Cui,et al.  Medical Image Fusion Based on Wavelet Transform and Independent Component Analysis , 2009, 2009 International Joint Conference on Artificial Intelligence.

[23]  Shutao Li,et al.  Performance comparison of different multi-resolution transforms for image fusion , 2011, Inf. Fusion.

[24]  T. Jayasree,et al.  Improved medical image fusion based on cascaded PCA and shift invariant wavelet transforms , 2018, International Journal of Computer Assisted Radiology and Surgery.

[25]  Vikrant Bhateja,et al.  Weighted-PCA Based Multimodal Medical Image Fusion in Contourlet Domain , 2016 .

[26]  Stewart Marshall,et al.  Multires-olution morphological fusion of mr and ct images of the human brain , 1994 .

[27]  Vikrant Bhateja,et al.  PCA Based Medical Image Fusion in Ridgelet Domain , 2014, FICTA.

[28]  Zheng Liu,et al.  A new contrast based multimodal medical image fusion framework , 2015, Neurocomputing.

[29]  Zheng Liu,et al.  Directive Contrast Based Multimodal Medical Image Fusion in NSCT Domain , 2013, IEEE Transactions on Multimedia.

[30]  Shutao Li,et al.  Pixel-level image fusion: A survey of the state of the art , 2017, Inf. Fusion.

[31]  Christine Pohl,et al.  Multisensor image fusion in remote sensing: concepts, methods and applications , 1998 .

[32]  William R. Hersh,et al.  Using Media Fusion and Domain Dimensions to Improve Precision in Medical Image Retrieval , 2009, CLEF.

[33]  Harpreet Singh,et al.  Image fusion using fuzzy logic and applications , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[34]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[35]  Feng Zhao,et al.  The nonsubsampled contourlet transform for image fusion , 2007, 2007 International Conference on Wavelet Analysis and Pattern Recognition.

[36]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[37]  Vinod Kumar,et al.  Nonsubsampled shearlet based CT and MR medical image fusion using biologically inspired spiking neural network , 2015, Biomed. Signal Process. Control..

[38]  Shutao Li,et al.  Multi-focus Image Fusion by Nonsubsampled Shearlet Transform , 2011, 2011 Sixth International Conference on Image and Graphics.

[39]  Ibrahim M. Eldokany,et al.  CURVELET FUSION OF MR AND CT IMAGES , 2008 .

[40]  M. L. Dewal,et al.  Medical image denoising using adaptive fusion of curvelet transform and total variation , 2013, Comput. Electr. Eng..

[41]  Jin-Jang Leou,et al.  Multispectral and multiresolution image fusion using particle swarm optimization , 2011, Multimedia Tools and Applications.

[42]  Huiqian Du,et al.  Multi-modality medical image fusion based on image decomposition framework and nonsubsampled shearlet transform , 2018, Biomed. Signal Process. Control..

[43]  Haidi Ibrahim,et al.  Medical image fusion scheme using complex contourlet transform based on PCA , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[44]  Richard Bamler,et al.  A Sparse Image Fusion Algorithm With Application to Pan-Sharpening , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Madheswari Kanmani,et al.  Entropy optimized image fusion: Using particle swarm technology and discrete wavelet transform , 2017, 2017 International Conference on Computation of Power, Energy Information and Commuincation (ICCPEIC).

[46]  Leida Li,et al.  Multimodal medical image fusion based on discrete Tchebichef moments and pulse coupled neural network , 2017, Int. J. Imaging Syst. Technol..

[47]  Vinod Kumar,et al.  Feature-Motivated Simplified Adaptive PCNN-Based Medical Image Fusion Algorithm in NSST Domain , 2016, Journal of Digital Imaging.

[48]  Udhaya Suriya Ts,et al.  Brain tumour detection using discrete wavelet transform based medical image fusion , 2017 .

[49]  C. Prakash,et al.  Medical image fusion based on redundancy DWT and Mamdani type min-sum mean-of-max techniques with quantitative analysis , 2012, 2012 International Conference on Recent Advances in Computing and Software Systems.

[50]  Yide Ma,et al.  Medical image fusion using m-PCNN , 2008, Inf. Fusion.

[51]  A. Hegde,et al.  A Review of Quality Metrics for Fused Image , 2015 .

[52]  Jean-Marc Constans,et al.  Knowledge based fuzzy information fusion applied to classification of abnormal brain tissues from MRI , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[53]  Vikrant Bhateja,et al.  Medical Image Fusion in Curvelet Domain Employing PCA and Maximum Selection Rule , 2016 .

[54]  Mohd Zamin Jumaat,et al.  Development of Hybrid Jute Fibre Composite Plates for Potential Application in Shear Retrofitting of RC Structure , 2015 .

[55]  Simon X. Yang,et al.  A Novel approach for Multimodal Medical Image Fusion using Hybrid Fusion Algorithms for Disease Analysis , 2017 .

[56]  S. Rajkumar,et al.  Hybrid method for multimodality medical image fusion using Discrete Wavelet Transform and Entropy concepts with quantitative analysis , 2013, 2013 International Conference on Communication and Signal Processing.

[57]  Aimin Hao,et al.  Multimodal medical image fusion using improved multi-channel PCNN. , 2014, Bio-medical materials and engineering.

[58]  Dong Sun Park,et al.  Medical Image Fusion via an Effective Wavelet-Based Approach , 2010, EURASIP J. Adv. Signal Process..