Multiscale fusion of multimodal medical images using lifting scheme based biorthogonal wavelet transform

Abstract Medical image fusion has been used to improve useful and relevant information (e.g., precise localization of abnormalities) of multimodal medical images, such as computed tomography (CT), magnetic resonance imaging (MRI) and positron emission tomography (PET) images, and the information obtained from the fused images can then be utilized as an assistive tool for better diagnosis and treatment. In this work, we propose an algorithm for fusing multimodal medical images using lifting scheme-based biorthogonal wavelet transform. The multiscale fusion scheme is performed for the multimodal medical images in the wavelet domain at multiple scales by adopting either average or absolute maximum fusion rules. To verify the effectiveness of the proposed method, a series of visual and quantitative performance evaluations of the proposed method are compared with those of five representative wavelet-based fusion methods including contourlet transform (CLT), nonsubsampled CLT (NSCLT), lifting wavelet transform (LWT), multiwavelet transform (MWT), and stationary wavelet transform (SWT). For the quantitative performance evaluations, we adopted five metrics: fusion factor, fusion symmetry, entropy, standard deviation and edge strength. The experimental results demonstrated that the proposed method could yield better results than other wavelet transform-based fusion methods. Furthermore, from the additional comparison study of fusing noise-contaminated images, we could conclude that the proposed method is noise resilient in fusing images corrupted by Gaussian and speckle noise with varying variances.

[1]  Kun Gao,et al.  A novel image registration approach via combining local features and geometric invariants , 2018, PloS one.

[2]  Ying Han,et al.  Structure-aware image fusion , 2018, Optik.

[3]  Haitao Yin.  Tensor Sparse Representation for 3-D Medical Image Fusion Using Weighted Average Rule. , 2018, IEEE transactions on bio-medical engineering.

[4]  Belur V. Dasarathy,et al.  Medical Image Fusion: A survey of the state of the art , 2013, Inf. Fusion.

[5]  Arif Mahmood,et al.  Multi-focus image fusion using Content Adaptive Blurring , 2019, Inf. Fusion.

[6]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Shutao Li,et al.  Pixel-level image fusion: A survey of the state of the art , 2017, Inf. Fusion.

[8]  Haixu Wang,et al.  Multimodal medical image fusion based on IHS and PCA , 2010 .

[9]  A. Hegde,et al.  A Review of Quality Metrics for Fused Image , 2015 .

[10]  Ashish Khare,et al.  Daubechies Complex Wavelet Transform Based Multilevel Shrinkage for Deblurring of Medical Images in Presence of Noise , 2009, Int. J. Wavelets Multiresolution Inf. Process..

[11]  Paul Suetens,et al.  Medical image registration using mutual information , 2003, Proc. IEEE.

[12]  Ashish Khare,et al.  Biorthogonal wavelet transform based image fusion using absolute maximum fusion rule , 2013, 2013 IEEE CONFERENCE ON INFORMATION AND COMMUNICATION TECHNOLOGIES.

[13]  Leida Li,et al.  No reference quality evaluation of medical image fusion , 2018, Int. J. Imaging Syst. Technol..

[14]  Michael Unser,et al.  A review of wavelets in biomedical applications , 1996, Proc. IEEE.

[15]  U. Tiwary,et al.  Feature level fusion of multimodal medical images in lifting wavelet transform domain , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  Ashish Khare,et al.  Multilevel Threshold Based Image Denoising in Curvelet Domain , 2010, Journal of Computer Science and Technology.

[17]  Yixin Yin,et al.  Decision-level fusion of infrared and visible images for face recognition , 2008, 2008 Chinese Control and Decision Conference.

[18]  J. R. Taylor,et al.  Active mode-locking of lasers using GaAs and GaP picosecond switches , 1980 .

[19]  W. Sweldens.  The Lifting Scheme: A Custom - Design Construction of Biorthogonal Wavelets "Industrial Mathematics , 1996 .

[20]  Jiancheng Yu,et al.  An image fusion method based on biorthogonal wavelet , 2007, SPIE/COS Photonics Asia.

[21]  Kun Wang.  Rock particle image fusion based on sparse representation and non-subsampled contourlet transform , 2019 .

[22]  Srinivasa Rao Dammavalam,et al.  Quality Assessment of Pixel-Level ImageFusion Using Fuzzy Logic , 2012, ArXiv.

[23]  Jinzhu Yang,et al.  PET/CT medical image fusion algorithm based on multiwavelet transform , 2010, 2010 2nd International Conference on Advanced Computer Control.

[24]  Wim Sweldens,et al.  The lifting scheme: a construction of second generation wavelets , 1998 .

[25]  R. Vijaya Durga,et al.  Region-Based Image Fusion Using Complex Wavelets , 2014 .

[26]  Simon X. Yang,et al.  A Novel approach for Multimodal Medical Image Fusion using Hybrid Fusion Algorithms for Disease Analysis , 2017 .

[27]  U. Rajendra Acharya,et al.  Cardiac health diagnosis using data fusion of cardiovascular and haemodynamic signals , 2006, Comput. Methods Programs Biomed..

[28]  Roberto Orecchia,et al.  MR and CT image fusion for postimplant analysis in permanent prostate seed implants. , 2004, International journal of radiation oncology, biology, physics.

[29]  Gong Zhang,et al.  Image fusion method of SAR and infrared image based on Curvelet transform with adaptive weighting , 2015, Multimedia Tools and Applications.

[30]  Xinming Tang,et al.  IMAGE FUSION AND IMAGE QUALITY ASSESSMENT OF FUSED IMAGES , 2013 .

[31]  Zhongfu Ye,et al.  First-order fusion of volumetric medical imagery , 2006 .

[32]  L. Tugan Muftuler,et al.  Wavelet Domain Radiofrequency Pulse Design Applied to Magnetic Resonance Imaging , 2015, PloS one.

[33]  Morand Piert,et al.  Reirradiation of recurrent high-grade gliomas using amino acid PET (SPECT)/CT/MRI image fusion to determine gross tumor volume for stereotactic fractionated radiotherapy. , 2004, International journal of radiation oncology, biology, physics.

[34]  Mohammad Sadegh Helfroush,et al.  Hierarchical classification of normal, fatty and heterogeneous liver diseases from ultrasound images using serial and parallel feature fusion , 2016 .

[35]  Shutao Li,et al.  Performance comparison of different multi-resolution transforms for image fusion , 2011, Inf. Fusion.

[36]  Wei Li,et al.  A Novel Approach to Fast Medical Image Fusion Based on Lifting Wavelet Transform , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[37]  A. Khare,et al.  Mixed scheme based multimodal medical image fusion using Daubechies Complex Wavelet Transform , 2012, 2012 International Conference on Informatics, Electronics & Vision (ICIEV).

[38]  Stanislaw Osowski,et al.  Ensemble of classifiers and wavelet transformation for improved recognition of Fuhrman grading in clear-cell renal carcinoma , 2017 .

[39]  Ashish Khare,et al.  Multimodal Medical Image Fusion in Dual Tree Complex Wavelet Transform Domain Using Maximum and Average Fusion Rules , 2012 .

[40]  Ashish Khare,et al.  Medical Image Denoising Based on Soft Thresholding Using Biorthogonal Multiscale Wavelet Transform , 2014, Int. J. Image Graph..

[41]  Frank Lindseth,et al.  Image fusion of ultrasound and MRI as an aid for assessing anatomical shifts and improving overview and interpretation in ultrasound-guided neurosurgery , 2001, CARS.

[42]  Zheng Liu,et al.  Human visual system inspired multi-modal medical image fusion framework , 2013, Expert Syst. Appl..

[43]  M V Knopp,et al.  Image fusion using CT, MRI and PET for treatment planning, navigation and follow up in percutaneous RFA. , 2009, Experimental oncology.

[44]  U. Raff,et al.  Image fusion in neuroradiology: Three clinical examples including MRI of Parkinson disease , 2007, Comput. Medical Imaging Graph..

[45]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[46]  Belur V. Dasarathy,et al.  Information fusion in the realm of medical applications - A bibliographic glimpse at its growing appeal , 2012, Inf. Fusion.

[47]  Tatsuya Higashi,et al.  Clinical Value of Image Fusion from MR and PET in Patients with Head and Neck Cancer , 2008, Molecular Imaging and Biology.

[48]  Guy Gilboa,et al.  Spectral Total-Variation Local Scale Signatures for Image Manipulation and Fusion , 2019, IEEE Transactions on Image Processing.

[49]  Vince D. Calhoun,et al.  Feature-Based Fusion of Medical Imaging Data , 2009, IEEE Transactions on Information Technology in Biomedicine.

[50]  Yu Liu,et al.  A general framework for image fusion based on multi-scale transform and sparse representation , 2015, Inf. Fusion.

[51]  N. Kannathal,et al.  Data Fusion of Multimodal Cardiovascular Signals , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[52]  Jin Wu,et al.  An Image Fusion Algorithm Based on Discrete Wavelet Transform and Canny Operator , 2011 .

[53]  Kishore Rajendiran,et al.  Swarm intelligence based image fusion for noisy images using consecutive pixel intensity , 2018, Multimedia Tools and Applications.

[54]  James Llinas,et al.  Handbook of Multisensor Data Fusion : Theory and Practice, Second Edition , 2008 .

[55]  Michael Unser,et al.  Statistical analysis of functional MRI data in the wavelet domain , 1998, IEEE Transactions on Medical Imaging.

[56]  John R. Deller,et al.  Pixel-level multisensor image fusion based on matrix completion and robust principal component analysis , 2016, J. Electronic Imaging.

[57]  Isha Mehra,et al.  Wavelet-based image fusion for securing multiple images through asymmetric keys , 2015 .

[58]  T. Cover,et al.  Entropy, Relative Entropy and Mutual Information , 2001 .

[59]  Nemir Ahmed Al-Azzawi,et al.  An efficient medical image fusion method using contourlet transform based on PCM , 2009, 2009 IEEE Symposium on Industrial Electronics & Applications.