A Comprehensive Survey Analysis for Present Solutions of Medical Image Fusion and Future Directions

The track of medical imaging has witnessed several advancements in the last years. Several medical imaging modalities have appeared in the last decades including X-ray, Computed Tomography (CT), Magnetic Resonance (MR), Positron Emission Tomography (PET), Single-Photon Emission Computed Tomography (SPECT) and ultrasound imaging. Generally, medical images are used for the diagnosis purpose. Each type of acquired images has some merits and limitations. To maximize medical images utilization for the purpose of diagnosis, medical imaging fusion trend has appeared as a hot research field. Different medical imaging modalities are fused to obtain new images with complementary information. This paper presents a survey study of medical imaging modalities and their characteristics. In addition, different medical image fusion approaches and their appropriate quality metrics are presented. The main aim of this comprehensive survey analysis is to contribute in the advancement of medical image approaches that can help for better diagnosis of different diseases.

[1]  Daneshvar Sabalan,et al.  Fusion of multimodal medical images using nonsubsampled shearlet transform and particle swarm optimization , 2020 .

[2]  O. Barachini,et al.  The impact of 18F-FDOPA-PET/MRI image fusion in detecting liver metastasis in patients with neuroendocrine tumors of the gastrointestinal tract , 2020, BMC Medical Imaging.

[3]  Ganggang Sha,et al.  Multiple damage detection in laminated composite beams by data fusion of Teager energy operator-wavelet transform mode shapes , 2020 .

[4]  Naoto Yokoya,et al.  Guided Deep Decoder: Unsupervised Image Pair Fusion , 2020, ECCV.

[5]  Sabalan Daneshvar,et al.  Extraction of brain regions affected by Alzheimer disease via fusion of brain multispectral MR images , 2015, 2015 7th Conference on Information and Knowledge Technology (IKT).

[6]  Albert Montillo,et al.  Anatomically-Informed Data Augmentation for functional MRI with Applications to Deep Learning , 2019, Medical Imaging: Image Processing.

[7]  Pei-guang Wang,et al.  Image fusion algorithm based on regional variance and multi-wavelet bases , 2010, International Conference on Future Computer and Communication.

[8]  Shrouk A. El-Masry,et al.  Review of Different Image Fusion Techniques: Comparative Study , 2020 .

[9]  Ahmed Atwan,et al.  Current trends in medical image registration and fusion , 2016 .

[10]  Nils Große Hokamp,et al.  Advanced Fusion Imaging and Contrast-Enhanced Imaging (CT/MRI–CEUS) in Oncology , 2020, Cancers.

[11]  Pierre Croisille,et al.  Free-Breathing Diffusion Tensor Imaging and Tractography of the Human Heart in Healthy Volunteers Using Wavelet-Based Image Fusion , 2015, IEEE Transactions on Medical Imaging.

[12]  Zheng Qin,et al.  Effective Image Fusion Rules Of Multi-scale Image Decomposition , 2007, 2007 5th International Symposium on Image and Signal Processing and Analysis.

[13]  Nils Gessert,et al.  Skin lesion classification using ensembles of multi-resolution EfficientNets with meta data , 2019, MethodsX.

[14]  Dong Liu,et al.  Brain CT and MRI medical image fusion using convolutional neural networks and a dual-channel spiking cortical model , 2018, Medical & Biological Engineering & Computing.

[15]  Di Wang,et al.  A Phase Congruency and Local Laplacian Energy Based Multi-Modality Medical Image Fusion Method in NSCT Domain , 2019, IEEE Access.

[16]  S. El-Rabaie,et al.  An optimal wavelet-based multi-modality medical image fusion approach based on modified central force optimization and histogram matching , 2019, Multimedia Tools and Applications.

[17]  Xiao Han,et al.  MR‐based synthetic CT generation using a deep convolutional neural network method , 2017, Medical physics.

[18]  Wei Lu,et al.  Deep learning for variational multimodality tumor segmentation in PET/CT , 2020, Neurocomputing.

[19]  Lei Wang,et al.  Multi-focus image fusion: A Survey of the state of the art , 2020, Inf. Fusion.

[20]  Zheng Liu,et al.  Enhanced situation awareness through CNN-based deep multimodal image fusion , 2020, Optical Engineering.

[21]  Xie Xiao-zhu,et al.  Multi-sensor image fusion scheme based on dual-tree complex wavelet transform , 2015, 2015 34th Chinese Control Conference (CCC).

[22]  Max Blendowski,et al.  Multimodal 3D medical image registration guided by shape encoder–decoder networks , 2020, Int. J. Comput. Assist. Radiol. Surg..

[23]  Zhikui Chen,et al.  A Survey on Deep Learning for Multimodal Data Fusion , 2020, Neural Computation.

[24]  Satya Prakash Yadav,et al.  Image fusion using hybrid methods in multimodality medical images , 2020, Medical & Biological Engineering & Computing.

[25]  Fuchun Sun,et al.  Deep Multimodal Fusion by Channel Exchanging , 2020, NeurIPS.

[26]  Avinash Keskar,et al.  Performance evaluation of wavelet, ridgelet, curvelet and contourlet transforms based techniques for digital image denoising , 2015, Artificial Intelligence Review.

[27]  Vikrant Bhateja,et al.  Multimodal Medical Image Sensor Fusion Framework Using Cascade of Wavelet and Contourlet Transform Domains , 2015, IEEE Sensors Journal.

[28]  Wenbin Hu,et al.  Review-material degradation assessed by digital image processing: Fundamentals, progresses, and challenges , 2020, Journal of Materials Science & Technology.

[29]  Er. Jyoti Rani,et al.  Analytical Comparison of Various Image Fusion Techniques , 2015 .

[30]  Po Li,et al.  Artificial Intelligence (AI) Assisted CT/MRI Image Fusion Technique in Preoperative Evaluation of a Pelvic Bone Osteosarcoma , 2020, Frontiers in Oncology.

[31]  Ying Zhu,et al.  Multi-modal medical image fusion by Laplacian pyramid and adaptive sparse representation , 2020, Comput. Biol. Medicine.

[32]  Dejun Zhang,et al.  A full migration BBO algorithm with enhanced population quality bounds for multimodal biomedical image registration , 2020, Appl. Soft Comput..

[33]  Yikun Yang,et al.  Multi-Sensor Image Fusion Using Optimized Support Vector Machine and Multiscale Weighted Principal Component Analysis , 2020, Electronics.

[34]  K.P.Indira,et al.  Analysis on Image Fusion Techniques forMedical Applications , 2014 .

[35]  Houman Mahallati,et al.  The use of image fusion in prenatal medicine , 2020, Prenatal diagnosis.

[36]  Cedric Nishan Canagarajah,et al.  Pixel- and region-based image fusion with complex wavelets , 2007, Inf. Fusion.

[37]  Guojun Wang,et al.  Fast curvelet transform through genetic algorithm for multimodal medical image fusion , 2019, Soft Computing.

[38]  F. E. Abd El-Samie,et al.  Security of 3D-HEVC transmission based on fusion and watermarking techniques , 2019, Multimedia Tools and Applications.

[39]  Khaled Khelil,et al.  GA-based design of optimal discrete wavelet filters for efficient wind speed forecasting , 2020, Neural Computing and Applications.

[40]  Srinivasu Polinati,et al.  Multimodal medical image fusion using empirical wavelet decomposition and local energy maxima , 2020 .

[41]  J. Anitha,et al.  Optimum spectrum mask based medical image fusion using Gray Wolf Optimization , 2017, Biomed. Signal Process. Control..

[42]  Hans Torp,et al.  Reducing color flow artifacts caused by parallel beamforming , 2009, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[43]  G. Sasibhushana Rao,et al.  Investigation of Optimal Wavelet Techniques for De-noising of MRI Brain Abnormal Image , 2016 .

[44]  Pingkun Yan,et al.  Deep learning in medical image registration: a survey , 2019, Machine Vision and Applications.

[45]  Nurhan Karaboga,et al.  Medical Image Denoising Using Metaheuristics , 2017, Metaheuristics for Medicine and Biology.

[46]  Belur V. Dasarathy,et al.  Medical Image Fusion: A survey of the state of the art , 2013, Inf. Fusion.

[47]  Fathi E. Abd El-Samie,et al.  An efficient DT-CWT medical image fusion system based on modified central force optimization and histogram matching , 2018, Infrared Physics & Technology.

[48]  Christos Davatzikos,et al.  Harmonization of large MRI datasets for the analysis of brain imaging patterns throughout the lifespan , 2019, NeuroImage.

[49]  Hui Zhang,et al.  An Image Fusion Method Based on Curvelet Transform and Guided Filter Enhancement , 2020 .

[50]  Abdollah Homaifar,et al.  Optimization of Image Fusion using Genetic Algorithms and Discrete Wavelet Transform , 2010, Proceedings of the IEEE 2010 National Aerospace & Electronics Conference.

[51]  Aizhu Zhang,et al.  Fusion of PCA and Segmented-PCA Domain Multiscale 2-D-SSA for Effective Spectral-Spatial Feature Extraction and Data Classification in Hyperspectral Imagery , 2022, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Ali Aghagolzadeh,et al.  Ensemble of CNN for multi-focus image fusion , 2019, Inf. Fusion.

[53]  Padma Ganasala,et al.  Medical image fusion based on laws of texture energy measures in stationary wavelet transform domain , 2020, Int. J. Imaging Syst. Technol..

[54]  Amlan Chakrabarti,et al.  Spine medical image fusion using wiener filter in shearlet domain , 2015, 2015 IEEE 2nd International Conference on Recent Trends in Information Systems (ReTIS).

[55]  Jun Fu,et al.  Multimodal medical image fusion via laplacian pyramid and convolutional neural network reconstruction with local gradient energy strategy , 2020, Comput. Biol. Medicine.

[56]  D. J. Jagannath,et al.  Image fusion for stabilized medical video sequence using multimodal parametric registration , 2020, Pattern Recognit. Lett..

[57]  Sarmad Maqsood,et al.  Multi-modal Medical Image Fusion based on Two-scale Image Decomposition and Sparse Representation , 2020, Biomed. Signal Process. Control..

[58]  Yanna Zhao,et al.  Retinal Image Denoising via Bilateral Filter with a Spatial Kernel of Optimally Oriented Line Spread Function , 2017, Comput. Math. Methods Medicine.

[59]  Amit Kumar Jaiswal,et al.  Multimodal medical image fusion algorithm in the era of big data , 2020, Neural Computing and Applications.

[60]  Christina Lioma,et al.  Quantum-inspired multimodal fusion for video sentiment analysis , 2021, Inf. Fusion.

[61]  Hossein Rabbani,et al.  Forming Optimal Projection Images from Intra-Retinal Layers Using Curvelet-Based Image Fusion Method , 2020, Journal of medical signals and sensors.

[62]  Xiao-Ping Zhang,et al.  DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion , 2020, IEEE Transactions on Image Processing.

[63]  Hong Peng,et al.  Medical Image Fusion Method Based on Coupled Neural P Systems in Nonsubsampled Shearlet Transform Domain , 2020, Int. J. Neural Syst..

[64]  Amlan Chakrabarti,et al.  Medical image fusion by combining SVD and shearlet transform , 2015, 2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN).

[65]  Elijah Mwangi,et al.  An Improved Reconstruction Method for Compressively Sampled Magnetic Resonance Images Using Adaptive Gaussian Denoising , 2016 .

[66]  Chengxi Li,et al.  Application of Image Fusion in Diagnosis and Treatment of Liver Cancer , 2020, Applied Sciences.

[67]  Wael A. Mohamed,et al.  Optimal multi-scale geometrie fusion based on non sub-sampled contourlet transform and modified central force optimization , 2017 .

[68]  Xiaojie Guo,et al.  U2Fusion: A Unified Unsupervised Image Fusion Network , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  HYBRID FUSION APPROACH OF MEDICAL IMAGES USING DWT, DCT, LAPLACIAN PYRAMID AND CURVELET TRANSFORM , 2020, Journal of critical reviews.

[70]  Qiliang Du,et al.  An Enhanced Image Fusion Algorithm by Combined Histogram Equalization and Fast Gray Level Grouping Using Multi-Scale Decomposition and Gray-PCA , 2020, IEEE Access.

[71]  Fathi E. Abd El-Samie,et al.  Efficient multi-scale non-sub-sampled shearlet fusion system based on modified central force optimization and contrast enhancement , 2019, Infrared Physics & Technology.

[72]  Sumit Budhiraja,et al.  Multimodal medical image fusion using modified fusion rules and guided filter , 2015, International Conference on Computing, Communication & Automation.

[73]  N. Sivakumar,et al.  Hybrid medical image fusion using wavelet and curvelet transform with multi-resolution processing , 2017 .

[74]  Kunpeng Wang,et al.  Multi-Modality Medical Image Fusion Using Convolutional Neural Network and Contrast Pyramid , 2020, Sensors.

[75]  Padma Ganasala,et al.  Contrast Enhanced Multi Sensor Image Fusion Based on Guided Image Filter and NSST , 2020, IEEE Sensors Journal.

[76]  Rashid A. Fayadh,et al.  Design and implementation a prototype system for fusion image by using SWT-PCA algorithm with FPGA technique , 2020 .

[77]  R. S. Anand,et al.  Multimodal neurological image fusion based on adaptive biological inspired neural model in nonsubsampled Shearlet domain , 2018, Int. J. Imaging Syst. Technol..

[78]  Hyung-Jeong Yang,et al.  Multimodal learning using convolution neural network and Sparse Autoencoder , 2017, 2017 IEEE International Conference on Big Data and Smart Computing (BigComp).

[79]  Awais Ahmad,et al.  Medical image fusion method by using Laplacian pyramid and convolutional sparse representation , 2020, Concurr. Comput. Pract. Exp..

[80]  Takao Onoye,et al.  Halo artifacts reduction method for variational based realtime retinex image enhancement , 2012, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference.

[81]  Qinyong Lin,et al.  Improving medical image fusion method using fuzzy entropy and nonsubsampling contourlet transform , 2020, Int. J. Imaging Syst. Technol..

[82]  Jun Huang,et al.  A Generative Adversarial Network For Medical Image Fusion , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[83]  Lingli Cui,et al.  An Enhanced Intelligent Diagnosis Method Based on Multi-Sensor Image Fusion via Improved Deep Learning Network , 2020, IEEE Transactions on Instrumentation and Measurement.