A Review on Medical Image Analysis with Convolutional Neural Networks

Over the last few years, deep learning has grown rapidly from a promising to a viable option to analyze medical images. With an increase in use of medical imaging for diagnosis and treatment, the field offers a significant potential for research. A major advantage offered by deep learning is using large amounts of data to avoid tedious hand-crafting of features which requires extensive domain knowledge. This review introduces a few popular algorithms using Convolutional Neural Networks(CNNs) being used in the field along with their applications: Classification, Detection, Segmentation, Registration and Image Enhancement. The paper further provides some useful resources on some of the most promising anatomical areas of application in medical image analysis with Convolutional Neural Networks: brain, breast, chest, eye and skin.

[1]  Berkman Sahiner,et al.  3D convolutional neural network for automatic detection of lung nodules in chest CT , 2017, Medical Imaging.

[2]  Nima Tajbakhsh,et al.  Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation , 2019, Medical Image Anal..

[3]  Jiang Liu,et al.  Glaucoma detection based on deep convolutional neural network. , 2015, Conference proceedings : ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference.

[4]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[5]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[6]  Berkman Sahiner,et al.  Deep learning in medical imaging and radiation therapy. , 2018, Medical physics.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Mohammad H. Jafari,et al.  Melanoma detection by analysis of clinical images using convolutional neural network , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[9]  David R. Dance,et al.  Mammographic Image Analysis Society (MIAS) database v1.21 , 2015 .

[10]  Feng-Ping An,et al.  Medical Image Classification Algorithm Based on Weight Initialization-Sliding Window Fusion Convolutional Neural Network , 2019, Complex..

[11]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[12]  Harald Kittler,et al.  The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions , 2018, Scientific Data.

[13]  Giovanni Montana,et al.  Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks , 2015, ICPRAM 2015.

[14]  Syed Muhammad Anwar,et al.  Medical Image Analysis using Convolutional Neural Networks: A Review , 2017, Journal of Medical Systems.

[15]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[16]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Catarina Eloy,et al.  Classification of breast cancer histology images using Convolutional Neural Networks , 2017, PloS one.

[18]  Hai Su,et al.  Region segmentation in histopathological breast cancer images using deep convolutional neural network , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[19]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[20]  Ronald M. Summers,et al.  ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases , 2019, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[21]  Bo Zhou,et al.  Convolutional neural networks for computer-aided detection or diagnosis in medical image analysis: An overview. , 2019, Mathematical biosciences and engineering : MBE.

[22]  Chi-Hieu Pham,et al.  Brain MRI super-resolution using deep 3D convolutional networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[23]  Mohammad H. Jafari,et al.  Skin lesion segmentation in clinical images using deep learning , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[24]  Lovedeep Gondara,et al.  Medical Image Denoising Using Convolutional Denoising Autoencoders , 2016, 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).

[25]  Andrew Y. Ng,et al.  CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning , 2017, ArXiv.

[26]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[27]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[28]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[29]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[30]  Frans Coenen,et al.  Convolutional Neural Networks for Diabetic Retinopathy , 2016, MIUA.

[31]  Yabo Fu,et al.  Deep Learning in Medical Image Registration: A Review , 2020, Physics in medicine and biology.

[32]  Luiz Eduardo Soares de Oliveira,et al.  Breast cancer histopathological image classification using Convolutional Neural Networks , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[33]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.

[35]  Dinggang Shen,et al.  Convolutional Neural Network for Reconstruction of 7T-like Images from 3T MRI Using Appearance and Anatomical Features , 2016, LABELS/DLMIA@MICCAI.

[36]  Yifan Yu,et al.  CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison , 2019, AAAI.

[37]  Synho Do,et al.  How much data is needed to train a medical image deep learning system to achieve necessary high accuracy , 2015, 1511.06348.

[38]  A. Sevastopolsky,et al.  Optic disc and cup segmentation methods for glaucoma detection with modification of U-Net convolutional neural network , 2017, Pattern Recognition and Image Analysis.

[39]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[40]  Daniel P. Kennedy,et al.  The Autism Brain Imaging Data Exchange: Towards Large-Scale Evaluation of the Intrinsic Brain Architecture in Autism , 2013, Molecular Psychiatry.

[41]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[42]  Mike E. Davies,et al.  Improved Breast Mass Segmentation in Mammograms with Conditional Residual U-net , 2018, RAMBO+BIA+TIA@MICCAI.

[43]  Richard H. Moore,et al.  THE DIGITAL DATABASE FOR SCREENING MAMMOGRAPHY , 2007 .

[44]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[45]  Qiao Hu,et al.  Automated Separation of Binary Overlapping Trees in Low-Contrast Color Retinal Images , 2013, MICCAI.

[46]  Hayit Greenspan,et al.  Chest pathology detection using deep learning with non-medical training , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[47]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[48]  Jyoti Islam,et al.  Brain MRI analysis for Alzheimer’s disease diagnosis using an ensemble system of deep convolutional neural networks , 2018, Brain Informatics.

[49]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[50]  George Shih,et al.  A patient-centric dataset of images and metadata for identifying melanomas using clinical context , 2020, Scientific Data.

[51]  Maria S. Kulikova,et al.  Mitosis detection in breast cancer histological images An ICPR 2012 contest , 2013, Journal of pathology informatics.

[52]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[53]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[54]  Daniel S. Marcus,et al.  OASIS-3: Longitudinal Neuroimaging, Clinical, and Cognitive Dataset for Normal Aging and Alzheimer Disease , 2019 .

[55]  Pingkun Yan,et al.  Deep learning in medical image registration: a survey , 2020, Machine Vision and Applications.

[56]  Richard Alan Peters,et al.  A Review of Deep Learning with Special Emphasis on Architectures, Applications and Recent Trends , 2019, Knowl. Based Syst..

[57]  Tien Yin Wong,et al.  Glaucoma detection based on deep convolutional neural network , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[58]  Maria Wimmer,et al.  Fully Convolutional Architectures for Multiclass Segmentation in Chest Radiographs , 2017, IEEE Transactions on Medical Imaging.

[59]  Sheila Weinmann,et al.  Use of diagnostic imaging studies and associated radiation exposure for patients enrolled in large integrated health care systems, 1996-2010. , 2012, JAMA.

[60]  Sharath Pankanti,et al.  Deep learning ensembles for melanoma recognition in dermoscopy images , 2016, IBM J. Res. Dev..

[61]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[62]  Victor Alves,et al.  Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images , 2016, IEEE Transactions on Medical Imaging.

[63]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[64]  Lanfen Lin,et al.  Medical Image Classification Using Deep Learning , 2019 .

[65]  Parashkev Nachev,et al.  Computer Methods and Programs in Biomedicine NiftyNet: a deep-learning platform for medical imaging , 2022 .

[66]  Alexander Wong,et al.  COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images , 2020, Scientific reports.

[67]  Yinhao Li,et al.  Medical Image Enhancement Using Deep Learning , 2019 .

[68]  Tao Li,et al.  Lesion Detection and Grading of Diabetic Retinopathy via Two-Stages Deep Convolutional Neural Networks , 2017, MICCAI.

[69]  Amir Alansary,et al.  MRBrainS Challenge: Online Evaluation Framework for Brain Image Segmentation in 3T MRI Scans , 2015, Comput. Intell. Neurosci..

[70]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[71]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[72]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[73]  Miguel Ángel González Ballester,et al.  Medical Image Detection Using Deep Learning , 2019 .

[74]  Miguel Ángel González Ballester,et al.  Medical Image Segmentation Using Deep Learning , 2019, Intelligent Systems Reference Library.

[75]  Joseph Paul Cohen,et al.  COVID-19 Image Data Collection , 2020, ArXiv.

[76]  Nikhil Ketkar,et al.  Deep Learning with Python , 2017 .

[77]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[78]  David Dagan Feng,et al.  An Ensemble of Fine-Tuned Convolutional Neural Networks for Medical Image Classification , 2017, IEEE Journal of Biomedical and Health Informatics.

[79]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[80]  Brahim Ait Skourt,et al.  Lung CT Image Segmentation Using Deep Neural Networks , 2018 .

[81]  Shaohui Liu,et al.  Medical image denoising using convolutional neural network: a residual learning approach , 2017, The Journal of Supercomputing.

[82]  Xiangjian He,et al.  Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges , 2019, Journal of Digital Imaging.

[83]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.