Deep Learning for Medical Image Analysis: Applications to Computed Tomography and Magnetic Resonance Imaging

Following the recent development in artificial intelligence, where deep learning has become the main methodology, the paradigm of medical image analysis is shifting from the previous clinical experience and knowledge-based feature engineering to the data-driven objective feature analysis of deep learning. Especially, as the application of various techniques developed for natural images to medical images is being accelerated, we are no longer simply adapting the natural image models to medical images but developing new methods, which encompasses the unique characteristics of the medical image domain. Furthermore, as the research on interpretability of decisions made by deep learning models and the way of incorporating clinical knowledge into the model progresses, we have started to obtain promising results that will allow clinical implementation of deep learning. Among various deep learning models, convolutional neural networks (CNN) have become methodology of choice for visual recognition problems. CNN is a type of feed-forward artificial neural network, which learns hierarchical features by iterating convolution and pooling layers until the output prediction layer is reached. While the convolution layers learn specific patterns in the input or intermediate feature map with locally-connected shared weights, pooling layers reduce the feature map by spatially aggregating activations. In special cases where the output of the model is same as the input or its denoised version, we call the model as convolutional auto-enconder (CAE). In medical image analysis, machine learning methods have been used in various fields such as detection and classification Corresponding Author: Kyu-Hwan Jung VUNO Inc., 6F, 507, Gangnamdae-ro, Seocho-gu, Seoul, Korea Tel: +82-2-515-6646 Fax: +82-2-515-6647 E-mail: kyuhwanjung@gmail.com

[1]  Dinggang Shen,et al.  Convolutional Neural Network for Reconstruction of 7T-like Images from 3T MRI Using Appearance and Anatomical Features , 2016, LABELS/DLMIA@MICCAI.

[2]  Siegfried Wahl,et al.  Leveraging uncertainty information from deep neural networks for disease detection , 2016, Scientific Reports.

[3]  Su Ruan,et al.  Medical Image Synthesis with Context-Aware Generative Adversarial Networks , 2016, MICCAI.

[4]  M. P. van den Heuvel,et al.  Deep learning predictions of survival based on MRI in amyotrophic lateral sclerosis , 2016, NeuroImage: Clinical.

[5]  Ronald M. Summers,et al.  Holistic Interstitial Lung Disease Detection using Deep Convolutional Neural Networks: Multi-label Learning and Unordered Pooling , 2017, ArXiv.

[6]  Nikos Komodakis,et al.  A Deep Metric for Multimodal Registration , 2016, MICCAI.

[7]  Pablo Lamata,et al.  Recurrent Fully Convolutional Neural Networks for Multi-slice MRI Cardiac Segmentation , 2016, RAMBO+HVSMR@MICCAI.

[8]  Klaus H. Maier-Hein,et al.  Adversarial Networks for the Detection of Aggressive Prostate Cancer , 2017, ArXiv.

[9]  Max Welling,et al.  Visualizing Deep Neural Network Decisions: Prediction Difference Analysis , 2017, ICLR.

[10]  Dinggang Shen,et al.  3D Deep Learning for Multi-modal Imaging-Guided Survival Time Prediction of Brain Tumor Patients , 2016, MICCAI.

[11]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[12]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[13]  Martín Abadi,et al.  Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data , 2016, ICLR.

[14]  Bram van Ginneken,et al.  Towards automatic pulmonary nodule management in lung cancer screening with deep learning , 2016, Scientific Reports.

[15]  K L Lam,et al.  Computer-aided detection of mammographic microcalcifications: pattern recognition with an artificial neural network. , 1995, Medical physics.

[16]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.

[17]  Feng Lin,et al.  Low-Dose CT With a Residual Encoder-Decoder Convolutional Neural Network , 2017, IEEE Transactions on Medical Imaging.

[18]  Bram van Ginneken,et al.  Improving airway segmentation in computed tomography using leak detection with convolutional networks , 2017, Medical Image Anal..

[19]  Matthew T. Freedman,et al.  Computer-assisted diagnosis of lung nodule detection using artificial convoultion neural network , 1993 .

[20]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[21]  Jialin Peng,et al.  Automatic 3D liver segmentation based on deep learning and globally optimized surface evolution , 2016, Physics in medicine and biology.

[22]  Z. Jane Wang,et al.  A CNN Regression Approach for Real-Time 2D/3D Registration , 2016, IEEE Transactions on Medical Imaging.

[23]  Xiao Yang,et al.  Fast Predictive Image Registration , 2016, LABELS/DLMIA@MICCAI.

[24]  Hayit Greenspan,et al.  Content-Based Image Retrieval in Radiology: Current Status and Future Directions , 2010, Journal of Digital Imaging.

[25]  Elena Marchiori,et al.  Location Sensitive Deep Convolutional Neural Networks for Segmentation of White Matter Hyperintensities , 2016, Scientific Reports.

[26]  Lisa Tang,et al.  Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation , 2016, IEEE Transactions on Medical Imaging.

[27]  Dinggang Shen,et al.  Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis , 2014, MICCAI.

[28]  Ronald M. Summers,et al.  Spatial Aggregation of Holistically-Nested Networks for Automated Pancreas Segmentation , 2016, MICCAI.

[29]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[30]  Nico Karssemeijer,et al.  Deep multi-scale location-aware 3D convolutional neural networks for automated detection of lacunes of presumed vascular origin , 2016, NeuroImage: Clinical.

[31]  Ayman El-Baz,et al.  Alzheimer's disease diagnostics by adaptation of 3D convolutional network , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[32]  Lyle J. Palmer,et al.  Precision Radiology: Predicting longevity using feature engineering and deep learning methods in a radiomics framework , 2017, Scientific Reports.

[33]  Christopher Joseph Pal,et al.  Convolutional networks for kidney segmentation in contrast-enhanced CT scans , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[34]  Dinggang Shen,et al.  Segmenting Hippocampus from Infant Brains by Sparse Patch Matching with Deep-Learned Features , 2014, MICCAI.

[35]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[36]  Li Zhang,et al.  Deep similarity learning for multimodal medical images , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[37]  Andrew Zisserman,et al.  SpineNet: Automatically Pinpointing Classification Evidence in Spinal MRIs , 2016, MICCAI.

[38]  Hao Chen,et al.  Multilevel Contextual 3-D CNNs for False Positive Reduction in Pulmonary Nodule Detection , 2017, IEEE Transactions on Biomedical Engineering.

[39]  Konstantinos Kamnitsas,et al.  Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks , 2016, MICCAI.

[40]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[41]  Hao Chen,et al.  Volumetric ConvNets with Mixed Residual Connections for Automated Prostate Segmentation from 3D MR Images , 2017, AAAI.

[42]  Lisa Tang,et al.  Deep Learning of Brain Lesion Patterns for Predicting Future Disease Activity in Patients with Early Symptoms of Multiple Sclerosis , 2016, LABELS/DLMIA@MICCAI.

[43]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[44]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[45]  Eric A. Hoffman,et al.  Extraction of Airways From CT (EXACT'09) , 2012, IEEE Transactions on Medical Imaging.