Survey of Deep Learning Applications to Medical Image Analysis

Recently, a machine learning (ML) area called deep learning emerged in the computer-vision field and became very popular in many fields. It started from an event in late 2012, when a deep-learning approach based on a convolutional neural network (CNN) won an overwhelming victory in the best-known worldwide com putervision competition, ImageNet Classification. Since then, researchers in many fields, including medical image analysis, have started actively participating in the explosively growing field of deep learning. In this paper, deep learning techniques and their applications to medical image analysis are surveyed. This survey overviewed 1) standard ML techniques in the computer-vision field, 2) what has changed in ML before and after the introduction of deep learning, 3) ML models in deep learning, and 4) applications of deep learning to medical image analysis. The comparisons between MLs before and after deep learning revealed that ML with feature input (or feature-based ML) was dominant before the introduction of deep learning, and that the major and essential difference between ML before and after deep learning is learning image data directly without object segmentation or feature extraction; thus, it is the source of the power of deep learning, although the depth of the model is an important attribute. The survey of deep learning also revealed that there is a long history of deep-learning techniques in the class of ML with image input, except a new term, “deep learning”. “Deep learning” even before the term existed, namely, the class of ML with image input was applied to various problems in medical image analysis including classification between lesions and non-lesions, classification between lesion types, segmentation of lesions or organs, and detection of lesions. ML with image input including deep learning is a very powerful, versatile technology with higher performance, which can bring the current state-of-the-art performance level of medical image analysis to the next level, and it is expected that deep learning will be the mainstream technology in medical image analysis in the next few decades. “Deep learning”, or ML with image input, in medical image analysis is an explosively growing, promising field. It is expected that ML with image input will be the mainstream area in the field of medical image analysis in the next few decades.

[1]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[2]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[3]  T. Ishigaki,et al.  One-shot dual-energy subtraction imaging. , 1986, Radiology.

[4]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[5]  J. Austin,et al.  Missed bronchogenic carcinoma: radiographic findings in 27 patients with a potentially resectable lesion evident in retrospect. , 1992, Radiology.

[6]  K Doi,et al.  Computerized detection of clustered microcalcifications in digital mammograms using a shift-invariant artificial neural network. , 1994, Medical physics.

[7]  Matthew T. Freedman,et al.  Artificial convolution neural network techniques and applications for lung nodule detection , 1995, IEEE Trans. Medical Imaging.

[8]  Kenji Suzuki,et al.  Recognition of Coronary Arterial Stenosis Using Neural Network on DSA System , 1995, Systems and Computers in Japan.

[9]  Huai Li,et al.  Artificial convolution neural network for medical image pattern recognition , 1995, Neural Networks.

[10]  Matthew T. Freedman,et al.  Reduction of false positives in lung nodule detection using a two-level neural classification , 1996, IEEE Trans. Medical Imaging.

[11]  Kazuyoshi Itoh,et al.  Generalization of shift invariant neural networks: Image processing of corneal endothelium , 1996, Neural Networks.

[12]  K Doi,et al.  An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms. , 1996, Medical physics.

[13]  Berkman Sahiner,et al.  Classification of mass and normal breast tissue: a convolution neural network classifier with spatial domain and texture images , 1996, IEEE Trans. Medical Imaging.

[14]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[15]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[16]  Kenji Suzuki,et al.  Edge detection from noisy images using a neural edge detector , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[17]  Ken-ichi Suzuki,et al.  Neural Filter with Selection of Input Features and Its Application to Image Quality Improvement of Medical Image Sequences , 2002 .

[18]  Kenji Suzuki,et al.  Efficient approximation of neural filters for removing quantum noise from images , 2002, IEEE Trans. Signal Process..

[19]  Huai Li,et al.  A multiple circular path convolution neural network system for detection of mammographic masses , 2002, IEEE Transactions on Medical Imaging.

[20]  J. Austin,et al.  Missed non-small cell lung cancer: radiographic findings of potentially resectable lesions evident only in retrospect. , 2003, Radiology.

[21]  S. Armato,et al.  Massive training artificial neural network (MTANN) for reduction of false positives in computerized detection of lung nodules in low-dose computed tomography. , 2003, Medical physics.

[22]  Kenji Suzuki,et al.  Linear-time connected-component labeling based on sequential local operations , 2003, Comput. Vis. Image Underst..

[23]  Kenji Suzuki,et al.  Neural Edge Enhancer for Supervised Edge Enhancement from Noisy Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Kenji Suzuki,et al.  Determining the receptive field of a neural filter , 2004, Journal of neural engineering.

[25]  K. Doi,et al.  Computerized scheme for automated detection of lung nodules in low-dose computed tomography images for lung cancer screening. , 2004, Academic radiology.

[26]  Kenji Suzuki,et al.  Extraction of left ventricular contours from left ventriculograms by means of a neural edge detector , 2004, IEEE Transactions on Medical Imaging.

[27]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[28]  Kenji Suzuki,et al.  A Simple Neural Network Pruning Algorithm with Application to Filter Synthesis , 2001, Neural Processing Letters.

[29]  K. Doi.  Overview on research and development of computer-aided diagnostic schemes. , 2004, Seminars in ultrasound, CT, and MR.

[30]  Kunio Doi,et al.  Suppression of the contrast of ribs in chest radiographs by means of massive training artificial neural network , 2004, SPIE Medical Imaging.

[31]  K. Doi,et al.  False-positive reduction in computer-aided diagnostic scheme for detecting nodules in chest radiographs by means of massive training artificial neural network. , 2005, Academic radiology.

[32]  K. Doi,et al.  Computer-aided detection of peripheral lung cancers missed at CT: ROC analyses without and with localization. , 2005, Radiology.

[33]  Kunio Doi,et al.  Computer-aided diagnostic scheme for distinction between benign and malignant nodules in thoracic low-dose CT by use of massive training artificial neural network , 2005, IEEE Transactions on Medical Imaging.

[34]  K. Doi,et al.  Current status and future potential of computer-aided diagnosis in medical imaging. , 2005, The British journal of radiology.

[35]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[36]  Kunio Doi,et al.  Diagnostic imaging over the last 50 years: research and development in medical imaging science and technology , 2006, Physics in medicine and biology.

[37]  Hiroyuki Yoshida,et al.  Massive-training artificial neural network (MTANN) for reduction of false positives in computer-aided detection of polyps: Suppression of rectal tubes. , 2006, Medical physics.

[38]  Kunio Doi,et al.  Image-processing technique for suppressing ribs in chest radiographs by means of massive training artificial neural network (MTANN) , 2006, IEEE Transactions on Medical Imaging.

[39]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[40]  Bram van Ginneken,et al.  Segmentation of the posterior ribs in chest radiographs using iterated contextual pixel classification , 2006, IEEE Transactions on Medical Imaging.

[41]  Kunio Doi,et al.  Computer-aided diagnosis in medical imaging: Historical review, current status and future potential , 2007, Comput. Medical Imaging Graph..

[42]  S. Armato,et al.  Mixture of expert 3D massive-training ANNs for reduction of multiple types of false positives in CAD for detection of polyps in CT colonography. , 2008, Medical physics.

[43]  Kenji Suzuki,et al.  A Run-Based Two-Scan Labeling Algorithm , 2008, IEEE Transactions on Image Processing.

[44]  Kenji Suzuki,et al.  Flat lesions in CT colonography , 2010, Abdominal Imaging.

[45]  Kesheng Wu,et al.  Fast connected-component labeling , 2009, Pattern Recognit..

[46]  Kenji Suzuki.  A supervised 'lesion-enhancement' filter by use of a massive-training artificial neural network (MTANN) in computer-aided diagnosis (CAD). , 2009, Physics in medicine and biology.

[47]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[48]  H. MacMahon,et al.  Performance of radiologists in detection of small pulmonary nodules on chest radiographs: effect of rib suppression with a massive-training artificial neural network. , 2009, AJR. American journal of roentgenology.

[49]  Kenji Suzuki,et al.  CT colonography: advanced computer-aided detection scheme utilizing MTANNs for detection of "missed" polyps in a multicenter clinical trial. , 2009, Medical physics.

[50]  Kenji Suzuki,et al.  Massive-Training Artificial Neural Network Coupled With Laplacian-Eigenfunction-Based Dimensionality Reduction for Computer-Aided Detection of Polyps in CT Colonography , 2010, IEEE Transactions on Medical Imaging.

[51]  Kenji Suzuki,et al.  Comparison of 2D and 3D views for evaluation of flat lesions in CT colonography. , 2010, Academic radiology.

[52]  Kenji Suzuki,et al.  Massive-training support vector regression and Gaussian process for false-positive reduction in computer-aided detection of polyps in CT colonography. , 2011, Medical physics.

[53]  Dinggang Shen,et al.  Machine Learning in Medical Imaging , 2012, Lecture Notes in Computer Science.

[54]  Kenji Suzuki,et al.  Pixel-Based Machine Learning in Medical Imaging , 2012, Int. J. Biomed. Imaging.

[55]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[56]  Kenji Suzuki,et al.  Machine Learning in Computer-Aided Diagnosis of the Thorax and Colon in CT: A Survey , 2013, IEICE Trans. Inf. Syst..

[57]  Kenji Suzuki.  Computational intelligence in biomedical imaging , 2014 .

[58]  Ronald M. Summers,et al.  A New 2.5D Representation for Lymph Node Detection Using Random Sets of Deep Convolutional Neural Network Observations , 2014, MICCAI.

[59]  Bram van Ginneken,et al.  Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box , 2015, Medical Image Anal..

[60]  Satoshi Suzuki,et al.  A Transfer Learning Method with Deep Convolutional Neural Network for Diffuse Lung Disease Classification , 2015, ICONIP.

[61]  Ronald M. Summers,et al.  DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation , 2015, MICCAI.

[62]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[63]  Lubomir M. Hadjiiski,et al.  Mass detection in digital breast tomosynthesis: Deep convolutional neural network with transfer learning from mammography. , 2016, Medical physics.

[64]  Song Wang,et al.  First trial and evaluation of anatomical structure segmentations in 3D CT images based only on deep learning , 2016 .

[65]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[66]  Marleen de Bruijne,et al.  Combining Generative and Discriminative Representation Learning for Lung CT Analysis With Convolutional Restricted Boltzmann Machines , 2016, IEEE Transactions on Medical Imaging.

[67]  Max A. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[68]  Lubomir M. Hadjiiski,et al.  Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets. , 2016, Medical physics.

[69]  Hao Chen,et al.  Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[70]  Hiroshi Fujita,et al.  Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique. , 2016, Medical physics.

[71]  Ayman El-Baz,et al.  Machine Learning Applications in Medical Image Analysis , 2017, Comput. Math. Methods Medicine.

[72]  Xiangrong Zhou,et al.  Classification of teeth in cone-beam CT using deep convolutional neural network , 2017, Comput. Biol. Medicine.

[73]  N. Karssemeijer,et al.  Discriminating solitary cysts from soft tissue lesions in mammography using a pretrained deep convolutional neural network , 2017, Medical physics.

[74]  Petia Radeva,et al.  A Convolutional Neural Network for Automatic Characterization of Plaque Composition in Carotid Ultrasound , 2017, IEEE Journal of Biomedical and Health Informatics.