Applications of deep learning for the analysis of medical data

Over the past decade, deep learning has demonstrated superior performances in solving many problems in various fields of medicine compared with other machine learning methods. To understand how deep learning has surpassed traditional machine learning techniques, in this review, we briefly explore the basic learning algorithms underlying deep learning. In addition, the procedures for building deep learning-based classifiers for seizure electroencephalograms and gastric tissue slides are described as examples to demonstrate the simplicity and effectiveness of deep learning applications. Finally, we review the clinical applications of deep learning in radiology, pathology, and drug discovery, where deep learning has been actively adopted. Considering the great advantages of deep learning techniques, deep learning will be increasingly and widely utilized in a wide variety of different areas in medicine in the coming decades.

[1]  Saeed Hassanpour,et al.  Deep Learning for Classification of Colorectal Polyps on Whole-slide Images , 2017, Journal of pathology informatics.

[2]  Germain Forestier,et al.  Detection of lobular structures in normal breast tissue , 2016, Comput. Biol. Medicine.

[3]  Lu Zhang,et al.  From machine learning to deep learning: progress in machine intelligence for rational drug discovery. , 2017, Drug discovery today.

[4]  June-Goo Lee,et al.  Deep Learning in Medical Imaging: General Overview , 2017, Korean journal of radiology.

[5]  Alexander Tropsha,et al.  Best Practices for QSAR Model Development, Validation, and Exploitation , 2010, Molecular informatics.

[6]  Yaozong Gao,et al.  Deformable MR Prostate Segmentation via Deep Feature Learning and Sparse Patch Matching , 2016, IEEE Transactions on Medical Imaging.

[7]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[8]  Ming Wen,et al.  Deep-Learning-Based Drug-Target Interaction Prediction. , 2017, Journal of proteome research.

[9]  Anne E Carpenter,et al.  Opportunities and obstacles for deep learning in biology and medicine , 2017, bioRxiv.

[10]  C. Pal,et al.  Deep Learning: A Primer for Radiologists. , 2017, Radiographics : a review publication of the Radiological Society of North America, Inc.

[11]  P. Lakhani,et al.  Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks. , 2017, Radiology.

[12]  R S Weinstein,et al.  Expert systems: robot physicians of the future? , 1985, Human pathology.

[13]  Michel E. Vandenberghe,et al.  Relevance of deep learning to facilitate the diagnosis of HER2 status in breast cancer , 2017, Scientific Reports.

[14]  J. Elmore,et al.  Diagnostic concordance among pathologists interpreting breast biopsy specimens. , 2015, JAMA.

[15]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[16]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[17]  Feng Liu,et al.  Deep Learning and Its Applications in Biomedicine , 2018, Genom. Proteom. Bioinform..

[18]  Bram van Ginneken,et al.  Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks , 2016, IEEE Transactions on Medical Imaging.

[19]  Daniel L. Rubin,et al.  Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions , 2017, Journal of Digital Imaging.

[20]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[21]  Lubomir M. Hadjiiski,et al.  Bladder Cancer Segmentation in CT for Treatment Response Assessment: Application of Deep-Learning Convolution Neural Network—A Pilot Study , 2016, Tomography.

[22]  Isaac S Kohane,et al.  Artificial Intelligence in Healthcare , 2019, Artificial Intelligence and Machine Learning for Business for Non-Engineers.

[23]  Jae Yong Ryu,et al.  Deep learning improves prediction of drug–drug and drug–food interactions , 2018, Proceedings of the National Academy of Sciences.

[24]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[25]  Hyun-Jong Jang,et al.  Dual deep neural network-based classifiers to detect experimental seizures , 2019, The Korean journal of physiology & pharmacology : official journal of the Korean Physiological Society and the Korean Society of Pharmacology.

[26]  Hao Chen,et al.  Automatic Detection of Cerebral Microbleeds From MR Images via 3D Convolutional Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[27]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[28]  J. Epstein,et al.  The value of mandatory second opinion pathology review of prostate needle biopsy interpretation before radical prostatectomy. , 2010, The Journal of urology.

[29]  Max A. Viergever,et al.  Automatic Segmentation of MR Brain Images With a Convolutional Neural Network , 2016, IEEE Transactions on Medical Imaging.

[30]  B. van Ginneken,et al.  Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis , 2016, Scientific Reports.

[31]  Le Lu,et al.  Pancreas Segmentation in MRI Using Graph-Based Decision Fusion on Convolutional Neural Networks , 2016, MICCAI.

[32]  Gang Fu,et al.  PubChem Substance and Compound databases , 2015, Nucleic Acids Res..

[33]  Tae-Yeong Kwak,et al.  Artificial Intelligence in Pathology , 2018, Journal of pathology and translational medicine.

[34]  Tao Liu,et al.  Searching for prostate cancer by fully automated magnetic resonance imaging classification: deep learning versus non-deep learning , 2017, Scientific Reports.

[35]  Lisa Tang,et al.  Deep 3D Convolutional Encoder Networks With Shortcuts for Multiscale Feature Integration Applied to Multiple Sclerosis Lesion Segmentation , 2016, IEEE Transactions on Medical Imaging.

[36]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[37]  Navid Farahani,et al.  whole slide imaging in pathology: advantages, limitations, and emerging perspectives , 2015 .

[38]  Eyal Toledano,et al.  Malignancy Detection on Mammography Using Dual Deep Convolutional Neural Networks and Genetically Discovered False Color Input Enhancement , 2017, Journal of Digital Imaging.

[39]  Chin-Teng Lin,et al.  A review of clustering techniques and developments , 2017, Neurocomputing.

[40]  Fang Lu,et al.  Automatic 3D liver location and segmentation via convolutional neural network and graph cut , 2016, International Journal of Computer Assisted Radiology and Surgery.

[41]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.

[42]  Nico Karssemeijer,et al.  Large scale deep learning for computer aided detection of mammographic lesions , 2017, Medical Image Anal..

[43]  Martin Urschler,et al.  Segmentation and classification of colon glands with deep convolutional neural networks and total variation regularization , 2017, PeerJ.

[44]  Pierre Baldi,et al.  Deep Architectures and Deep Learning in Chemoinformatics: The Prediction of Aqueous Solubility for Drug-Like Molecules , 2013, J. Chem. Inf. Model..

[45]  Marios Anthimopoulos,et al.  Multi-source Transfer Learning with Convolutional Neural Networks for Lung Pattern Analysis , 2016, IEEE journal of biomedical and health informatics.

[46]  Asad U Khan,et al.  Structure based virtual screening to discover putative drug candidates: necessary considerations and successful case studies. , 2015, Methods.

[47]  Max A. Viergever,et al.  Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks , 2016, Medical Image Anal..

[48]  A. Anderson The process of structure-based drug design. , 2003, Chemistry & biology.

[49]  Xavier Lladó,et al.  Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review , 2017, Artif. Intell. Medicine.

[50]  Daisuke Komura,et al.  Machine Learning Methods for Histopathological Image Analysis , 2017, Computational and structural biotechnology journal.

[51]  Ronald M. Summers,et al.  Holistic classification of CT attenuation patterns for interstitial lung diseases via deep convolutional neural networks , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[52]  Guang-Zhong Yang,et al.  Deep Learning for Health Informatics , 2017, IEEE Journal of Biomedical and Health Informatics.

[53]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[54]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[55]  Wenqing Sun,et al.  Enhancing deep convolutional neural network scheme for breast cancer diagnosis with unlabeled data , 2017, Comput. Medical Imaging Graph..

[56]  Alex Zhavoronkov,et al.  Applications of Deep Learning in Biomedicine. , 2016, Molecular pharmaceutics.

[57]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[58]  Edward Kim,et al.  A deep semantic mobile application for thyroid cytopathology , 2016, SPIE Medical Imaging.

[59]  H. K. Huang,et al.  Short history of PACS. Part I: USA. , 2011, European journal of radiology.

[60]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[61]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[62]  Marianne U. Prey,et al.  The AutoPap System for Primary Screening in Cervical Cytology , 1998, Acta Cytologica.

[63]  Ronald M. Summers,et al.  Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation , 2015, IEEE Transactions on Medical Imaging.

[64]  Luca Maria Gambardella,et al.  Assessment of algorithms for mitosis detection in breast cancer histopathology images , 2014, Medical Image Anal..

[65]  Michael K. Gilson,et al.  BindingDB in 2015: A public database for medicinal chemistry, computational chemistry and systems pharmacology , 2015, Nucleic Acids Res..

[66]  Bram van Ginneken,et al.  Improving airway segmentation in computed tomography using leak detection with convolutional networks , 2017, Medical Image Anal..

[67]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[68]  Ayman El-Baz,et al.  Alzheimer's Disease Diagnostics by a Deeply Supervised Adaptable 3D Convolutional Network , 2016, ArXiv.

[69]  Fabio A. González,et al.  Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks , 2014, Medical Imaging.

[70]  Eui Jin Hwang,et al.  Development and Validation of Deep Learning-based Automatic Detection Algorithm for Malignant Pulmonary Nodules on Chest Radiographs. , 2019, Radiology.

[71]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[72]  Dinggang Shen,et al.  Multi‐channel multi‐scale fully convolutional network for 3D perivascular spaces segmentation in 7T MR images , 2018, Medical Image Anal..

[73]  Qingmao Hu,et al.  Lung nodule classification using deep feature fusion in chest radiography , 2017, Comput. Medical Imaging Graph..

[74]  Thomas Blaschke,et al.  The rise of deep learning in drug discovery. , 2018, Drug discovery today.

[75]  Alexander D. MacKerell,et al.  Recent advances in ligand-based drug design: relevance and utility of the conformationally sampled pharmacophore approach. , 2011, Current computer-aided drug design.

[76]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[77]  Zhenjie Yao,et al.  Applying Deep Learning to Individual and Community Health Monitoring Data: A Survey , 2018, International Journal of Automation and Computing.

[78]  Dmitrii Bychkov,et al.  Deep learning based tissue analysis predicts outcome in colorectal cancer , 2018, Scientific Reports.

[79]  K. Borgwardt,et al.  Machine Learning in Medicine , 2015, Mach. Learn. under Resour. Constraints Vol. 3.

[80]  Robert P. Sheridan,et al.  Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling , 2003, J. Chem. Inf. Comput. Sci..

[81]  Ronald M. Summers,et al.  Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning , 2016, IEEE Transactions on Medical Imaging.

[82]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[83]  Shu Liao,et al.  Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition , 2016, IEEE Transactions on Medical Imaging.

[84]  Nasir M. Rajpoot,et al.  Locality Sensitive Deep Learning for Detection and Classification of Nuclei in Routine Colon Cancer Histology Images , 2016, IEEE Trans. Medical Imaging.

[85]  George Papadatos,et al.  Activity, assay and target data curation and quality in the ChEMBL database , 2015, Journal of Computer-Aided Molecular Design.

[86]  J. Prewitt,et al.  Morphological Analysis of Cells and Chromosomes by Digital Computer , 1965, Methods of Information in Medicine.

[87]  Simon K. Warfield,et al.  Automatic renal segmentation in DCE-MRI using convolutional neural networks , 2017, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[88]  Jeffrey Dean,et al.  Machine Learning in Medicine , 2019, The New England journal of medicine.

[89]  Ronald M. Summers,et al.  Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[90]  Bram van Ginneken,et al.  Towards automatic pulmonary nodule management in lung cancer screening with deep learning , 2016, Scientific Reports.

[91]  Hiroyuki Yoshida,et al.  Deep transfer learning of virtual endoluminal views for the detection of polyps in CT colonography , 2016, SPIE Medical Imaging.

[92]  Ben Glocker,et al.  Automated cardiovascular magnetic resonance image analysis with fully convolutional networks , 2017, Journal of Cardiovascular Magnetic Resonance.

[93]  Fei Wang,et al.  Deep learning for healthcare: review, opportunities and challenges , 2018, Briefings Bioinform..

[94]  Kyong Hwan Jin,et al.  Fast and robust segmentation of the striatum using deep convolutional neural networks , 2016, Journal of Neuroscience Methods.