Deep Learning in Computer-Aided Diagnosis and Treatment of Tumors: A Survey

Computer-Aided Diagnosis and Treatment of Tumors is a hot topic of deep learning in recent years, which constitutes a series of medical tasks, such as detection of tumor markers, the outline of tumor leisures, subtypes and stages of tumors, prediction of therapeutic effect, and drug development. Meanwhile, there are some deep learning models with precise positioning and excellent performance produced in mainstream task scenarios. Thus follow to introduce deep learning methods from task-orient, mainly focus on the improvements for medical tasks. Then to summarize the recent progress in four stages of tumor diagnosis and treatment, which named In-Vitro Diagnosis (IVD), Imaging Diagnosis (ID), Pathological Diagnosis (PD), and Treatment Planning (TP). According to the specific data types and medical tasks of each stage, we present the applications of deep learning in the Computer-Aided Diagnosis and Treatment of Tumors and analyzing the excellent works therein. This survey concludes by discussing research issues and suggesting challenges for future improvement.

[1]  D. Maintz,et al.  Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric MRI , 2018, European Radiology.

[2]  Zhenghua Xu,et al.  Segmentation is All You Need , 2019, ArXiv.

[3]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Ilya Sutskever,et al.  Generating Long Sequences with Sparse Transformers , 2019, ArXiv.

[5]  Xiaodong Wu,et al.  3D fully convolutional networks for co-segmentation of tumors on PET-CT images , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[6]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  A. Jemal,et al.  Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries , 2018, CA: a cancer journal for clinicians.

[8]  Leonid Karlinsky,et al.  A Region Based Convolutional Network for Tumor Detection and Classification in Breast Mammography , 2016, LABELS/DLMIA@MICCAI.

[9]  Suk-Hwan Lee,et al.  Blood Cell Images Segmentation using Deep Learning Semantic Segmentation , 2018, 2018 IEEE International Conference on Electronics and Communication Engineering (ICECE).

[10]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Alex Lallement,et al.  Survey on deep learning for radiotherapy , 2018, Comput. Biol. Medicine.

[12]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[13]  Lyle J. Palmer,et al.  Precision Radiology: Predicting longevity using feature engineering and deep learning methods in a radiomics framework , 2017, Scientific Reports.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Kumardeep Chaudhary,et al.  Deep Learning–Based Multi-Omics Integration Robustly Predicts Survival in Liver Cancer , 2017, Clinical Cancer Research.

[16]  Steve B. Jiang,et al.  A feasibility study for predicting optimal radiation therapy dose distributions of prostate cancer patients from patient anatomy using deep learning , 2017, Scientific Reports.

[17]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Dinggang Shen,et al.  Segmentation of Organs at Risk in thoracic CT images using a SharpMask architecture and Conditional Random Fields , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[20]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[21]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[22]  Ehsan Kazemi,et al.  Deep Convolutional Neural Networks Enable Discrimination of Heterogeneous Digital Pathology Images , 2017, bioRxiv.

[23]  H. Aerts,et al.  Deep Learning for Fully-Automated Localization and Segmentation of Rectal Cancer on Multiparametric MR , 2017, Scientific Reports.

[24]  Dmitrii Bychkov,et al.  Deep learning based tissue analysis predicts outcome in colorectal cancer , 2018, Scientific Reports.

[25]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[26]  Jin Liu,et al.  Applications of deep learning to MRI images: A survey , 2018, Big Data Min. Anal..

[27]  Bruce R. Rosen,et al.  Sequential 3D U-Nets for Biologically-Informed Brain Tumor Segmentation , 2017, ArXiv.

[28]  Catarina Eloy,et al.  Classification of breast cancer histology images using Convolutional Neural Networks , 2017, PloS one.

[29]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[31]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[32]  Robert J. Gillies,et al.  Quantitative imaging of cancer in the postgenomic era: Radio(geno)mics, deep learning, and habitats , 2018, Cancer.

[33]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[34]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[35]  Yong Fan,et al.  A deep learning model integrating FCNNs and CRFs for brain tumor segmentation , 2017, Medical Image Anal..

[36]  Frank Thiele,et al.  Clinical Evaluation of a Multiparametric Deep Learning Model for Glioblastoma Segmentation Using Heterogeneous Magnetic Resonance Imaging Data From Clinical Routine , 2018, Investigative radiology.

[37]  Dong Ni,et al.  Deep Learning in Medical Ultrasound Analysis: A Review , 2019, Engineering.

[38]  Carsten Denkert,et al.  MALDI‐Imaging for Classification of Epithelial Ovarian Cancer Histotypes from a Tissue Microarray Using Machine Learning Methods , 2018, Proteomics. Clinical applications.

[39]  Sebastian J. Schlecht,et al.  Automatic Liver and Tumor Segmentation of CT and MRI Volumes using Cascaded Fully Convolutional Neural Networks , 2017, ArXiv.

[40]  Bingbing Ni,et al.  3D Deep Learning from CT Scans Predicts Tumor Invasiveness of Subcentimeter Pulmonary Adenocarcinomas. , 2018, Cancer research.

[41]  Andrew L. Beam,et al.  Adversarial attacks on medical machine learning , 2019, Science.

[42]  Anne E Carpenter,et al.  Opportunities and obstacles for deep learning in biology and medicine , 2017, bioRxiv.

[43]  Abhinav K. Jha,et al.  A deep-learning-based fully automated segmentation approach to delineate tumors in FDG-PET images of patients with lung cancer , 2018 .

[44]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[45]  Feng Liu,et al.  Deep Learning and Its Applications in Biomedicine , 2018, Genom. Proteom. Bioinform..

[46]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[47]  Qiaoliang Li,et al.  Tumor Segmentation in Contrast-Enhanced Magnetic Resonance Imaging for Nasopharyngeal Carcinoma: Deep Learning with Convolutional Neural Network , 2018, BioMed research international.

[48]  B. S. Manjunath,et al.  Brain Tumor Segmentation and Tractographic Feature Extraction from Structural MR Images for Overall Survival Prediction , 2018, BrainLes@MICCAI.

[49]  Berkman Sahiner,et al.  Deep learning in medical imaging and radiation therapy. , 2018, Medical physics.

[50]  Anant Madabhushi,et al.  Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent , 2017, Scientific Reports.

[51]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[52]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Steve B. Jiang,et al.  Dose Prediction with U-net: A Feasibility Study for Predicting Dose Distributions from Contours using Deep Learning on Prostate IMRT Patients , 2017, ArXiv.

[54]  Steve B. Jiang,et al.  3D radiotherapy dose prediction on head and neck cancer patients with a hierarchically densely connected U-net deep learning architecture , 2018, Physics in medicine and biology.

[55]  Osamu Abe,et al.  Deep learning with convolutional neural network in radiology , 2018, Japanese Journal of Radiology.

[56]  Klaus H. Maier-Hein,et al.  Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge , 2017, BrainLes@MICCAI.

[57]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[58]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[59]  Bahram Parvin,et al.  Deep Learning Models Differentiate Tumor Grades from H&E Stained Histology Sections , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[60]  Luc Van Gool,et al.  Detection-aided liver lesion segmentation using deep learning , 2017, NIPS 2017.

[61]  Tae Soon Kim,et al.  Cancer Drug Response Profile scan (CDRscan): A Deep Learning Model That Predicts Drug Effectiveness from Cancer Genomic Signature , 2018, Scientific Reports.

[62]  Muhammad Imran Razzak,et al.  Deep Learning for Medical Image Processing: Overview, Challenges and Future , 2017, ArXiv.

[63]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[64]  Lipo Wang,et al.  Deep Learning Applications in Medical Image Analysis , 2018, IEEE Access.

[65]  Stephen J. McKenna,et al.  Boundary-Aware Fully Convolutional Network for Brain Tumor Segmentation , 2017, MICCAI.

[66]  Mateusz Buda,et al.  Deep learning in radiology: An overview of the concepts and a survey of the state of the art with focus on MRI , 2018, Journal of magnetic resonance imaging : JMRI.

[67]  Tomas Kron,et al.  Deep Learning Renal Segmentation for Fully Automated Radiation Dose Estimation in Unsealed Source Therapy , 2018, Front. Oncol..

[68]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[69]  Silvia Conforto,et al.  Automatic segmentation of colorectal cancer in 3D MRI by combining deep learning and 3D level-set algorithm-a preliminary study , 2018, 2018 IEEE 4th Middle East Conference on Biomedical Engineering (MECBME).

[70]  Milan Sonka,et al.  Deep LOGISMOS: Deep learning graph-based 3D segmentation of pancreatic tumors on CT scans , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[71]  Zachary Chase Lipton A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[72]  Pablo Arbeláez,et al.  Volumetric multimodality neural network for brain tumor segmentation , 2017, Symposium on Medical Information Processing and Analysis.

[73]  Kai Zhang,et al.  Deep learning for image-based cancer detection and diagnosis - A survey , 2018, Pattern Recognit..

[74]  Bradley J Erickson,et al.  A Survey of Deep-Learning Applications in Ultrasound: Artificial Intelligence-Powered Ultrasound for Improving Clinical Workflow. , 2019, Journal of the American College of Radiology : JACR.

[75]  Zhenyu Liu,et al.  Deep learning provides a new computed tomography-based prognostic biomarker for recurrence prediction in high-grade serous ovarian cancer. , 2019, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[76]  Jun Wu,et al.  A deep learning-based multi-model ensemble method for cancer prediction , 2018, Comput. Methods Programs Biomed..

[77]  Ronald M. Summers,et al.  Personalized Pancreatic Tumor Growth Prediction via Group Learning , 2017, MICCAI.

[78]  István Csabai,et al.  Detecting and classifying lesions in mammograms with Deep Learning , 2017, Scientific Reports.

[79]  Linghong Zhou,et al.  Deep convolutional neural network with transfer learning for rectum toxicity prediction in cervical cancer radiotherapy: a feasibility study , 2017, Physics in medicine and biology.

[80]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[81]  Konstantinos Kamnitsas,et al.  Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation , 2017, BrainLes@MICCAI.

[82]  Andrew Janowczyk,et al.  A resolution adaptive deep hierarchical (RADHicaL) learning scheme applied to nuclear segmentation of digital pathology images , 2018, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[83]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[84]  Thomas Brox,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[85]  Siddhant Rao,et al.  MITOS-RCNN: A Novel Approach to Mitotic Figure Detection in Breast Cancer Histopathology Images using Region Based Convolutional Neural Networks , 2018, ArXiv.

[86]  Yiming Yang,et al.  Transformer-XL: Attentive Language Models beyond a Fixed-Length Context , 2019, ACL.

[87]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[88]  Leonid Karlinsky,et al.  A CNN based method for automatic mass detection and classification in mammograms , 2019, Comput. methods Biomech. Biomed. Eng. Imaging Vis..

[89]  Christoph Meinel,et al.  Deep Learning for Medical Image Analysis , 2018, Journal of Pathology Informatics.

[90]  Sarang C. Joshi,et al.  Real-Time 2D-3D Deformable Registration with Deep Learning and Application to Lung Radiotherapy Targeting , 2019, IPMI.

[91]  Seyed-Ahmad Ahmadi,et al.  Hough-CNN: Deep learning for segmentation of deep brain regions in MRI and ultrasound , 2016, Comput. Vis. Image Underst..

[92]  Amit Sethi,et al.  Classification of Breast Cancer Histology using Deep Learning , 2018, ICIAR.

[93]  Wei Cao,et al.  Deep Learning based Radiomics (DLR) and its usage in noninvasive IDH1 prediction for low grade glioma , 2017, Scientific Reports.

[94]  Christopher Joseph Pal,et al.  The Importance of Skip Connections in Biomedical Image Segmentation , 2016, LABELS/DLMIA@MICCAI.

[95]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[96]  Olaf Wolkenhauer,et al.  Diagnostic Potential of Imaging Flow Cytometry. , 2018, Trends in biotechnology.

[97]  Rajarsi R. Gupta,et al.  Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images. , 2018, Cell reports.

[98]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[99]  Bram van Ginneken,et al.  Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard , 2018, Scientific Reports.

[100]  Mohamed Akil,et al.  Fully Automatic Brain Tumor Segmentation using End-To-End Incremental Deep Neural Networks in MRI images , 2018, Comput. Methods Programs Biomed..

[101]  Weiwei Liu,et al.  An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images , 2018, Scientific Reports.

[102]  Xianhe Sun,et al.  Efficient Mitosis Detection in Breast Cancer Histology Images by RCNN , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[103]  Daiju Ueda,et al.  Technical and clinical overview of deep learning in radiology , 2018, Japanese Journal of Radiology.

[104]  Christopher Joseph Pal,et al.  Learning normalized inputs for iterative estimation in medical image segmentation , 2017, Medical Image Anal..

[105]  Andrew J. Schaumberg,et al.  D R A F T H&E-stained Whole Slide Image Deep Learning Predicts SPOP Mutation State in Prostate Cancer , 2017 .

[106]  G. Corrado,et al.  End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography , 2019, Nature Medicine.