Deep learning for heterogeneous medical data analysis

At present, how to make use of massive medical information resources to provide scientific decision-making for the diagnosis and treatment of diseases, summarize the curative effect of various treatment schemes, and better serve the decision-making management, medical treatment, and scientific research, has drawn more and more attention of researchers. Deep learning, as the focus of most concern by both academia and industry, has been effectively applied in many fields and has outperformed most of the machine learning methods. Under this background, deep learning based medical data analysis emerged. In this survey, we focus on reviewing and then categorizing the current development. Firstly, we fully discuss the scope, characteristic and structure of the heterogeneous medical data. Afterward and primarily, the main deep learning models involved in medical data analysis, including their variants and various hybrid models, as well as main tasks in medical data analysis are all analyzed and reviewed in a series of typical cases respectively. Finally, we provide a brief introduction to certain useful online resources of deep learning development tools.

[1]  Hong Yu,et al.  Structured prediction models for RNN based sequence labeling in clinical text , 2016, EMNLP.

[2]  Li Li,et al.  Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records , 2016, Scientific Reports.

[3]  Guang-Zhong Yang,et al.  Deep Learning for Health Informatics , 2017, IEEE Journal of Biomedical and Health Informatics.

[4]  Athanasios V. Vasilakos,et al.  Fast and Scalable Distributed Deep Convolutional Autoencoder for fMRI Big Data Analytics , 2017, Neurocomputing.

[5]  Aidong Zhang,et al.  Identifying informative risk factors and predicting bone disease progression via deep belief networks. , 2014, Methods.

[6]  Na Lu,et al.  A Deep Learning Scheme for Motor Imagery Classification based on Restricted Boltzmann Machines , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  Hercules Dalianis,et al.  Applying deep learning on electronic health records in Swedish to predict healthcare-associated infections , 2016, BioNLP@ACL.

[8]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[9]  Charles Elkan,et al.  Learning to Diagnose with LSTM Recurrent Neural Networks , 2015, ICLR.

[10]  Parisa Rashidi,et al.  Deep neural network architectures for forecasting analgesic response , 2016, EMBC.

[11]  Patrick van der Smagt,et al.  CNN-based Segmentation of Medical Imaging Data , 2017, ArXiv.

[12]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[13]  Yu Cheng,et al.  Segment convolutional neural networks (Seg-CNNs) for classifying relations in clinical notes , 2018, J. Am. Medical Informatics Assoc..

[14]  Casey S. Greene,et al.  Semi-supervised learning of the electronic health record for phenotype stratification , 2016, J. Biomed. Informatics.

[15]  Carmen C. Y. Poon,et al.  Automatic Detection and Classification of Colorectal Polyps by Transferring Low-Level CNN Features From Nonmedical Domain , 2017, IEEE Journal of Biomedical and Health Informatics.

[16]  Pushpak Bhattacharyya,et al.  Deep Learning Architecture for Patient Data De-identification in Clinical Records , 2016, ClinicalNLP@COLING 2016.

[17]  Volker Tresp,et al.  Predicting Clinical Events by Combining Static and Dynamic Information Using Recurrent Neural Networks , 2016, 2016 IEEE International Conference on Healthcare Informatics (ICHI).

[18]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[19]  Albert Y. Zomaya,et al.  A Survey of Mobile Device Virtualization , 2016, ACM Comput. Surv..

[20]  Svetha Venkatesh,et al.  DeepCare: A Deep Dynamic Memory Model for Predictive Medicine , 2016, PAKDD.

[21]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[22]  Hossam Faris,et al.  An efficient hybrid multilayer perceptron neural network with grasshopper optimization , 2018, Soft Computing.

[23]  Nataliya Sokolovska,et al.  Using Unlabeled Data to Discover Bivariate Causality with Deep Restricted Boltzmann Machines , 2020, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[24]  Hua Xu,et al.  Named Entity Recognition in Chinese Clinical Text Using Deep Neural Network , 2015, MedInfo.

[25]  Hongfang Liu,et al.  Temporal Pattern and Association Discovery of Diagnosis Codes Using Deep Learning , 2015, 2015 International Conference on Healthcare Informatics.

[26]  Jimeng Sun,et al.  Using recurrent neural network models for early detection of heart failure onset , 2016, J. Am. Medical Informatics Assoc..

[27]  Svetha Venkatesh,et al.  $\mathtt {Deepr}$: A Convolutional Net for Medical Records , 2016, IEEE Journal of Biomedical and Health Informatics.

[28]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[29]  S. S. Vinod Chandra,et al.  Long-Term Forecasting the Survival in Liver Transplantation Using Multilayer Perceptron Networks , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[30]  Parisa Rashidi,et al.  Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis , 2017, IEEE Journal of Biomedical and Health Informatics.

[31]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[32]  Nilanjan Dey,et al.  A Survey of Data Mining and Deep Learning in Bioinformatics , 2018, Journal of Medical Systems.

[33]  Vipin Kumar,et al.  Mining Electronic Health Records: A Survey , 2017, 1702.03222.

[34]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[35]  Zhen Lin,et al.  Genomic Research and Human Subject Privacy , 2004, Science.

[36]  Nataliya Sokolovska,et al.  Risk Scores Learned by Deep Restricted Boltzmann Machines with Trained Interval Quantization , 2018, MLDM.

[37]  Xiangji Huang,et al.  Deep learning for healthcare decision making with EMRs , 2014, 2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[38]  Rema Padman,et al.  A Deep Learning Architecture for De-identification of Patient Notes: Implementation and Evaluation , 2018, ArXiv.

[39]  L. Sweeney.  Simple Demographics Often Identify People Uniquely , 2000 .

[40]  Christian Igel,et al.  Training restricted Boltzmann machines: An introduction , 2014, Pattern Recognit..

[41]  Jason Alan Fries.  Brundlefly at SemEval-2016 Task 12: Recurrent Neural Networks vs. Joint Inference for Clinical Temporal Information Extraction , 2016, *SEMEVAL.

[42]  Hayit Greenspan,et al.  GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification , 2018, Neurocomputing.

[43]  David Sontag,et al.  Learning Low-Dimensional Representations of Medical Concepts , 2016, CRI.

[44]  Franck Dernoncourt,et al.  De-identification of patient notes with recurrent neural networks , 2016, J. Am. Medical Informatics Assoc..

[45]  Walter F. Stewart,et al.  Doctor AI: Predicting Clinical Events via Recurrent Neural Networks , 2015, MLHC.

[46]  Marius George Linguraru,et al.  Deep Learning Guided Partitioned Shape Model for Anterior Visual Pathway Segmentation , 2016, IEEE Transactions on Medical Imaging.

[47]  Jimeng Sun,et al.  Medical Concept Representation Learning from Electronic Health Records and its Application on Heart Failure Prediction , 2016, ArXiv.

[48]  Svetha Venkatesh,et al.  Learning vector representation of medical objects via EMR-driven nonnegative restricted Boltzmann machines (eNRBM) , 2015, J. Biomed. Informatics.

[49]  Hugo Larochelle,et al.  Efficient Learning of Deep Boltzmann Machines , 2010, AISTATS.

[50]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[51]  Michael Bowie,et al.  Improving Quality of Observational Streaming Medical Data by Using Long Short-Term Memory Networks (LSTMs) , 2018, 2018 IEEE 34th International Conference on Data Engineering Workshops (ICDEW).

[52]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[53]  Klaus H. Maier-Hein,et al.  Deep MRI brain extraction: A 3D convolutional neural network for skull stripping , 2016, NeuroImage.

[54]  David Dagan Feng,et al.  An Ensemble of Fine-Tuned Convolutional Neural Networks for Medical Image Classification , 2017, IEEE Journal of Biomedical and Health Informatics.

[55]  Shu-Ching Chen,et al.  Computational Health Informatics in the Big Data Age , 2016, ACM Comput. Surv..

[56]  Philip S. Yu,et al.  On the Generative Discovery of Structured Medical Knowledge , 2018, KDD.

[57]  Jimeng Sun,et al.  RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data , 2018, KDD.

[58]  Harald Binder,et al.  Partitioned Learning of Deep Boltzmann Machines for SNP Data , 2016, bioRxiv.

[59]  Jinfeng Yang,et al.  Clinical Relation Extraction with Deep Learning , 2016 .

[60]  Jimeng Sun,et al.  RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records , 2018, IEEE Transactions on Visualization and Computer Graphics.

[61]  Amir-Masoud Eftekhari-Moghadam,et al.  Knowledge discovery in medicine: Current issue and future trend , 2014, Expert Syst. Appl..

[62]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[63]  Shiguang Shan,et al.  Representation Learning with Smooth Autoencoder , 2014, ACCV.

[64]  Marc'Aurelio Ranzato,et al.  Fast Inference in Sparse Coding Algorithms with Applications to Object Recognition , 2010, ArXiv.

[65]  ChenShu-Ching,et al.  Computational Health Informatics in the Big Data Age , 2016 .

[66]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[67]  Pascal Vincent,et al.  Contractive Auto-Encoders: Explicit Invariance During Feature Extraction , 2011, ICML.

[68]  Jimeng Sun,et al.  Multi-layer Representation Learning for Medical Concepts , 2016, KDD.

[69]  Meng Wang,et al.  Disease Inference from Health-Related Questions via Sparse Deep Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[70]  Saeid Nahavandi,et al.  A sequential search-space shrinking using CNN transfer learning and a Radon projection pool for medical image retrieval , 2018, Expert Syst. Appl..

[71]  Dinggang Shen,et al.  A Robust Deep Model for Improved Classification of AD/MCI Patients , 2015, IEEE Journal of Biomedical and Health Informatics.

[72]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[73]  Khawar Khurshid,et al.  An expert system for diabetes prediction using auto tuned multi-layer perceptron , 2017, 2017 Intelligent Systems Conference (IntelliSys).

[74]  Alan Rubel,et al.  Four ethical priorities for neurotechnologies and AI , 2017, Nature.

[75]  Hong Yu,et al.  Bidirectional RNN for Medical Event Detection in Electronic Health Records , 2016, NAACL.

[76]  Ping Zhang,et al.  Risk Prediction with Electronic Health Records: A Deep Learning Approach , 2016, SDM.