Deep learning for cardiovascular medicine: a practical primer.

Deep learning (DL) is a branch of machine learning (ML) showing increasing promise in medicine, to assist in data classification, novel disease phenotyping and complex decision making. Deep learning is a form of ML typically implemented via multi-layered neural networks. Deep learning has accelerated by recent advances in computer hardware and algorithms and is increasingly applied in e-commerce, finance, and voice and image recognition to learn and classify complex datasets. The current medical literature shows both strengths and limitations of DL. Strengths of DL include its ability to automate medical image interpretation, enhance clinical decision-making, identify novel phenotypes, and select better treatment pathways in complex diseases. Deep learning may be well-suited to cardiovascular medicine in which haemodynamic and electrophysiological indices are increasingly captured on a continuous basis by wearable devices as well as image segmentation in cardiac imaging. However, DL also has significant weaknesses including difficulties in interpreting its models (the 'black-box' criticism), its need for extensive adjudicated ('labelled') data in training, lack of standardization in design, lack of data-efficiency in training, limited applicability to clinical trials, and other factors. Thus, the optimal clinical application of DL requires careful formulation of solvable problems, selection of most appropriate DL algorithms and data, and balanced interpretation of results. This review synthesizes the current state of DL for cardiovascular clinicians and investigators, and provides technical context to appreciate the promise, pitfalls, near-term challenges, and opportunities for this exciting new area.

[1]  Jane E. Marshall,et al.  Artificial Intelligence and Echocardiography: A Primer for Cardiac Sonographers , 2020, Journal of the American Society of Echocardiography.

[2]  G. Lip,et al.  Role for machine learning in sex-specific prediction of successful electrical cardioversion in atrial fibrillation? , 2020, Open Heart.

[3]  Giuseppe Jurman,et al.  Machine learning can predict survival of patients with heart failure from serum creatinine and ejection fraction alone , 2020, BMC Medical Informatics and Decision Making.

[4]  Chang-ming Huang,et al.  Application of preoperative artificial neural network based on blood biomarkers and clinicopathological parameters for predicting long-term survival of patients with gastric cancer , 2019, World journal of gastroenterology.

[5]  Claudio Napoli,et al.  Network Medicine: A Clinical Approach for Precision Medicine and Personalized Therapy in Coronary Heart Disease , 2019, Journal of atherosclerosis and thrombosis.

[6]  Andy Kitchen,et al.  Chest Radiographs in Congestive Heart Failure: Visualizing Neural Network Learning. , 2019, Radiology.

[7]  Masoumeh Haghpanahi,et al.  Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network , 2019, Nature Medicine.

[8]  D. Dey,et al.  Deep Learning for Prediction of Obstructive Disease From Fast Myocardial Perfusion SPECT: A Multicenter Study. , 2018, JACC. Cardiovascular imaging.

[9]  Daniel Rueckert,et al.  Deep learning cardiac motion analysis for human survival prediction , 2018, Nature Machine Intelligence.

[10]  Kipp W. Johnson,et al.  Big data, artificial intelligence, and cardiovascular precision medicine , 2018, Expert Review of Precision Medicine and Drug Development.

[11]  Susanna Price,et al.  2018 ESC Guidelines for the management of cardiovascular diseases during pregnancy. , 2018, European heart journal.

[12]  Jeroen J. Bax,et al.  Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging. , 2018, European heart journal.

[13]  S. Bangalore,et al.  Future Direction for Using Artificial Intelligence to Predict and Manage Hypertension , 2018, Current Hypertension Reports.

[14]  Henggui Zhang,et al.  VoxelAtlasGAN: 3D Left Ventricle Segmentation on Echocardiography with Atlas Guided Generation and Voxel-to-voxel Discrimination , 2018, MICCAI.

[15]  Kipp W. Johnson,et al.  Artificial Intelligence in Cardiology. , 2018, Journal of the American College of Cardiology.

[16]  B. Lindsay,et al.  Smartwatch Algorithm for Automated Detection of Atrial Fibrillation. , 2018, Journal of the American College of Cardiology.

[17]  Mike Voets,et al.  Deep Learning: From Data Extraction to Large-Scale Analysis , 2018 .

[18]  N. Sohoni,et al.  Passive Detection of Atrial Fibrillation Using a Commercially Available Smartwatch , 2018, JAMA cardiology.

[19]  W. Tong,et al.  Comparing SVM and ANN based Machine Learning Methods for Species Identification of Food Contaminating Beetles , 2018, Scientific Reports.

[20]  Henggui Zhang,et al.  Multi-Views Fusion CNN for Left Ventricular Volumes Estimation on Cardiac MR Images , 2018, IEEE Transactions on Biomedical Engineering.

[21]  Andrew Janowczyk,et al.  A deep-learning classifier identifies patients with clinical heart failure using whole-slide images of H&E tissue , 2018, PloS one.

[22]  Ramy Arnaout,et al.  Fast and accurate view classification of echocardiograms using deep learning , 2018, npj Digital Medicine.

[23]  Pablo Laguna,et al.  Distinct ECG Phenotypes Identified in Hypertrophic Cardiomyopathy Using Machine Learning Associate With Arrhythmic Risk Markers , 2018, Front. Physiol..

[24]  Konstantinos N. Plataniotis,et al.  Brain Tumor Type Classification via Capsule Networks , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[25]  Mattias Ohlsson,et al.  Improving prediction of heart transplantation outcome using deep learning techniques , 2018, Scientific Reports.

[26]  Brett K. Beaulieu-Jones,et al.  Characterizing and Managing Missing Structured Data in Electronic Health Records: Data Analysis , 2017, bioRxiv.

[27]  Newton Howard,et al.  Deep clustering of longitudinal data , 2018, ArXiv.

[28]  Jascha Sohl-Dickstein,et al.  Adversarial Examples that Fool both Computer Vision and Time-Limited Humans , 2018, NeurIPS.

[29]  Henggui Zhang,et al.  Detecting atrial fibrillation by deep convolutional neural networks , 2018, Comput. Biol. Medicine.

[30]  Pan He,et al.  Adversarial Examples: Attacks and Defenses for Deep Learning , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[32]  Max A. Viergever,et al.  Deep learning analysis of the myocardium in coronary CT angiography for identification of patients with functionally significant coronary artery stenosis , 2017, Medical Image Anal..

[33]  T. Pyrkov,et al.  Extracting biological age from biomedical data via deep learning: too much of a good thing? , 2017, bioRxiv.

[34]  D. Singer,et al.  Association of of Atrial Fibrillation Clinical Phenotypes With Treatment Patterns and Outcomes: A Multicenter Registry Study , 2017, JAMA cardiology.

[35]  Pieter Abbeel,et al.  Meta Learning Shared Hierarchies , 2017, ICLR.

[36]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[37]  Ben Glocker,et al.  Automated cardiovascular magnetic resonance image analysis with fully convolutional networks , 2017, Journal of Cardiovascular Magnetic Resonance.

[38]  C. Krittanawong Future Physicians in the Era of Precision Cardiovascular Medicine , 2017, Circulation.

[39]  Marcin Andrychowicz,et al.  Sim-to-Real Transfer of Robotic Control with Dynamics Randomization , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[40]  P. Abbeel,et al.  Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environments , 2017, ICLR.

[41]  B Pyakillya,et al.  Deep Learning for ECG Classification , 2017 .

[42]  C. Krittanawong,et al.  Identifying Genotypes and Phenotypes of Cardiovascular Diseases Using Big Data Analytics. , 2017, JAMA cardiology.

[43]  Tathagato Rai Dastidar,et al.  Analyzing Microscopic Images of Peripheral Blood Smear Using Deep Learning , 2017, DLMIA/ML-CDS@MICCAI.

[44]  Yujian Li,et al.  Deep neural mapping support vector machines , 2017, Neural Networks.

[45]  Gregory S. Corrado,et al.  Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning , 2017, ArXiv.

[46]  Randal S. Olson,et al.  Data-driven advice for applying machine learning to bioinformatics problems , 2017, PSB.

[47]  Jaime S. Cardoso,et al.  Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support , 2017, Lecture Notes in Computer Science.

[48]  Kumar Sricharan,et al.  Recognizing Abnormal Heart Sounds Using Deep Learning , 2017, KHD@IJCAI.

[49]  Wei Li,et al.  A fused deep learning architecture for viewpoint classification of echocardiography , 2017, Inf. Fusion.

[50]  C. Krittanawong,et al.  Artificial Intelligence in Precision Cardiovascular Medicine. , 2017, Journal of the American College of Cardiology.

[51]  Khalid Ashraf,et al.  Abnormality Detection and Localization in Chest X-Rays using Deep Convolutional Neural Networks , 2017, ArXiv.

[52]  Hao Chen,et al.  MagNet: A Two-Pronged Defense against Adversarial Examples , 2017, CCS.

[53]  Konstantinos Kamnitsas,et al.  Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation , 2017, IEEE Transactions on Medical Imaging.

[54]  Kevin Miller,et al.  Forward Thinking: Building Deep Random Forests , 2017, ArXiv.

[55]  Felix Lau,et al.  FastVentricle: Cardiac Segmentation with ENet , 2017, FIMH.

[56]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[57]  T. Hastie,et al.  Accuracy in Wrist-Worn, Sensor-Based Measurements of Heart Rate and Energy Expenditure in a Diverse Cohort , 2016, bioRxiv.

[58]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[59]  Purang Abolmaesumi,et al.  Abstract 17562: Automatic Quality Assessment of Echo Apical 4-chamber Images Using Computer Deep Learning , 2016 .

[60]  J. Karch A machine learning perspective on repeated measures , 2016 .

[61]  P. Kirchhof,et al.  2016 ESC Guidelines for the management of atrial fibrillation developed in collaboration with EACTS. , 2016, Europace : European pacing, arrhythmias, and cardiac electrophysiology : journal of the working groups on cardiac pacing, arrhythmias, and cardiac cellular electrophysiology of the European Society of Cardiology.

[62]  Konstantinos Kamnitsas,et al.  Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks , 2016, MICCAI.

[63]  P. Kirchhof,et al.  2016 ESC Guidelines for the management of atrial fibrillation developed in collaboration with EACTS. , 2016, European heart journal.

[64]  Jimeng Sun,et al.  Using recurrent neural network models for early detection of heart failure onset , 2016, J. Am. Medical Informatics Assoc..

[65]  Carlos Guestrin,et al.  Model-Agnostic Interpretability of Machine Learning , 2016, ArXiv.

[66]  Josef Urban,et al.  DeepMath - Deep Sequence Models for Premise Selection , 2016, NIPS.

[67]  Konrad P. Körding,et al.  Toward an Integration of Deep Learning and Neuroscience , 2016, bioRxiv.

[68]  M. Motwani,et al.  Machine learning for prediction of all-cause mortality in patients with suspected coronary artery disease: a 5-year multicentre prospective registry analysis , 2016, European heart journal.

[69]  Chen Wang,et al.  Standardized Evaluation System for Left Ventricular Segmentation Algorithms in 3D Echocardiography , 2016, IEEE Transactions on Medical Imaging.

[70]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[71]  Ananthram Swami,et al.  Practical Black-Box Attacks against Machine Learning , 2016, AsiaCCS.

[72]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[73]  Michael J Ackerman,et al.  Association of Arrhythmia-Related Genetic Variants With Phenotypes Documented in Electronic Medical Records. , 2016, JAMA.

[74]  Hamid Jafarkhani,et al.  A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI , 2015, Medical Image Anal..

[75]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[76]  Marcus Schreckenberg,et al.  Fully Automated Versus Standard Tracking of Left Ventricular Ejection Fraction and Longitudinal Strain: The FAST-EFs Multicenter Study. , 2015, Journal of the American College of Cardiology.

[77]  Marc G. Bellemare,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[78]  Rahul C. Deo,et al.  Phenomapping for Novel Classification of Heart Failure With Preserved Ejection Fraction , 2015, Circulation.

[79]  Gary S Collins,et al.  Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration , 2015, Annals of Internal Medicine.

[80]  Daniel Rueckert,et al.  Endocardial 3D Ultrasound Segmentation using Autocontext Random Forests , 2014, The MIDAS Journal.

[81]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[82]  Mary Beth Terry,et al.  Assessing the goodness of fit of personal risk models , 2014, Statistics in medicine.

[83]  M. Kattan,et al.  Calibration plots for risk prediction models in the presence of competing risks , 2014, Statistics in medicine.

[84]  Yvonne Vergouwe,et al.  Towards better clinical prediction models: seven steps for development and an ABCD for validation. , 2014, European heart journal.

[85]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[86]  Ofer Harel,et al.  Addressing Missing Data Mechanism Uncertainty using Multiple-Model Multiple Imputation: Application to a Longitudinal Clinical Trial. , 2012, The annals of applied statistics.

[87]  Mike Kirby,et al.  ESC Guidelines on the Management of Cardiovascular Diseases During Pregnancy , 2012 .

[88]  Helmut Baumgartner,et al.  ESC Guidelines for the management of grown-up congenital heart disease (new version 2010). , 2010, European heart journal.

[89]  R. Pallàs-Areny,et al.  Heart rate detection from single-foot plantar bioimpedance measurements in a weighing scale , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[90]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[91]  Remco C. Veltkamp,et al.  An Ensemble of Deep Support Vector Machines for Image Categorization , 2009, 2009 International Conference of Soft Computing and Pattern Recognition.

[92]  David J. Hand,et al.  Measuring classifier performance: a coherent alternative to the area under the ROC curve , 2009, Machine Learning.

[93]  Martin Jansche,et al.  Maximum Expected F-Measure Training of Logistic Regression Models , 2005, HLT.

[94]  Raymond J. Mooney,et al.  Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing , 2005 .

[95]  Guy Carrault,et al.  Atrial activity enhancement by Wiener filtering using an artificial neural network , 2001, IEEE Transactions on Biomedical Engineering.

[96]  J. Habbema,et al.  Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. , 2001, Journal of clinical epidemiology.

[97]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[98]  O. Pahlm,et al.  Agreement between artificial neural networks and experienced electrocardiographer on electrocardiographic diagnosis of healed myocardial infarction. , 1996, Journal of the American College of Cardiology.

[99]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[100]  W R Webber,et al.  Practical detection of epileptiform discharges (EDs) in the EEG using an artificial neural network: a comparison of raw and parameterized EEG data. , 1994, Electroencephalography and clinical neurophysiology.

[101]  J. Vos,et al.  Computer-simulated neural networks: an appropriate model for motor development? , 1993, Early human development.

[102]  Geoffrey E. Hinton A Parallel Computation that Assigns Canonical Object-Based Frames of Reference , 1981, IJCAI.

[103]  D. Rubin INFERENCE AND MISSING DATA , 1975 .

[104]  C. Krittanawong,et al.  Crowdfunding for cardiovascular research. , 2018, International journal of cardiology.

[105]  Qiao Li,et al.  A deep learning approach to monitoring and detecting atrial fibrillation using wearable technology , 2017, 2017 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI).

[106]  Haifeng Liu,et al.  Using Machine Learning Models to Predict In-Hospital Mortality for ST-Elevation Myocardial Infarction Patients , 2017, MedInfo.

[107]  W. Ouwerkerk A systems biology study to tailored treatment in chronic heart failure , 2017 .

[108]  P. Ponikowski,et al.  2016 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure , 2016 .

[109]  Survi Kyal,et al.  Detection of atrial fibrillation using contactless facial video monitoring. , 2015, Heart rhythm.

[110]  Danilo Samuel Jodas,et al.  Comparing Support Vector Machines and Artificial Neural Networks in the Recognition Of Steering Angle for Driving of Mobile Robots Through Paths in Plantations , 2013, ICCS.