Explainable Deep Learning Improves Physician Interpretation of Myocardial Perfusion Imaging

Visual Abstract Artificial intelligence may improve accuracy of myocardial perfusion imaging (MPI) but will likely be implemented as an aid to physician interpretation rather than an autonomous tool. Deep learning (DL) has high standalone diagnostic accuracy for obstructive coronary artery disease (CAD), but its influence on physician interpretation is unknown. We assessed whether access to explainable DL predictions improves physician interpretation of MPI. Methods: We selected a representative cohort of patients who underwent MPI with reference invasive coronary angiography. Obstructive CAD, defined as stenosis ≥50% in the left main artery or ≥70% in other coronary segments, was present in half of the patients. We used an explainable DL model (CAD-DL), which was previously developed in a separate population from different sites. Three physicians interpreted studies first with clinical history, stress, and quantitative perfusion, then with all the data plus the DL results. Diagnostic accuracy was assessed using area under the receiver-operating-characteristic curve (AUC). Results: In total, 240 patients with a median age of 65 y (interquartile range 58–73) were included. The diagnostic accuracy of physician interpretation with CAD-DL (AUC 0.779) was significantly higher than that of physician interpretation without CAD-DL (AUC 0.747, P = 0.003) and stress total perfusion deficit (AUC 0.718, P < 0.001). With matched specificity, CAD-DL had higher sensitivity when operating autonomously compared with readers without DL results (P < 0.001), but not compared with readers interpreting with DL results (P = 0.122). All readers had numerically higher accuracy with CAD-DL, with AUC improvement 0.02–0.05, and interpretation with DL resulted in overall net reclassification improvement of 17.2% (95% CI 9.2%–24.4%, P < 0.001). Conclusion: Explainable DL predictions lead to meaningful improvements in physician interpretation; however, the improvement varied across the readers, reflecting the acceptance of this new technology. This technique could be implemented as an aid to physician diagnosis, improving the diagnostic accuracy of MPI.

[1]  Abhinav K. Jha,et al.  Nuclear Medicine and Artificial Intelligence: Best Practices for Algorithm Development , 2021, The Journal of Nuclear Medicine.

[2]  T. Kudo,et al.  Usefulness of an artificial neural network for a beginner to achieve similar interpretations to an expert when examining myocardial perfusion images , 2020, The International Journal of Cardiovascular Imaging.

[3]  D. Dey,et al.  Impact of Early Revascularization on Major Adverse Cardiovascular Events in Relation to Automatically Quantified Ischemia. , 2020, JACC. Cardiovascular imaging.

[4]  D. Dey,et al.  Prognostically safe stress-only single-photon emission computed tomography myocardial perfusion imaging guided by machine learning: report from REFINE SPECT. , 2020, European heart journal cardiovascular Imaging.

[5]  S. Gerke,et al.  Ethical and legal challenges of artificial intelligence-driven healthcare , 2020, Artificial Intelligence in Healthcare.

[6]  T. Engel Diagnosis of Hypertrophic Cardiomyopathy: Who Is in Charge Here-The Physician or the Computer? , 2020, Journal of the American College of Cardiology.

[7]  Damini Dey,et al.  5-Year Prognostic Value of Quantitative Versus Visual MPI in Subtle Perfusion Defects: Results From REFINE SPECT. , 2020, JACC. Cardiovascular imaging.

[8]  Damini Dey,et al.  Machine learning predicts per-vessel early coronary revascularization after fast myocardial perfusion SPECT: results from multicentre REFINE SPECT registry. , 2019, European heart journal cardiovascular Imaging.

[9]  Nassir Navab,et al.  Classification of Polar Maps from Cardiac Perfusion Imaging with Graph-Convolutional Neural Networks , 2019, Scientific Reports.

[10]  Jaron J. R. Chong,et al.  Canadian Association of Radiologists White Paper on Ethical and Legal Issues Related to Artificial Intelligence in Radiology , 2019, Canadian Association of Radiologists journal = Journal l'Association canadienne des radiologistes.

[11]  Kipp W. Johnson,et al.  Deep learning for cardiovascular medicine: a practical primer. , 2019, European heart journal.

[12]  D. Dey,et al.  Deep Learning for Prediction of Obstructive Disease From Fast Myocardial Perfusion SPECT: A Multicenter Study. , 2018, JACC. Cardiovascular imaging.

[13]  Piotr J. Slomka,et al.  Deep Learning Analysis of Upright-Supine High-Efficiency SPECT Myocardial Perfusion Imaging for Prediction of Obstructive Coronary Artery Disease: A Multicenter Study , 2018, The Journal of Nuclear Medicine.

[14]  F. Sardanelli,et al.  Artificial intelligence as a medical device in radiology: ethical and regulatory issues in Europe and the United States , 2018, Insights into Imaging.

[15]  Damini Dey,et al.  Prognostic Value of Combined Clinical and Myocardial Perfusion Imaging Data Using Machine Learning. , 2017, JACC. Cardiovascular imaging.

[16]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  D. Berman,et al.  Two-position supine/prone myocardial perfusion SPECT (MPS) imaging improves visual inter-observer correlation and agreement , 2014, Journal of Nuclear Cardiology.

[18]  Damini Dey,et al.  Improved Accuracy of Myocardial Perfusion SPECT for the Detection of Coronary Artery Disease Using a Support Vector Machine Algorithm , 2013, The Journal of Nuclear Medicine.

[19]  Guido Germano,et al.  Comparison of Fully Automated Computer Analysis and Visual Scoring for Detection of Coronary Artery Disease from Myocardial Perfusion SPECT in a Large Population , 2013, The Journal of Nuclear Medicine.

[20]  D. Berman,et al.  Direct Quantification of Left Ventricular Motion and Thickening Changes Using Rest–Stress Myocardial Perfusion SPECT , 2012, The Journal of Nuclear Medicine.

[21]  D. Berman,et al.  Quantitative Upright–Supine High-Speed SPECT Myocardial Perfusion Imaging for Detection of Coronary Artery Disease: Correlation with Invasive Coronary Angiography , 2010, The Journal of Nuclear Medicine.

[22]  R. F. Wagner,et al.  A Framework for Random-Effects ROC Analysis: Biases with the Bootstrap and Other Variance Estimators , 2009 .

[23]  Ian A Scott,et al.  Errors in clinical reasoning: causes and remedial strategies , 2009, BMJ : British Medical Journal.

[24]  Eric Clarkson,et al.  A probabilistic model for the MRMC method, part 1: theoretical development. , 2006, Academic radiology.

[25]  D. Berman,et al.  Automated quantification of myocardial perfusion SPECT using simplified normal limits , 2004, Journal of nuclear cardiology : official publication of the American Society of Nuclear Cardiology.

[26]  D. Berman,et al.  Comparison of the Short‐Term Survival Benefit Associated With Revascularization Compared With Medical Therapy in Patients With No Prior Coronary Artery Disease Undergoing Stress Myocardial Perfusion Single Photon Emission Computed Tomography , 2003, Circulation.

[27]  D. Berman,et al.  Value of Stress Myocardial Perfusion Single Photon Emission Computed Tomography in Patients With Normal Resting Electrocardiograms: An Evaluation of Incremental Prognostic Value and Cost-Effectiveness , 2002, Circulation.

[28]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.