A Computer Vision Pipeline for Automated Determination of Cardiac Structure and Function and Detection of Disease by Two-Dimensional Echocardiography

Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways including enabling low-cost serial assessment of cardiac function in the primary care and rural setting. We hypothesized that advances in computer vision could enable building a fully automated, scalable analysis pipeline for echocardiogram (echo) interpretation. Our approach entailed: 1) preprocessing; 2) convolutional neural networks (CNN) for view identification, image segmentation, and phasing of the cardiac cycle; 3) quantification of chamber volumes and left ventricular mass; 4) particle tracking to compute longitudinal strain; and 5) targeted disease detection. CNNs accurately identified views (e.g. 99% for apical 4-chamber) and segmented individual cardiac chambers. Cardiac structure measurements agreed with study report values (e.g. mean absolute deviations (MAD) of 7.7 mL/kg/m2 for left ventricular diastolic volume index, 2918 studies). We computed automated ejection fraction and longitudinal strain measurements (within 2 cohorts), which agreed with commercial software-derived values [for ejection fraction, MAD=5.3%, N=3101 studies; for strain, MAD=1.5% (n=197) and 1.6% (n=110)], and demonstrated applicability to serial monitoring of breast cancer patients for trastuzumab cardiotoxicity. Overall, we found that, compared to manual measurements, automated measurements had superior performance across seven internal consistency metrics with an average increase in the Spearman correlation coefficient of 0.05 (p=0.02). Finally, we developed disease detection algorithms for hypertrophic cardiomyopathy and cardiac amyloidosis, with C-statistics of 0.93 and 0.84, respectively. Our pipeline lays the groundwork for using automated interpretation to support point-of-care handheld cardiac ultrasound and large-scale analysis of the millions of echos archived within healthcare systems.

[1]  J. Seidman,et al.  The Genetic Basis for Cardiomyopathy from Mutation Identification to Mechanistic Paradigms , 2001, Cell.

[2]  Peter J. Hunter,et al.  ICMA: an integrated cardiac modeling and analysis platform , 2014, Bioinform..

[3]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  L Wigström,et al.  Regional mean systolic myocardial velocity estimation by real-time color Doppler myocardial imaging: a new technique for quantifying regional systolic function. , 1998, Journal of the American Society of Echocardiography : official publication of the American Society of Echocardiography.

[6]  Ruzena Bajcsy,et al.  Multiresolution elastic matching , 1989, Comput. Vis. Graph. Image Process..

[7]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[8]  Benjamin French,et al.  Detailed Echocardiographic Phenotyping in Breast Cancer Patients , 2017 .

[9]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[10]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[11]  Tomoko Ishizu,et al.  Left Ventricular Strain and Transmural Distribution of Structural Remodeling in Hypertensive Heart Disease , 2014, Hypertension.

[12]  E. Braunwald.  The war against heart failure: the Lancet lecture , 2015, The Lancet.

[13]  G. Pedrizzetti,et al.  Definitions for a common standard for 2D speckle tracking echocardiography: consensus document of the EACVI/ASE/Industry Task Force to standardize deformation imaging. , 2015, Journal of the American Society of Echocardiography : official publication of the American Society of Echocardiography.

[14]  E. Olson,et al.  Cardiac plasticity. , 2008, The New England journal of medicine.

[15]  Giuseppe Limongelli,et al.  A novel clinical risk prediction model for sudden cardiac death in hypertrophic cardiomyopathy (HCM risk-SCD). , 2014, European heart journal.

[16]  R. Falk,et al.  Cardiac Amyloidosis: A Treatable Disease, Often Overlooked , 2011, Circulation.

[17]  H. Gilbert Welch,et al.  Medicare Services Provided by Cardiologists in the United States: 1999–2008 , 2012, Circulation. Cardiovascular quality and outcomes.

[18]  Victor Mor-Avi,et al.  Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. , 2015, European heart journal cardiovascular Imaging.

[19]  Milan Sonka,et al.  Automatic segmentation of echocardiographic sequences by active appearance motion models , 2002, IEEE Transactions on Medical Imaging.

[20]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[21]  Wei Li,et al.  A fused deep learning architecture for viewpoint classification of echocardiography , 2017, Inf. Fusion.

[22]  Dan Adam,et al.  Assessment of myocardial regional strain and strain rate by tissue tracking in B-mode echocardiograms. , 2006, Ultrasound in medicine & biology.

[23]  J. Dudley,et al.  Machine-Learning Algorithms to Automate Morphological and Functional Assessments in 2D Echocardiography. , 2016, Journal of the American College of Cardiology.

[24]  Subhashini Venugopalan,et al.  Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. , 2016, JAMA.

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.