An End-to-End Computer Vision Pipeline for Automated Cardiac Function Assessment by Echocardiography

Background: Automated cardiac image interpretation has the potential to transform clinical practice in multiple ways including enabling low-cost assessment of cardiac function in the primary care setting. We hypothesized that advances in computer vision could enable building a fully automated, scalable analysis pipeline for echocardiogram (echo) interpretation, with a focus on evaluation of left ventricular function. Methods: Our approach entailed: 1) preprocessing, which includes auto-downloading of echo studies, metadata extraction, de-identification, and conversion of images into numerical arrays; 2) convolutional neural networks (CNN) for view identification; 3) localization of the left ventricle and delineation of cardiac boundaries using active appearance models (AAM); 4) identification of properly segmented images using gradient boosting; and 5) particle tracking to compute longitudinal strain. Results: CNNs identified views with high accuracy (e.g. 95% for apical 4-chamber) and the combination of CNN/bounding box determination/AAM effectively segmented 67-88% of videos. We analyzed 2775 apical videos from patients with heart failure and found good concordance with vendor-derived longitudinal strain measurements, both at the individual video level (r= 0.77) and at the patient level (r= 0.52). We also analyzed 9402 videos from breast cancer patients undergoing serial monitoring for trastuzumab cardiotoxicity to illustrate the potential for automated, quality-weighted modeling of patient trajectories. Conclusions: We demonstrate the feasibility of a fully automated echocardiography analysis pipeline for assessment of left ventricular function. Our work lays the groundwork for using automated interpretation to support point-of-care handheld cardiac ultrasound and *To whom correspondence should be addressed: rahul.deo@ucsf.edu may enable large-scale analysis of the millions of echos currently archived within healthcare systems.