Ejection Fraction Classification in Transthoracic Echocardiography Using a Deep Learning Approach

Cardiovascular diseases are the leading cause of death worldwide. These diseases are related with a broad range of factors but usually show high correlation with diminished left ventricle function, which can be evaluated by measuring the ventricular ejection fraction through transthoracic echocardiography (TTE), a cost-effective and highly portable first-line diagnosing technique. Ejection fraction (EF) is currently determined through a semi-automatic process that requires manual delineation of the left ventricle area both in a diastolic and systolic frame of the patient's exam. To remove this manual annotation step, which is both time-consuming and user dependent, automatic Computer-Aided Diagnosis (CAD) systems can be used. Herein, we propose the first steps for such a system that classifies ejection fraction in four classes, based on TTE exams, with the objective of automatically providing valuable information to physicians. Our classification method is based on a 3D-Convolutional Neural Network (3D-CNN) trained on a dataset constructed with exams from a cardiology reference center. The dataset creation consisted of three main steps: firstly, for each exam, cine-loops showing the apical 4 chambers view were manually selected; then, 30 sequential frames were extracted from each cine-loop; finally, each frame was pre-processed to mask burned-in metadata. The neural network was designed to explore concepts such as convolutions using asymmetric filters and residual learning blocks. The model was trained on a dataset with 4000 TTE exams and tested on a separate dataset containing 1600 TTE cases. We obtained an accuracy of 78% and a F1 score of 71.3% for unhealthy EF (below 45%), 63.3% for intermediate EF (45-55%), 72.3% for healthy EF (55-75%) and 54.6% for abnormally high EF (above 75%). These results are promising and show that convolutional neural networks can be applied to this domain. Furthermore, this work will serve as a foundation for future research where other relevant cardiac metrics will be determined.

[1]  B. C. Loh,et al.  Deep learning for cardiac computer-aided diagnosis: benefits, issues & solutions. , 2017, mHealth.

[2]  Maxime Cannesson,et al.  A novel two-dimensional echocardiographic image analysis system using artificial intelligence-learned pattern recognition for rapid automated ejection fraction. , 2007, Journal of the American College of Cardiology.

[3]  N.V. Kiruthika,et al.  Automated Assessment of Aortic Regurgitation using 2D Doppler Echocardiogram , 2006, Proceedings of the 2006 IEEE International Workshop on Imagining Systems and Techniques (IST 2006).

[4]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Suet-Peng Yong,et al.  An Evaluation of Convolutional Neural Nets for Medical Image Anatomy Classification , 2016 .

[6]  Wei Li,et al.  A fused deep learning architecture for viewpoint classification of echocardiography , 2017, Inf. Fusion.

[7]  Charu C. Aggarwal,et al.  Neural Networks and Deep Learning , 2018, Springer International Publishing.

[8]  A. Lahiri,et al.  Radionuclide left ventricular ejection fraction: a comparison of three methods. , 1987, British heart journal.

[9]  C. Lamberti,et al.  Maximum likelihood segmentation of ultrasound images with Rayleigh distribution , 2005, IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control.

[10]  Gustavo Carneiro,et al.  Combining Multiple Dynamic Models and Deep Learning Architectures for Tracking the Left Ventricle Endocardium in Ultrasound Data , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  J. Cohn,et al.  Echocardiographic Variables as Prognostic Indicators and Therapeutic Monitors in Chronic Congestive Heart Failure Veterans Affairs Cooperative Studies V‐HeFT I and II , 1993, Circulation.

[12]  Luis Cano,et al.  Automatic Lymphocyte Detection on Gastric Cancer IHC Images Using Deep Learning , 2017, 2017 IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS).

[13]  史飛碩 Cardiovascular diseases (CVDs) patients’ in-hospital mortality rate and length of hospital stay in Swaziland: 2001-2013 , 2015 .

[14]  Alexei A. Efros,et al.  A Web-Deployed Computer Vision Pipeline for Automated Determination of Cardiac Structure and Function and Detection of Disease by Two-Dimensional Echocardiography , 2017 .

[15]  Henggui Zhang,et al.  A novel left ventricular volumes prediction method based on deep learning network in cardiac MRI , 2016, 2016 Computing in Cardiology Conference (CinC).

[16]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[17]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[18]  Gustavo Carneiro,et al.  The Segmentation of the Left Ventricle of the Heart From Ultrasound Data Using Deep Learning Architectures and Derivative-Based Search Methods , 2012, IEEE Transactions on Image Processing.

[19]  Dorin Comaniciu,et al.  Database-guided segmentation of anatomical structures with complex appearance , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Henggui Zhang,et al.  A combined multi-scale deep learning and random forests approach for direct left ventricular volumes estimation in 3D echocardiography , 2016, 2016 Computing in Cardiology Conference (CinC).

[21]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[22]  Milan Sonka,et al.  Automatic segmentation of echocardiographic sequences by active appearance motion models , 2002, IEEE Transactions on Medical Imaging.

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Alexei A. Efros,et al.  A Computer Vision Pipeline for Automated Determination of Cardiac Structure and Function and Detection of Disease by Two-Dimensional Echocardiography , 2017 .

[25]  Miguel Tavares Coimbra,et al.  Tracking large Anterior Mitral Leaflet displacements by incorporating optical flow in an active contours framework , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[26]  William D Freeman,et al.  Emerging role of cardiovascular CT and MRI in the evaluation of stroke. , 2015, AJR. American journal of roentgenology.

[27]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[28]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[29]  Alan M. Braga,et al.  Level Set Based on Brain Radiological Densities for Stroke Segmentation in CT Images , 2017, 2017 IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS).

[30]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[31]  Yuanyuan Wang,et al.  Automatic endocardium extraction for echocardiogram , 2011, 2011 4th International Conference on Biomedical Engineering and Informatics (BMEI).

[32]  Henggui Zhang,et al.  A left ventricular segmentation method on 3D echocardiography using deep learning and snake , 2016, 2016 Computing in Cardiology Conference (CinC).

[33]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[34]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[35]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[36]  Marcus Schreckenberg,et al.  Fully Automated Versus Standard Tracking of Left Ventricular Ejection Fraction and Longitudinal Strain: The FAST-EFs Multicenter Study. , 2015, Journal of the American College of Cardiology.

[37]  Qiang Chen,et al.  Network In Network , 2013, ICLR.