Simultaneous Right Ventricle End-diastolic and End-systolic Frame Identification and Landmark Detection on Echocardiography

End-diastolic (ED) and end-systolic (ES) frame identification and landmark detection are crucial steps of estimating right ventricle function in clinic practice. However, the complex morphology of the right ventricle and low-quality echocardiography pose challenges to these tasks. This study proposes a multi-task learning (MTL) framework to simultaneously identify the right ventricle ED and ES frames and detect anatomical landmarks for echocardiography. The framework contains an encoder and two branches: frame-branch and landmark-branch. The convolution neural network (CNN) encoder is employed for extracting the shared features of two branches. The frame-branch is built with a recurrent neural network (RNN) to select ED and ES frames. A heatmap-based model is used as the landmark-branch to detect the landmarks. Furthermore, instead of directly regressing the indexes of ED/ES frames, we form the frame identification as a curve regression problem, which achieves considerable performance. Experiments performed on the echocardiography dataset of 105 patients validate the effectiveness of the proposed approach, which leads to the average frame difference of 1.59 (±1.34) frames (ED) and 1.56 (±1.35) frames (ES) on the frame identification task, and the percentage of correctly predicted landmarks is 83.3%. These results demonstrated that our method outperforms most existing methods.

[1]  Ben Glocker,et al.  Automatic Localization and Identification of Vertebrae in Arbitrary Field-of-View CT Scans , 2012, MICCAI.

[2]  Chengwen Chu,et al.  Evaluation and Comparison of Anatomical Landmark Detection Methods for Cephalometric X-Ray Images: A Grand Challenge , 2015, IEEE Transactions on Medical Imaging.

[3]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  F. Sheehan,et al.  Accuracy of knowledge-based reconstruction for measurement of right ventricular volume and function in patients with tetralogy of Fallot. , 2010, The American journal of cardiology.

[7]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[8]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Amit R. Patel,et al.  Three-dimensional modeling of the right ventricle from two-dimensional transthoracic echocardiographic images: utility of knowledge-based reconstruction in pulmonary arterial hypertension. , 2013, Journal of the American Society of Echocardiography : official publication of the American Society of Echocardiography.

[10]  Wufeng Xue,et al.  Full left ventricle quantification via deep multitask relationships learning , 2018, Medical Image Anal..

[11]  Horst Bischof,et al.  Regressing Heatmaps for Multiple Landmark Localization Using CNNs , 2016, MICCAI.

[12]  Jie Li,et al.  An Attention-Guided Deep Regression Model for Landmark Detection in Cephalograms , 2019, MICCAI.

[13]  Victor Mor-Avi,et al.  Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. , 2015, European heart journal cardiovascular Imaging.

[14]  Andrew Zisserman,et al.  Flowing ConvNets for Human Pose Estimation in Videos , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Shaoting Zhang,et al.  Recognizing End-Diastole and End-Systole Frames via Deep Temporal Regression Network , 2016, MICCAI.

[16]  Shaohua Kevin Zhou,et al.  Less is More: Simultaneous View Classification and Landmark Detection for Abdominal Ultrasound Images , 2018, MICCAI.

[17]  N. Kachenoura,et al.  Automatic Detection of End Systole within a Sequence of Left Ventricular Echocardiographic Images using Autocorrelation and Mitral Valve Motion Detection , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  Bao-Gang Hu,et al.  Learning with Average Top-k Loss , 2017, NIPS.

[19]  Purang Abolmaesumi,et al.  Cardiac Phase Detection in Echocardiograms With Densely Gated Recurrent Neural Networks and Global Extrema Loss , 2019, IEEE Transactions on Medical Imaging.

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.