Automatic Detection of Cardiac Chambers Using an Attention-based YOLOv4 Framework from Four-chamber View of Fetal Echocardiography

Echocardiography is a powerful prenatal examination tool for early diagnosis of fetal congenital heart diseases (CHDs). The four-chamber (FC) view is a crucial and easily accessible ultrasound (US) image among echocardiography images. Automatic analysis of FC views contributes significantly to the early diagnosis of CHDs. The first step to automatically analyze fetal FC views is locating the fetal four crucial chambers of heart in a US image. However, it is a greatly challenging task due to several key factors, such as numerous speckles in US images, the fetal cardiac chambers with small size and unfixed positions, and category indistinction caused by the similarity of cardiac chambers. These factors hinder the process of capturing robust and discriminative features, hence destroying fetal cardiac anatomical chambers precise localization. Therefore, we first propose a multistage residual hybrid attention module (MRHAM) to improve the feature learning. Then, we present an improved YOLOv4 detection model, namely MRHAM-YOLOv4-Slim. Specially, the residual identity mapping is replaced with the MRHAM in the backbone of MRHAM-YOLOv4-Slim, accurately locating the four important chambers in fetal FC views. Extensive experiments demonstrate that our proposed method outperforms current state-of-the-art, including the precision of 0.890, the recall of 0.920, the F1 score of 0.907 and the mAP of 0.913.

[1]  Vaanathi Sundaresan,et al.  Automated characterization of the fetal heart in ultrasound images using fully convolutional neural networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[2]  J. Alison Noble,et al.  A framework for analysis of linear ultrasound videos to detect fetal presentation and heartbeat , 2017, Medical Image Anal..

[3]  Konstantinos Kamnitsas,et al.  Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation , 2017, IEEE Transactions on Medical Imaging.

[4]  Charturong Tantibundhit,et al.  Automated embolic signal detection using Deep Convolutional Neural Network , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[5]  Konstantinos Kamnitsas,et al.  SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound , 2016, IEEE Transactions on Medical Imaging.

[6]  Hiroshi Fujita,et al.  Automated detection of masses on whole breast volume ultrasound scanner: false positive reduction using deep convolutional neural network , 2017, Medical Imaging.

[7]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[9]  Dorin Comaniciu,et al.  Four-Chamber Heart Modeling and Automatic Segmentation for 3-D Cardiac CT Volumes Using Marginal Space Learning and Steerable Features , 2008, IEEE Transactions on Medical Imaging.

[10]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Qi Zhang,et al.  Context-Aware Attention Network for Image-Text Retrieval , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Shu Liu,et al.  Path Aggregation Network for Instance Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[13]  Geoffrey E. Hinton,et al.  Learning to combine foveal glimpses with a third-order Boltzmann machine , 2010, NIPS.

[14]  D. Shen,et al.  Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans , 2016, Scientific Reports.

[15]  Yu-Wing Tai,et al.  Few-Shot Object Detection With Attention-RPN and Multi-Relation Detector , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[17]  Jianhua Guo,et al.  DW-Net: A cascaded convolutional neural network for apical four-chamber view segmentation in fetal echocardiography , 2019, Comput. Medical Imaging Graph..

[18]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[19]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Arijit Patra,et al.  Multi-anatomy localization in fetal echocardiography videos , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[21]  Xiao Liu,et al.  Stacked deep polynomial network based representation learning for tumor classification with small ultrasound image dataset , 2016, Neurocomputing.

[22]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[23]  Kai Chen,et al.  Prime Sample Attention in Object Detection , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Xudong Jiang,et al.  Accurate Cervical Cell Segmentation from Overlapping Clumps in Pap Smear Images , 2017, IEEE Trans. Medical Imaging.

[25]  J. Alison Noble,et al.  Automated annotation and quantitative description of ultrasound videos of the fetal heart , 2017, Medical Image Anal..

[26]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[27]  Hao Chen,et al.  Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks , 2015, IEEE Journal of Biomedical and Health Informatics.

[28]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Shaogang Gong,et al.  Image Search With Text Feedback by Visiolinguistic Attention Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[34]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[35]  Zhaohui Zheng,et al.  Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression , 2019, AAAI.

[36]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[37]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[38]  Suresh Manandhar,et al.  Focusnet: An Attention-Based Fully Convolutional Network for Medical Image Segmentation , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[39]  Frederic Cervenansky,et al.  Deep Learning for Segmentation Using an Open Large-Scale Dataset in 2D Echocardiography , 2019, IEEE Transactions on Medical Imaging.

[40]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Tianfu Wang,et al.  A Generic Quality Control Framework for Fetal Ultrasound Cardiac Four-Chamber Planes , 2020, IEEE Journal of Biomedical and Health Informatics.

[42]  Gustavo Carneiro,et al.  The Segmentation of the Left Ventricle of the Heart From Ultrasound Data Using Deep Learning Architectures and Derivative-Based Search Methods , 2012, IEEE Transactions on Image Processing.

[43]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.