A Generic Quality Control Framework for Fetal Ultrasound Cardiac Four-Chamber Planes

Quality control/assessment of ultrasound (US) images is an essential step in clinical diagnosis. This process is usually done manually, suffering from some drawbacks, such as dependence on operator's experience and extensive labors, as well as high inter- and intra-observer variation. Automatic quality assessment of US images is therefore highly desirable. Fetal US cardiac four-chamber plane (CFP) is one of the most commonly used cardiac views, which was used in the diagnosis of heart anomalies in the early 1980s. In this paper, we propose a generic deep learning framework for automatic quality control of fetal US CFPs. The proposed framework consists of three networks: (1) a basic CNN (B-CNN), roughly classifying four-chamber views from the raw data; (2) a deeper CNN (D-CNN), determining the gain and zoom of the target images in a multi-task learning manner; and (3) the aggregated residual visual block net (ARVBNet), detecting the key anatomical structures on a plane. Based on the output of the three networks, overall quantitative score of each CFP is obtained, so as to achieve fully automatic quality control. Experiments on a fetal US dataset demonstrated our proposed method achieved a highest mean average precision (mAP) of 93.52% at a fast speed of 101 frames per second (FPS). In order to demonstrate the adaptability and generalization capacity, the proposed detection network (i.e., ARVBNet) has also been validated on the PASCAL VOC dataset, obtaining a highest mAP of 81.2% when input size is approximately 300 × 300.

[1]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Rabih Chaoui,et al.  A Review of Findings in Fetal Cardiac Section Drawings , 2007, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[3]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[4]  D J Sahn,et al.  Qualitative Real‐time Cross‐sectional Echocardiographic Imaging of the Human Fetus During the Second Half of Pregnancy , 1980, Circulation.

[5]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[6]  Y. Ville,et al.  Feasibility and reproducibility of an image‐scoring method for quality control of fetal biometry in the second trimester , 2005, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[7]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[8]  Max A. Viergever,et al.  Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities , 2016, MICCAI.

[9]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Pheng-Ann Heng,et al.  Ultrasound Standard Plane Detection Using a Composite Neural Network Framework , 2017, IEEE Transactions on Cybernetics.

[11]  Konstantinos Kamnitsas,et al.  SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound , 2016, IEEE Transactions on Medical Imaging.

[12]  Yunhong Wang,et al.  Receptive Field Block Net for Accurate and Fast Object Detection , 2017, ECCV.

[13]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Fuqiang Zhou,et al.  FSSD: Feature Fusion Single Shot Multibox Detector , 2017, ArXiv.

[15]  S. Yagel,et al.  Examination of the fetal heart by five short‐axis views: a proposed screening method for comprehensive cardiac evaluation , 2001, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[16]  Dong Ni,et al.  Deep Learning in Medical Ultrasound Analysis: A Review , 2019, Engineering.

[17]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[18]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[19]  Purang Abolmaesumi,et al.  Automatic Quality Assessment of Echocardiograms Using Convolutional Neural Networks: Feasibility on the Apical Four-Chamber View , 2017, IEEE Transactions on Medical Imaging.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Kenji Suzuki,et al.  Overview of deep learning in medical imaging , 2017, Radiological Physics and Technology.

[22]  Purang Abolmaesumi,et al.  Quality Assessment of Echocardiographic Cine Using Recurrent Neural Networks: Feasibility on Five Standard View Planes , 2017, MICCAI.

[23]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[24]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[25]  J. Alison Noble,et al.  A framework for analysis of linear ultrasound videos to detect fetal presentation and heartbeat , 2017, Medical Image Anal..

[26]  J. Alison Noble,et al.  Quality control of fetal ultrasound images: Detection of abdomen anatomical landmarks using AdaBoost , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[27]  Nojun Kwak,et al.  Enhancement of SSD by concatenating feature maps for object detection , 2017, BMVC.

[28]  Dong Ni,et al.  FUIQA: Fetal Ultrasound Image Quality Assessment With Deep Convolutional Networks , 2017, IEEE Transactions on Cybernetics.

[29]  Y. Ville,et al.  A score‐based method for quality control of fetal images at routine second‐trimester ultrasound examination , 2008, Prenatal diagnosis.

[30]  Zhiqiang Shen,et al.  DSOD: Learning Deeply Supervised Object Detectors from Scratch , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Stéphane Mallat,et al.  Rigid-Motion Scattering for Texture Classification , 2014, ArXiv.

[32]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[33]  L. Nappi,et al.  Four chamber view plus three-vessel and trachea view for a complete evaluation of the fetal heart during the second trimester , 2006, Journal of perinatal medicine.

[34]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[35]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Peiyun Hu,et al.  Finding Tiny Faces , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Jonathan Winawer,et al.  Computational neuroimaging and population receptive fields , 2015, Trends in Cognitive Sciences.

[39]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[40]  N. Dudley,et al.  The importance of quality management in fetal measurement , 2002, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[41]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[42]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Hao Chen,et al.  Automatic Fetal Ultrasound Standard Plane Detection Using Knowledge Transferred Recurrent Neural Networks , 2015, MICCAI.

[44]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Lubomir M. Hadjiiski,et al.  Multi-task transfer learning deep convolutional neural network: application to computer-aided diagnosis of breast cancer on mammograms , 2017, Physics in medicine and biology.

[46]  Geoffrey Zweig,et al.  Recent advances in deep learning for speech research at Microsoft , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[47]  Yi Li,et al.  Deformable Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[48]  Abhinav Gupta,et al.  A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[50]  Y. Ville,et al.  Quality control of prenatal ultrasound , 2005 .

[51]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.