Region Proposal Network with Graph Prior and Iou-Balance Loss for Landmark Detection in 3D Ultrasound

3D ultrasound (US) can facilitate detailed prenatal examinations for fetal growth monitoring. To analyze a 3D US volume, it is fundamental to identify anatomical landmarks of the evaluated organs accurately. Typical deep learning methods usually regress the coordinates directly or involve heatmap-matching. However, these methods struggle to deal with volumes with large sizes and the highly-varying positions and orientations of fetuses. In this work, we exploit an object detection framework to detect landmarks in 3D fetal facial US volumes. By regressing multiple parameters of the landmark-centered bounding box (B-box) with a strict criteria, the proposed model is able to pinpoint the exact location of the targeted landmarks. Specifically, the model uses a 3D region proposal network (RPN) to generate 3D candidate regions, followed by several 3D classification branches to select the best candidate. It also adopts an IoU-balance loss to improve communications between branches that benefit the learning process. Furthermore, it leverage a distance-based graph prior to regularize the training and helps to reduce false positive predictions. The performance of the proposed framework is evaluated on a 3D US dataset to detect five key fetal facial landmarks. Results showed the proposed method outperforms some of the state-of-the-art methods in efficacy and efficiency.

[1]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[2]  Fausto Milletari,et al.  Fully Convolutional Regression Network for Accurate Detection of Measurement Points , 2017, DLMIA/ML-CDS@MICCAI.

[3]  J. Alison Noble,et al.  VP‐Nets : Efficient automatic localization of key brain structures in 3D fetal neurosonography , 2018, Medical Image Anal..

[4]  Xiangzhi Bai,et al.  Efficient Multiple Organ Localization in CT Image Using 3D Region Proposal Network , 2019, IEEE Transactions on Medical Imaging.

[5]  Dong Ni,et al.  Deep Learning in Medical Ultrasound Analysis: A Review , 2019, Engineering.

[6]  Xiaoping Li,et al.  IoU-balanced Loss Functions for Single-stage Object Detection , 2019, Pattern Recognit. Lett..

[7]  Xin Yang,et al.  Joint Segmentation and Landmark Localization of Fetal Femur in Ultrasound Volumes , 2019, 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI).

[8]  Horst Bischof,et al.  Regressing Heatmaps for Multiple Landmark Localization Using CNNs , 2016, MICCAI.

[9]  Dong Ni,et al.  Multi-task learning for quality assessment of fetal head ultrasound images , 2019, Medical Image Anal..

[10]  Jeremy Tan,et al.  Deep Adversarial Context-Aware Landmark Detection for Ultrasound Imaging , 2018, MICCAI.

[11]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.