Model-Agnostic Method for Thoracic Wall Segmentation in Fetal Ultrasound Videos

The application of segmentation methods to medical imaging has the potential to create novel diagnostic support models. With respect to fetal ultrasound, the thoracic wall is a key structure on the assessment of the chest region for examiners to recognize the relative orientation and size of structures inside the thorax, which are critical components in neonatal prognosis. In this study, to improve the segmentation performance of the thoracic wall in fetal ultrasound videos, we proposed a novel model-agnostic method using deep learning techniques: the Multi-Frame + Cylinder method (MFCY). The Multi-frame method (MF) uses time-series information of ultrasound videos, and the Cylinder method (CY) utilizes the shape of the thoracic wall. To evaluate the achieved improvement, we performed segmentation using five-fold cross-validation on 538 ultrasound frames in the four-chamber view (4CV) of 256 normal cases using U-net and DeepLabv3+. MFCY increased the mean values of the intersection over union (IoU) of thoracic wall segmentation from 0.448 to 0.493 for U-net and from 0.417 to 0.470 for DeepLabv3+. These results demonstrated that MFCY improved the segmentation performance of the thoracic wall in fetal ultrasound videos without altering the network structure. MFCY is expected to facilitate the development of diagnostic support models in fetal ultrasound by providing further accurate segmentation of the thoracic wall.

[1]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Bart Bijnens,et al.  Machine Learning in Fetal Cardiology: What to Expect , 2020, Fetal Diagnosis and Therapy.

[3]  G. Satomi.  Guidelines for fetal echocardiography , 2015, Pediatrics international : official journal of the Japan Pediatric Society.

[4]  Satoshi Takahashi,et al.  Application of Artificial Intelligence Technology in Oncology: Towards the Establishment of Precision Medicine , 2020, Cancers.

[5]  Syed Muhammad Anwar,et al.  Medical Image Analysis using Convolutional Neural Networks: A Review , 2017, Journal of Medical Systems.

[6]  Xavier P. Burgos-Artizzu,et al.  Evaluation of an improved tool for non-invasive prediction of neonatal respiratory morbidity based on fully automated fetal lung ultrasound analysis , 2019, Scientific Reports.

[7]  Jianxin Wang,et al.  A survey on U-shaped networks in medical image segmentations , 2020, Neurocomputing.

[8]  José García Rodríguez,et al.  A Review on Deep Learning Techniques Applied to Semantic Segmentation , 2017, ArXiv.

[9]  Dong Ni,et al.  Deep Learning in Medical Ultrasound Analysis: A Review , 2019, Engineering.

[10]  D Paladini,et al.  ISUOG Practice Guidelines (updated): sonographic screening examination of the fetal heart , 2013, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[11]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  D. Paladini,et al.  Prenatal measurement of cardiothoracic ratio in evaluation of heart disease. , 1990, Archives of disease in childhood.

[13]  Xiangjian He,et al.  Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges , 2019, Journal of Digital Imaging.

[14]  H. Ishikawa,et al.  Reliability of the lung to thorax transverse area ratio as a predictive parameter in fetuses with congenital diaphragmatic hernia , 2010, Pediatric Surgery International.

[15]  Ryuji Hamamoto,et al.  Image Segmentation of the Ventricular Septum in Fetal Cardiac Ultrasound Videos Based on Deep Learning Using Time-Series Information , 2020, Biomolecules.

[16]  Anatomy of the Thoracic Wall, Pulmonary Cavities, and Mediastinum , 2005 .

[17]  José García Rodríguez,et al.  A survey on deep learning techniques for image and video semantic segmentation , 2018, Appl. Soft Comput..

[18]  B. Ginneken,et al.  Automated measurement of fetal head circumference using 2D ultrasound images , 2018, PloS one.

[19]  Z Alfirevic,et al.  Practice guidelines for performance of the routine mid‐trimester fetal ultrasound scan , 2011, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[20]  Yu Liu,et al.  A review of semantic segmentation using deep neural networks , 2017, International Journal of Multimedia Information Retrieval.

[21]  Zhidong Deng,et al.  Recent progress in semantic image segmentation , 2018, Artificial Intelligence Review.

[22]  Ping Chen,et al.  Segmentation of Fetal Left Ventricle in Echocardiographic Sequences Based on Dynamic Convolutional Neural Networks , 2017, IEEE Transactions on Biomedical Engineering.

[23]  E. Alibrahim,et al.  A pictorial guide for the second trimester ultrasound , 2013, Australasian journal of ultrasound in medicine.

[24]  Haipeng Shen,et al.  Artificial intelligence in healthcare: past, present and future , 2017, Stroke and Vascular Neurology.

[25]  J. Sasahara,et al.  Correlation between lung to thorax transverse area ratio and observed/expected lung area to head circumference ratio in fetuses with left‐sided diaphragmatic hernia , 2015, Congenital anomalies.

[26]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[27]  Walter Plasencia,et al.  Fully automated, real-time 3D ultrasound segmentation to estimate first trimester placental volume using deep learning. , 2018, JCI insight.

[28]  Ryuji Hamamoto,et al.  Epigenetics Analysis and Integrated Analysis of Multiomics Data, Including Epigenetic Data, Using Artificial Intelligence in the Era of Precision Medicine , 2019, Biomolecules.

[29]  A. Moon‐Grady,et al.  Expert-level prenatal detection of complex congenital heart disease from screening ultrasound using deep learning , 2020, medRxiv.

[30]  Yan Li,et al.  Automatic fetal body and amniotic fluid segmentation from fetal ultrasound images by encoder-decoder network with inner layers , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[31]  J. Bland,et al.  Assessment of the intraobserver variability in the measurement of fetal cardiothoracic ratio using ellipse and diameter methods , 2006, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[32]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[33]  Lior Rokach,et al.  Ensemble learning: A survey , 2018, WIREs Data Mining Knowl. Discov..

[34]  Akihiro Haga,et al.  A Deep Learning Approach for Assessment of Regional Wall Motion Abnormality From Echocardiographic Images. , 2020, JACC. Cardiovascular imaging.

[35]  Y. Ville,et al.  Prenatal diagnosis of fetal skeletal dysplasias by combining two‐dimensional and three‐dimensional ultrasound and intrauterine three‐dimensional helical computer tomography , 2004, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[36]  Cihan Çetin,et al.  Prenatal diagnosis of pectus excavatum , 2016, Turkish journal of obstetrics and gynecology.

[37]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Harvey Lui,et al.  Dense-UNet: a novel multiphoton in vivo cellular image segmentation model based on a convolutional neural network. , 2020, Quantitative imaging in medicine and surgery.