Convolutional neural network for semantic segmentation of fetal echocardiography based on four-chamber view

The acute shortage of trained and experienced sonographers causes the detection of congenital heart defects (CHDs) extremely difficult. In order to minimize this difficulty, an accurate fetal heart segmentation to the early location of such structural heart abnormalities prior to delivery is essential. However, the segmentation process is not an easy task due to the small size of the fetal heart structure. Moreover, the manual task for identifying the standard cardiac planes, primarily based on a four-chamber view, requires a well-trained clinician and experience. In this paper, a CNN method using U-Net architecture was proposed to automate fetal cardiac standard planes segmentation from ultrasound images. A total of 519 fetal cardiac images was obtained from three videos. All data is divided into training and testing data. The testing data consist of 106 slices of the four-chamber segmentation tasks, i.e. atrial septal defect (ASD), ventricular septal defect (VSD), and Normal. The segmentation of the post-processing method is needed to enhanced the segmentation result. In this paper, a combination technique with U-Net and Otsu thresholding gives the best performances with 99.48%-pixel accuracy, 96.73% mean accuracy, 94.92% mean intersection over union, and 0.21% segmentation error. In the future, the implementation of Deep Learning in the study of CHDs holds significant potential for identifying novel CHDs in heterogeneous fetal hearts.

[1]  Eliasz Kantoch,et al.  Recognition of Sedentary Behavior by Machine Learning Analysis of Wearable Sensors during Activities of Daily Living for Telemedical Assessment of Cardiovascular Risk , 2018, Sensors.

[2]  A. Fanaroff.  Birth Prevalence of Congenital Heart Disease Worldwide: A Systematic Review and Meta-Analysis , 2012 .

[3]  Yanbo Zhang,et al.  Predicting congenital heart defects: A comparison of three data mining methods , 2017, PloS one.

[4]  S. Nurmaini,et al.  Cardiac Arrhythmias Classification Using Deep Neural Networks and Principle Component Analysis Algorithm , 2018 .

[5]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[6]  Jan Marek,et al.  Does First-Trimester Screening Modify the Natural History of Congenital Heart Disease?: Analysis of Outcome of Regional Cardiac Screening at 2 Different Time Periods , 2017, Circulation.

[7]  Yassine Ruichek,et al.  Survey on semantic segmentation using deep learning techniques , 2019, Neurocomputing.

[8]  Dong Yu,et al.  Deep Learning: Methods and Applications , 2014, Found. Trends Signal Process..

[9]  Jie Yang,et al.  A Novel Adaptive Gaussian Mixture Model for Background Subtraction , 2005, IbPRIA.

[10]  N. Ramachandra,et al.  A rare case of congenital heart disease with ambiguous genitalia , 2010, Indian journal of human genetics.

[11]  J. Alison Noble,et al.  Automated annotation and quantitative description of ultrasound videos of the fetal heart , 2017, Medical Image Anal..

[12]  Vaanathi Sundaresan,et al.  Automated characterization of the fetal heart in ultrasound images using fully convolutional neural networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[13]  J. Alison Noble,et al.  A framework for analysis of linear ultrasound videos to detect fetal presentation and heartbeat , 2017, Medical Image Anal..

[14]  Abraham J. Wyner,et al.  Modern Neural Networks Generalize on Small Data Sets , 2018, NeurIPS.

[15]  Suchi Saria,et al.  Better medicine through machine learning: What’s real, and what’s artificial? , 2018, PLoS medicine.

[16]  Muhammad Imran Razzak,et al.  Deep Learning for Medical Image Processing: Overview, Challenges and Future , 2017, ArXiv.

[17]  E. Araújo Júnior,et al.  Prenatal diagnosis of congenital heart disease: A review of current knowledge , 2017, Indian heart journal.

[18]  Guang Yang,et al.  Automatic Brain Tumor Detection and Segmentation Using U-Net Based Fully Convolutional Networks , 2017, MIUA.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  L. Hunter,et al.  EDUCATIONAL SERIES IN CONGENITAL HEART DISEASE: Prenatal diagnosis of congenital heart disease , 2018, Echo research and practice.

[21]  J. Alison Noble,et al.  Describing ultrasound video content using deep convolutional neural networks , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[22]  Annisa Darmawahyuni,et al.  Deep Learning with a Recurrent Network Structure in the Sequence Modeling of Imbalanced Data for ECG-Rhythm Classifier , 2019, Algorithms.

[23]  Teddy Mantoro,et al.  Segmentation and Classification of Cervical Cells Using Deep Learning , 2019, IEEE Access.

[24]  J P Rossiter,et al.  Prenatal diagnosis of congenital heart disease. , 1993, Obstetrics and gynecology clinics of North America.

[25]  F. Sardanelli,et al.  Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine , 2018, European Radiology Experimental.

[26]  Annisa Darmawahyuni,et al.  Coronary Heart Disease Interpretation Based on Deep Neural Network , 2019, Computer Engineering and Applications Journal.

[27]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[28]  Thomas Brox,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[29]  Robert H. Anderson,et al.  Anatomy of the normal fetal heart: The basis for understanding fetal echocardiography , 2018, Annals of Pediatric Cardiology.

[30]  Linda Sulistiawati,et al.  INFLUENCE OF MOTIVATION OF EM- PLOYEES IN THE PERFORMANCE DIRECTORATE GENERAL OF NUTRITION AND KIA (MOM FAMILY CHILD) MINISTRY OF HEALTH REPUBLIC OF INDONESIA , 2012 .

[31]  Po-Ling Loh,et al.  Machine learning to detect signatures of disease in liquid biopsies - a user's guide. , 2018, Lab on a chip.

[32]  Ping Chen,et al.  Segmentation of Fetal Left Ventricle in Echocardiographic Sequences Based on Dynamic Convolutional Neural Networks , 2017, IEEE Transactions on Biomedical Engineering.

[33]  Joanna Jaworek-Korjakowska,et al.  Region Adjacency Graph Approach for Acral Melanocytic Lesion Segmentation , 2018, Applied Sciences.

[34]  R. Filly,et al.  Prenatal Imaging: Ultrasonography and Magnetic Resonance Imaging , 2008, Obstetrics and gynecology.

[35]  Annisa Darmawahyuni,et al.  An Automated ECG Beat Classification System Using Deep Neural Networks with an Unsupervised Feature Extraction Technique , 2019, Applied Sciences.

[36]  Taghi M. Khoshgoftaar,et al.  A survey on Image Data Augmentation for Deep Learning , 2019, Journal of Big Data.

[37]  Yihua He,et al.  Convolutional-Neural-Network-Based Approach for Segmentation of Apical Four-Chamber View from Fetal Echocardiography , 2020, IEEE Access.

[38]  Hao Chen,et al.  Automatic Fetal Ultrasound Standard Plane Detection Using Knowledge Transferred Recurrent Neural Networks , 2015, MICCAI.

[39]  Jianhua Guo,et al.  DW-Net: A cascaded convolutional neural network for apical four-chamber view segmentation in fetal echocardiography , 2019, Comput. Medical Imaging Graph..

[40]  Konstantinos Kamnitsas,et al.  Real-Time Standard Scan Plane Detection and Localisation in Fetal Ultrasound Using Fully Convolutional Neural Networks , 2016, MICCAI.