Shadow Estimation for Ultrasound Images Using Auto-Encoding Structures and Synthetic Shadows

Acoustic shadows are common artifacts in medical ultrasound imaging. The shadows are caused by objects that reflect ultrasound such as bones, and they are shown as dark areas in ultrasound images. Detecting such shadows is crucial for assessing the quality of images. This will be a pre-processing for further image processing or recognition aiming computer-aided diagnosis. In this paper, we propose an auto-encoding structure that estimates the shadowed areas and their intensities. The model once splits an input image into an estimated shadow image and an estimated shadow-free image through its encoder and decoder. Then, it combines them to reconstruct the input. By generating plausible synthetic shadows based on relatively coarse domain-specific knowledge on ultrasound images, we can train the model using unlabeled data. If pixel-level labels of the shadows are available, we also utilize them in a semi-supervised fashion. By experiments on ultrasound images for fetal heart diagnosis, we show that our method achieved 0.720 in the DICE score and outperformed conventional image processing methods and a segmentation method based on deep neural networks. The capability of the proposed method on estimating the intensities of shadows and the shadow-free images is also indicated through the experiments.

[1]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[2]  T. Desser,et al.  Tissue harmonic imaging: Utility in abdominal and pelvic sonography , 1999, Journal of clinical ultrasound : JCU.

[3]  Daniel Rueckert,et al.  Deep Learning for Cardiac Image Segmentation: A Review , 2020, Frontiers in Cardiovascular Medicine.

[4]  Ilker Hacihaliloglu Enhancement of bone shadow region using local phase-based ultrasound transmission maps , 2017, International Journal of Computer Assisted Radiology and Surgery.

[5]  J. Noble,et al.  Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology , 2020, Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology.

[6]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[7]  A. Samir,et al.  Machine learning for medical ultrasound: status, methods, and future opportunities , 2018, Abdominal Radiology.

[8]  José García Rodríguez,et al.  A survey on deep learning techniques for image and video semantic segmentation , 2018, Appl. Soft Comput..

[9]  A. Mcneilly,et al.  A Comparison of the Imaging Performance of High Resolution Ultrasound Scanners for Preclinical Imaging , 2011, Ultrasound in Medicine and Biology.

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  D. Louis Collins,et al.  An automatic geometrical and statistical method to detect acoustic shadows in intraoperative ultrasound brain images , 2010, Medical Image Anal..

[12]  Martin D. Fox,et al.  Ultrasound image enhancement: A review , 2012, Biomed. Signal Process. Control..

[13]  Suguru Yasutomi,et al.  Shadow Detection for Ultrasound Images Using Unlabeled Data and Synthetic Shadows , 2019, 1908.01439.

[14]  Nassir Navab,et al.  Ultrasound confidence maps using random walks , 2012, Medical Image Anal..

[15]  Ryuji Hamamoto,et al.  Model-Agnostic Method for Thoracic Wall Segmentation in Fetal Ultrasound Videos , 2020, Biomolecules.

[16]  Ken Asada,et al.  Detection of Cardiac Structural Abnormalities in Fetal Ultrasound Videos Using Deep Learning , 2021, Applied Sciences.

[17]  J. Alison Noble,et al.  Ultrasound image segmentation: a survey , 2006, IEEE Transactions on Medical Imaging.

[18]  Tapani Raiko,et al.  Semi-supervised Learning with Ladder Networks , 2015, NIPS.

[19]  Daniel Rueckert,et al.  Weakly Supervised Estimation of Shadow Confidence Maps in Fetal Ultrasound Imaging , 2019, IEEE Transactions on Medical Imaging.

[20]  Ryuji Hamamoto,et al.  Image Segmentation of the Ventricular Septum in Fetal Cardiac Ultrasound Videos Based on Deep Learning Using Time-Series Information , 2020, Biomolecules.

[21]  Dong Ni,et al.  Deep Learning in Medical Ultrasound Analysis: A Review , 2019, Engineering.

[22]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[23]  D. Thickman,et al.  The comet tail artifact. , 1982, Journal of ultrasound in medicine : official journal of the American Institute of Ultrasound in Medicine.

[24]  Peter A. Calabresi,et al.  Evaluating White Matter Lesion Segmentations with Refined Sørensen-Dice Analysis , 2020, Scientific Reports.

[25]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[26]  Navdeep Jaitly,et al.  Adversarial Autoencoders , 2015, ArXiv.

[27]  B A Porter,et al.  Real-time spatial compound imaging: application to breast, vascular, and musculoskeletal ultrasound. , 2001, Seminars in ultrasound, CT, and MR.

[28]  Jeremy Tan,et al.  Automatic Shadow Detection in 2D Ultrasound Images , 2018, DATRA/PIPPI@MICCAI.

[29]  Andrea Scorza,et al.  Image quality evaluation of ultrasound imaging systems: advanced B‐modes , 2019, Journal of applied clinical medical physics.