Ultrasound Image Segmentation: A Deeply Supervised Network With Attention to Boundaries

Objective: Segmentation of anatomical structures in ultrasound images requires vast radiological knowledge and experience. Moreover, the manual segmentation often results in subjective variations, therefore, an automatic segmentation is desirable. We aim to develop a fully convolutional neural network (FCNN) with attentional deep supervision for the automatic and accurate segmentation of the ultrasound images. Method: FCNN/CNNs are used to infer high-level context using low-level image features. In this paper, a sub-problem specific deep supervision of the FCNN is performed. The attention of fine resolution layers is steered to learn object boundary definitions using auxiliary losses, whereas coarse resolution layers are trained to discriminate object regions from the background. Furthermore, a customized scheme for downweighting the auxiliary losses and a trainable fusion layer are introduced. This produces an accurate segmentation and helps in dealing with the broken boundaries, usually found in the ultrasound images. Results: The proposed network is first tested for blood vessel segmentation in liver images. It results in $F1$ score, mean intersection over union, and dice index of 0.83, 0.83, and 0.79, respectively. The best values observed among the existing approaches are produced by U-net as 0.74, 0.81, and 0.75, respectively. The proposed network also results in dice index value of 0.91 in the lumen segmentation experiments on MICCAI 2011 IVUS challenge dataset, which is near to the provided reference value of 0.93. Furthermore, the improvements similar to vessel segmentation experiments are also observed in the experiment performed to segment lesions. Conclusion: Deep supervision of the network based on the input-output characteristics of the layers results in improvement in overall segmentation accuracy. Significance: Sub-problem specific deep supervision for ultrasound image segmentation is the main contribution of this paper. Currently the network is trained and tested for fixed size inputs. It requires image resizing and limits the performance in small size images.

[1]  Hao Chen,et al.  Deep Contextual Networks for Neuronal Structure Segmentation , 2016, AAAI.

[2]  Frits Mastik,et al.  Fully automatic luminal contour segmentation in intracoronary ultrasound imaging-a statistical approach , 2004, IEEE Transactions on Medical Imaging.

[3]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[4]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[5]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[6]  Liang Lin,et al.  Recognizing focal liver lesions in contrast-enhanced ultrasound with discriminatively trained spatio-temporal model , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[9]  Dorin Comaniciu,et al.  Database-guided segmentation of anatomical structures with complex appearance , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Christos P. Loizou,et al.  Segmentation of the Common Carotid Intima-Media Complex in Ultrasound Images Using Active Contours , 2012, IEEE Transactions on Biomedical Engineering.

[11]  Razvan Pascanu,et al.  Theano: A CPU and GPU Math Compiler in Python , 2010, SciPy.

[12]  G. Macones,et al.  Ultrasound-Detected Subchorionic Hemorrhage and the Obstetric Implications , 2010, Obstetrics and gynecology.

[13]  David J. Hawkes,et al.  Alignment of sparse freehand 3-D ultrasound with preoperative images of the liver using models of respiratory motion and deformation , 2005, IEEE Transactions on Medical Imaging.

[14]  Septimiu E. Salcudean,et al.  Real-Time Vessel Segmentation and Tracking for Ultrasound Imaging Applications , 2007, IEEE Transactions on Medical Imaging.

[15]  Lin Yang,et al.  Coarse-to-Fine Stacked Fully Convolutional Nets for lymph node segmentation in ultrasound images , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[16]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[17]  João Manuel R. S. Tavares,et al.  Automatic segmentation of the lumen region in intravascular images of the coronary artery , 2017, Medical Image Anal..

[18]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[19]  Zhuowen Tu,et al.  Training Deeper Convolutional Networks with Deep Supervision , 2015, ArXiv.

[20]  Zhuowen Tu,et al.  Deeply-Supervised Nets , 2014, AISTATS.

[21]  J A Noble,et al.  Ultrasound image segmentation and tissue characterization , 2010, Proceedings of the Institution of Mechanical Engineers. Part H, Journal of engineering in medicine.

[22]  Ioannis Pitas,et al.  Segmentation of ultrasonic images using Support Vector Machines , 2003, Pattern Recognit. Lett..

[23]  Alejandro F. Frangi,et al.  Muliscale Vessel Enhancement Filtering , 1998, MICCAI.

[24]  Jae Young Lee,et al.  Automatic registration between 3D intra-operative ultrasound and pre-operative CT images of the liver based on robust edge matching , 2012, Physics in medicine and biology.

[25]  Purang Abolmaesumi,et al.  Real-time extraction of carotid artery contours from ultrasound images , 2000, Proceedings 13th IEEE Symposium on Computer-Based Medical Systems. CBMS 2000.

[26]  Xiangmin Xu,et al.  An automatic energy-based region growing method for ultrasound image segmentation , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[27]  Hao Chen,et al.  Iterative Multi-domain Regularized Deep Learning for Anatomical Structure Detection and Segmentation from Ultrasound Images , 2016, MICCAI.

[28]  Ke Chen,et al.  Automated Vessel Segmentation Using Infinite Perimeter Active Contour Model with Hybrid Region Information with Application to Retinal Images , 2015, IEEE Transactions on Medical Imaging.

[29]  Petia Radeva,et al.  A Convolutional Neural Network for Automatic Characterization of Plaque Composition in Carotid Ultrasound , 2017, IEEE Journal of Biomedical and Health Informatics.

[30]  Wen-Hong Zhu,et al.  Image-guided control of a robot for medical ultrasound , 2002, IEEE Trans. Robotics Autom..

[31]  Jean-Louis Dillenseger,et al.  Liver Vessels Segmentation Using a Hybrid Geometrical Moments/Graph Cuts Method , 2010, IEEE Transactions on Biomedical Engineering.

[32]  Ohin Kwon,et al.  Fast segmentation of ultrasound images using robust Rayleigh distribution decomposition , 2012, Pattern Recognit..

[33]  J. Alison Noble,et al.  Ultrasound image segmentation: a survey , 2006, IEEE Transactions on Medical Imaging.

[34]  Iasonas Kokkinos,et al.  Pushing the Boundaries of Boundary Detection using Deep Learning , 2015, ICLR 2016.

[35]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[36]  Dimitrios I. Fotiadis,et al.  An automated method for lumen and media-adventitia border detection in a sequence of IVUS frames , 2004, IEEE Transactions on Information Technology in Biomedicine.

[37]  Jonathan T. Barron,et al.  Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Yongtian Wang,et al.  Recognizing Focal Liver Lesions in CEUS With Dynamically Trained Latent Structured Models , 2016, IEEE Transactions on Medical Imaging.

[39]  Hao Chen,et al.  DCAN: Deep contour‐aware networks for object instance segmentation from histology images , 2017, Medical Image Anal..

[40]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[41]  Y. Saijo,et al.  Detection of Luminal Contour Using Fuzzy Clustering and Mathematical Morphology in Intravascular Ultrasound Images , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[42]  J. Alison Noble.  Reflections on ultrasound image analysis , 2016, Medical Image Anal..

[43]  William E. Higgins,et al.  Methods for 2-D and 3-D Endobronchial Ultrasound Image Segmentation , 2016, IEEE Transactions on Biomedical Engineering.

[44]  Mohammad I. Daoud,et al.  Accurate and fully automatic segmentation of breast ultrasound images by combining image boundary and region information , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[45]  Syed Muhammad Anwar,et al.  Deep Learning in Medical Image Analysis , 2017 .

[46]  Naser Movahhedinia,et al.  An automated approach for segmentation of intravascular ultrasound images based on parametric active contour models , 2012, Australasian Physical & Engineering Sciences in Medicine.

[47]  Charless C. Fowlkes,et al.  Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation , 2016, ECCV.

[48]  Anderson Rocha,et al.  Object Delineation by -Connected Components , 2008, EURASIP J. Adv. Signal Process..

[49]  Gustavo Carneiro,et al.  Detection and Measurement of Fetal Anatomies from Ultrasound Images using a Constrained Probabilistic Boosting Tree , 2008, IEEE Transactions on Medical Imaging.

[50]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  José Ignacio Orlando,et al.  Assessment of image features for vessel wall segmentation in intravascular ultrasound images , 2016, International Journal of Computer Assisted Radiology and Surgery.

[52]  Sérgio Shiguemi Furuie,et al.  Automatic coronary wall segmentation in intravascular ultrasound images using binary morphological reconstruction. , 2011, Ultrasound in medicine & biology.