Automated Segmentation of Lesions in Ultrasound Using Semi-pixel-wise Cycle Generative Adversarial Nets

Breast cancer is the most common invasive cancer with the highest cancer occurrence in females. Handheld ultrasound is one of the most efficient ways to identify and diagnose the breast cancer. The area and the shape information of a lesion is very helpful for clinicians to make diagnostic decisions. In this study we propose a new deep-learning scheme, semi-pixel-wise cycle generative adversarial net (SPCGAN) for segmenting the lesion in 2D ultrasound. The method takes the advantage of a fully connected convolutional neural network (FCN) and a generative adversarial net to segment a lesion by using prior knowledge. We compared the proposed method to a fully connected neural network and the level set segmentation method on a test dataset consisting of 32 malignant lesions and 109 benign lesions. Our proposed method achieved a Dice similarity coefficient (DSC) of 0.92 while FCN and the level set achieved 0.90 and 0.79 respectively. Particularly, for malignant lesions, our method increases the DSC (0.90) of the fully connected neural network to 0.93 significantly (p$<$0.001). The results show that our SPCGAN can obtain robust segmentation results and may be used to relieve the radiologists' burden for annotation.

[1]  M. Giger,et al.  Automatic segmentation of breast lesions on ultrasound. , 2001, Medical physics.

[2]  Nico Karssemeijer,et al.  Computer-Aided Lesion Diagnosis in Automated 3-D Breast Ultrasound Using Coronal Spiculation , 2012, IEEE Transactions on Medical Imaging.

[3]  Xavier Lladó,et al.  Lesion Segmentation in Automated 3D Breast Ultrasound: Volumetric Analysis , 2018, Ultrasonic imaging.

[4]  T. Tukeva,et al.  Comparison of breast structure and findings by X-ray mammography, ultrasound, cytology and histology: a retrospective study , 1995 .

[5]  Pheng-Ann Heng,et al.  Cascaded Fully Convolutional Networks for automatic prenatal ultrasound image segmentation , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[6]  M. Soryani,et al.  Mass Segmentation in Automated 3-D Breast Ultrasound Using Adaptive Region Growing and Supervised Edge-Based Deformable Model , 2018, IEEE Transactions on Medical Imaging.

[7]  Nico Karssemeijer,et al.  Computer-Aided Detection of Cancer in Automated 3-D Breast Ultrasound , 2013, IEEE Transactions on Medical Imaging.

[8]  Lin Yang,et al.  Coarse-to-Fine Stacked Fully Convolutional Nets for lymph node segmentation in ultrasound images , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[9]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[10]  Mingyuan Gao,et al.  An adaptive Fuzzy C‐means method utilizing neighboring information for breast tumor segmentation in ultrasound images , 2017, Medical physics.

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  Nico Karssemeijer,et al.  Computer-aided detection of breast cancers using Haar-like features in automated 3D breast ultrasound. , 2015, Medical physics.

[13]  Tao Tan,et al.  Denoising of 3D magnetic resonance images with multi-channel residual learning of convolutional neural network , 2017, Japanese Journal of Radiology.

[14]  Reyer Zwiggelaar,et al.  Automated Breast Ultrasound Lesions Detection Using Convolutional Neural Networks , 2018, IEEE Journal of Biomedical and Health Informatics.

[15]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Hariharan Ravishankar,et al.  Joint Deep Learning of Foreground, Background and Shape for Robust Contextual Segmentation , 2017, IPMI.

[17]  Seyed-Ahmad Ahmadi,et al.  Hough-CNN: Deep learning for segmentation of deep brain regions in MRI and ultrasound , 2016, Comput. Vis. Image Underst..

[18]  Hamid Behnam,et al.  Breast cancer detection in automated 3D breast ultrasound using iso‐contours and cascaded RUSBoosts , 2017, Ultrasonics.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  R. Martí,et al.  Breast-lesion Segmentation Combining B-Mode and Elastography Ultrasound , 2016, Ultrasonic imaging.

[21]  Jun Ren,et al.  Breast Cancer Challenges and Screening in China: Lessons From Current Registry Data and Population Screening Studies. , 2015, The oncologist.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Vaanathi Sundaresan,et al.  Automated characterization of the fetal heart in ultrasound images using fully convolutional neural networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[24]  Konstantinos Kamnitsas,et al.  Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation , 2017, IEEE Transactions on Medical Imaging.

[25]  Gustavo Carneiro,et al.  Combining deep learning and level set for the automated segmentation of the left ventricle of the heart from cardiac cine magnetic resonance , 2017, Medical Image Anal..

[26]  Viksit Kumar,et al.  Automated and real-time segmentation of suspicious breast masses using convolutional neural network , 2018, PloS one.

[27]  Samuel J. Magny,et al.  Breast Imaging Reporting and Data System , 2020, Definitions.

[28]  Guillermo Sapiro,et al.  Geodesic Active Contours , 1995, International Journal of Computer Vision.

[29]  Enzo Galligioni,et al.  Breast screening with ultrasound in women with mammography-negative dense breasts: evidence on incremental cancer detection and false positives, and associated cost. , 2008, European journal of cancer.

[30]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[31]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[32]  Berkman Sahiner,et al.  A new automated method for the segmentation and characterization of breast masses on ultrasound images. , 2009, Medical physics.

[33]  Tao Tan,et al.  Optimize Transfer Learning for Lung Diseases in Bronchoscopy Using a New Concept: Sequential Fine-Tuning , 2018, IEEE Journal of Translational Engineering in Health and Medicine.

[34]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[35]  L. Liberman,et al.  Breast imaging reporting and data system (BI-RADS). , 2002, Radiologic clinics of North America.

[36]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Mitko Veta,et al.  Roto-Translation Covariant Convolutional Networks for Medical Image Analysis , 2018, MICCAI.

[38]  Nico Karssemeijer,et al.  Evaluation of the effect of computer-aided classification of benign and malignant lesions on reader performance in automated three-dimensional breast ultrasound. , 2013, Academic radiology.

[39]  Rafael Rodrigues,et al.  A Two-Step Segmentation Method for Breast Ultrasound Masses Based on Multi-resolution Analysis. , 2015, Ultrasound in medicine & biology.

[40]  Nico Karssemeijer,et al.  Incorporating texture features in a computer-aided breast lesion diagnosis system for automated three-dimensional breast ultrasound , 2014, Journal of medical imaging.

[41]  K. Kaul,et al.  Early Detection of Breast Cancer: Is Mammography Enough? , 2002 .

[42]  Nico Karssemeijer,et al.  Segmentation of malignant lesions in 3D breast ultrasound using a depth-dependent model. , 2016, Medical physics.