AUNet: attention-guided dense-upsampling networks for breast mass segmentation in whole mammograms

Mammography is one of the most commonly applied tools for early breast cancer screening. Automatic segmentation of breast masses in mammograms is essential but challenging due to the low signal-to-noise ratio and the wide variety of mass shapes and sizes. Existing methods deal with these challenges by extracting mass-centered image patches manually or automatically. However, manual patch extraction is time-consuming and automatic patch extraction brings errors that could not be compensated in the following segmentation step. In this study, we propose a novel attention-guided dense-upsampling network (AUNet) for accurate breast mass segmentation in whole mammograms directly. In AUNet, we employ an asymmetrical encoder-decoder structure and propose an effective upsampling block, attention-guided dense-upsampling block (AU block). Especially, the AU block is designed to have three merits. Firstly, it compensates the information loss of bilinear upsampling by dense upsampling. Secondly, it designs a more effective method to fuse high- and low-level features. Thirdly, it includes a channel-attention function to highlight rich-information channels. We evaluated the proposed method on two publicly available datasets, CBIS-DDSM and INbreast. Compared to three state-of-the-art fully convolutional networks, AUNet achieved the best performances with an average Dice similarity coefficient of 81.8% for CBIS-DDSM and 79.1% for INbreast.

[1]  Carrie M. Rochman,et al.  Digital Breast Tomosynthesis in the Diagnostic Setting: Indications and Clinical Applications. , 2015, Radiographics : a review publication of the Radiological Society of North America, Inc.

[2]  Bin Yan,et al.  Fully Convolutional DenseNet with Multiscale Context for Automated Breast Tumor Segmentation , 2019, Journal of healthcare engineering.

[3]  Jaime S. Cardoso,et al.  Closed Shortest Path in the Original Coordinates with an Application to Breast Cancer , 2015, Int. J. Pattern Recognit. Artif. Intell..

[4]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  D. Ikeda,et al.  Unsupervised Clustering of Quantitative Image Phenotypes Reveals Breast Cancer Subtypes with Distinct Prognoses and Molecular Pathways , 2017, Clinical Cancer Research.

[6]  C. Lehman,et al.  Diagnostic Accuracy of Digital Screening Mammography With and Without Computer-Aided Detection. , 2015, JAMA internal medicine.

[7]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[8]  A. Retico,et al.  Mammogram Segmentation by Contour Searching and Mass Lesions Classification With Neural Network , 2004, IEEE Transactions on Nuclear Science.

[9]  Arnau Oliver,et al.  A review of automatic mass detection and segmentation in mammographic images , 2010, Medical Image Anal..

[10]  Garrison W. Cottrell,et al.  Understanding Convolution for Semantic Segmentation , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[11]  Jia Gu,et al.  Superpixel-based and boundary-sensitive convolutional neural network for automated liver segmentation , 2018, Physics in medicine and biology.

[12]  Xiaohui Xie,et al.  AnatomyNet: Deep learning for fast and fully automated whole‐volume segmentation of head and neck anatomy , 2018, Medical physics.

[13]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Gustavo Carneiro,et al.  Deep structured learning for mass segmentation from mammograms , 2014, 2015 IEEE International Conference on Image Processing (ICIP).

[15]  Berkman Sahiner,et al.  Computer-aided detection of breast masses on mammograms: dual system approach with two-view analysis. , 2009, Medical physics.

[16]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[17]  Linda G. Shapiro,et al.  Y-Net: Joint Segmentation and Classification for Diagnosis of Breast Biopsy Images , 2018, MICCAI.

[18]  Nico Karssemeijer,et al.  Large scale deep learning for computer aided detection of mammographic lesions , 2017, Medical Image Anal..

[19]  Tat-Seng Chua,et al.  SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[21]  Yuan Hong,et al.  Automatic segmentation of levator hiatus from ultrasound images using U-net with dense connections , 2019, Physics in medicine and biology.

[22]  D. Rubin,et al.  Heterogeneous Enhancement Patterns of Tumor-adjacent Parenchyma at MR Imaging Are Associated with Dysregulated Signaling Pathways and Poor Survival in Breast Cancer. , 2017, Radiology.

[23]  Seokmin Han,et al.  A deep learning framework for supporting the classification of breast lesions in ultrasound images , 2017, Physics in medicine and biology.

[24]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Nassir Navab,et al.  Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks , 2018, MICCAI.

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Daniel L Rubin,et al.  A curated mammography data set for use in computer-aided detection and diagnosis research , 2017, Scientific Data.

[29]  G. Czarnota,et al.  Computer aided prognosis for cell death categorization and prediction in vivo using quantitative ultrasound and machine learning techniques. , 2016, Medical physics.

[30]  Yaozong Gao,et al.  ASDNet: Attention Based Semi-supervised Deep Networks for Medical Image Segmentation , 2018, MICCAI.

[31]  Ruijiang Li,et al.  Magnetic resonance imaging and molecular features associated with tumor-infiltrating lymphocytes in breast cancer , 2018, Breast Cancer Research.

[32]  Gustavo Carneiro,et al.  A deep learning approach for the analysis of masses in mammograms with minimal user intervention , 2017, Medical Image Anal..

[33]  Gustavo Carneiro,et al.  Automated Mass Detection in Mammograms Using Cascaded Deep Learning and Random Forests , 2015, 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[34]  Huai Li,et al.  A multiple circular path convolution neural network system for detection of mammographic masses , 2002, IEEE Transactions on Medical Imaging.

[35]  E. Denton,et al.  Eigendetection of masses considering false positive reduction and breast density information. , 2008, Medical physics.

[36]  Yong Man Ro,et al.  Visually interpretable deep network for diagnosis of breast masses on mammograms , 2018, Physics in medicine and biology.

[37]  Steve B. Jiang,et al.  Fully automated organ segmentation in male pelvic CT images , 2018, Physics in medicine and biology.

[38]  Jaime S. Cardoso,et al.  INbreast: toward a full-field digital mammographic database. , 2012, Academic radiology.

[39]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[40]  Georgia D. Tourassi,et al.  A Concentric Morphology Model for the Detection of Masses in Mammography , 2007, IEEE Transactions on Medical Imaging.

[41]  Gustavo Carneiro,et al.  Deep Learning and Structured Prediction for the Segmentation of Mass in Mammograms , 2015, MICCAI.

[42]  Anne Marie McCarthy,et al.  The use of the Gail model, body mass index and SNPs to predict breast cancer among women with abnormal (BI-RADS 4) mammograms , 2014, Breast Cancer Research.

[43]  Michael Bretthauer,et al.  Benefits and harms of mammography screening , 2015, Breast Cancer Research.

[44]  Gang Sun,et al.  Squeeze-and-Excitation Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[45]  Mohamed S. Kamel,et al.  Categorizing Extent of Tumor Cell Death Response to Cancer Therapy Using Quantitative Ultrasound Spectroscopy and Maximum Mean Discrepancy , 2014, IEEE Transactions on Medical Imaging.

[46]  Lubomir M. Hadjiiski,et al.  Characterization of mammographic masses based on level set segmentation with new image features and patient information. , 2007, Medical physics.

[47]  J. Alison Noble,et al.  {\Omega}-Net (Omega-Net): Fully Automatic, Multi-View Cardiac MR Detection, Orientation, and Segmentation with Deep Neural Networks. , 2017 .

[48]  M.R. El-Sakka,et al.  Fuzzy Entropy Based Detection of Suspicious Masses in Digital Mammogram Images , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[49]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  N. Dubrawsky Cancer statistics , 1989, CA: a cancer journal for clinicians.

[51]  Berkman Sahiner,et al.  Dual system approach to computer-aided detection of breast masses on mammograms. , 2006, Medical physics.

[52]  Krzysztof J. Geras,et al.  Artificial Intelligence for Mammography and Digital Breast Tomosynthesis: Current Concepts and Future Perspectives. , 2019, Radiology.

[53]  Sanjiv Kumar,et al.  On the Convergence of Adam and Beyond , 2018 .

[54]  KathleenR. Brandt,et al.  Screening mammograms: interpretation with computer-aided detection--prospective evaluation. , 2006, Radiology.

[55]  Reyer Zwiggelaar,et al.  Deep learning in mammography and breast histology, an overview and future trends , 2018, Medical Image Anal..

[56]  J. Alison Noble,et al.  VP‐Nets : Efficient automatic localization of key brain structures in 3D fetal neurosonography , 2018, Medical Image Anal..

[57]  Maryellen L. Giger,et al.  Automated seeded lesion segmentation on digital mammograms , 1998, IEEE Transactions on Medical Imaging.

[58]  Ghassan Hamarneh,et al.  Star Shape Prior in Fully Convolutional Networks for Skin Lesion Segmentation , 2018, MICCAI.

[59]  Tolga Tasdizen,et al.  Image reconstruction using priors from deep learning , 2018, Medical Imaging.

[60]  Jian Sun,et al.  ExFuse: Enhancing Feature Fusion for Semantic Segmentation , 2018, ECCV.

[61]  Chih-Cheng Hung,et al.  Hybrid segmentation of mass in mammograms using template matching and dynamic programming. , 2010, Academic radiology.

[62]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[63]  A. Jemal,et al.  Cancer statistics, 2017 , 2017, CA: a cancer journal for clinicians.

[64]  Won-Ki Jeong,et al.  FusionNet: A Deep Fully Residual Convolutional Neural Network for Image Segmentation in Connectomics , 2016, Frontiers in Computer Science.

[65]  Lubomir M. Hadjiiski,et al.  Mass detection in digital breast tomosynthesis: Deep convolutional neural network with transfer learning from mammography. , 2016, Medical physics.

[66]  Berkman Sahiner,et al.  Computer-aided characterization of mammographic masses: accuracy of mass segmentation and its effects on characterization , 2001, IEEE Transactions on Medical Imaging.

[67]  L.M. Bruce,et al.  Digital Mammographic Computer Aided Diagnosis (CAD) using Adaptive Level Set Segmentation , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[68]  T.O. Gulsrud,et al.  Watershed segmentation of detected masses in digital mammograms , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[69]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  Lubomir M. Hadjiiski,et al.  Breast Cancer Diagnosis in Digital Breast Tomosynthesis: Effects of Training Sample Size on Multi-Stage Transfer Learning Using Deep Neural Nets , 2019, IEEE Transactions on Medical Imaging.

[71]  Andrew P. Bradley,et al.  Automated Analysis of Unregistered Multi-View Mammograms With Deep Learning , 2017, IEEE Transactions on Medical Imaging.

[72]  Thomas A. Funkhouser,et al.  Dilated Residual Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  D. Ikeda,et al.  Mammographic characteristics of 115 missed cancers later detected with screening mammography and the potential utility of computer-aided detection. , 2001, Radiology.

[74]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[75]  M. Giger,et al.  Breast image analysis for risk assessment, detection, diagnosis, and treatment of cancer. , 2013, Annual review of biomedical engineering.

[76]  Natasha K. Stout,et al.  Association of Digital Breast Tomosynthesis vs Digital Mammography With Cancer Detection and Recall Rates by Age and Breast Density , 2019, JAMA oncology.

[77]  Richard H. Moore,et al.  THE DIGITAL DATABASE FOR SCREENING MAMMOGRAPHY , 2007 .

[78]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[79]  Yoshua Bengio,et al.  The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[80]  P. Langenberg,et al.  Breast Imaging Reporting and Data System: inter- and intraobserver variability in feature analysis and final assessment. , 2000, AJR. American journal of roentgenology.

[81]  Shuyi Li,et al.  Attention Dense-U-Net for Automatic Breast Mass Segmentation in Digital Mammogram , 2019, IEEE Access.

[82]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[83]  Dimitris N. Metaxas,et al.  Computer-Aided Diagnosis of Mammographic Masses Using Scalable Image Retrieval , 2015, IEEE Transactions on Biomedical Engineering.

[84]  Xin Yang,et al.  Towards Automated Semantic Segmentation in Prenatal Volumetric Ultrasound , 2019, IEEE Transactions on Medical Imaging.

[85]  Xiaohui Xie,et al.  AnatomyNet: Deep 3D Squeeze-and-excitation U-Nets for fast and fully automated whole-volume anatomical segmentation , 2018, bioRxiv.

[86]  L.M. Bruce,et al.  Towards automated segmentation and classification of masses in mammograms , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[87]  Lubomir M. Hadjiiski,et al.  Deep-learning convolution neural network for computer-aided detection of microcalcifications in digital breast tomosynthesis , 2016, SPIE Medical Imaging.

[88]  Chi-Wing Fu,et al.  H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes , 2018, IEEE Transactions on Medical Imaging.

[89]  Hiroshi Fujita,et al.  Development of an automated method for detecting mammographic masses with a partial loss of region , 2001, IEEE Transactions on Medical Imaging.

[90]  Ghassan Hamarneh,et al.  Mammography Segmentation with Maximum Likelihood Active Contours , 2022 .

[91]  Lihua Li,et al.  Computer-aided diagnosis of masses with full-field digital mammography. , 2002, Academic radiology.

[92]  Rangaraj M. Rangayyan,et al.  Polygonal Modeling of Contours of Breast Tumors With the Preservation of Spicules , 2008, IEEE Transactions on Biomedical Engineering.

[93]  Ruijiang Li,et al.  Intratumoral Spatial Heterogeneity at Perfusion MR Imaging Predicts Recurrence-free Survival in Locally Advanced Breast Cancer Treated with Neoadjuvant Chemotherapy. , 2018, Radiology.

[94]  Konstantinos Kamnitsas,et al.  Efficient multi‐scale 3D CNN with fully connected CRF for accurate brain lesion segmentation , 2016, Medical Image Anal..

[95]  Yuanjie Zheng,et al.  Mammographic Mass Segmentation with Online Learned Shape and Appearance Priors , 2016, MICCAI.