A deep learning approach for the analysis of masses in mammograms with minimal user intervention

HighlightsWe introduce a novel automated CAD system with minimal user intervention that can detect, segment and classify breast masses from mammograms. We explore deep learning and structured output models for the design and development of the proposed CAD system. More specifically for the detection, we propose a cascade of deep learning methods to select hypotheses that are refined based on Bayesian optimization. For the segmentation, we propose the use of deep structured output learning that is subsequently refined by a level set method. Finally, for the classification, we propose a deep learning classifier that is pre‐trained with a regression to hand‐crafted feature values and fine‐tuned based on the annotations of the breast mass classification dataset. Our proposed CAD system produces the current state‐of‐the‐art detection, segmentation and classification results for the INbreast dataset. Graphical abstract Figure. No Caption available. Abstract We present an integrated methodology for detecting, segmenting and classifying breast masses from mammograms with minimal user intervention. This is a long standing problem due to low signal‐to‐noise ratio in the visualisation of breast masses, combined with their large variability in terms of shape, size, appearance and location. We break the problem down into three stages: mass detection, mass segmentation, and mass classification. For the detection, we propose a cascade of deep learning methods to select hypotheses that are refined based on Bayesian optimisation. For the segmentation, we propose the use of deep structured output learning that is subsequently refined by a level set method. Finally, for the classification, we propose the use of a deep learning classifier, which is pre‐trained with a regression to hand‐crafted feature values and fine‐tuned based on the annotations of the breast mass classification dataset. We test our proposed system on the publicly available INbreast dataset and compare the results with the current state‐of‐the‐art methodologies. This evaluation shows that our system detects 90% of masses at 1 false positive per image, has a segmentation accuracy of around 0.85 (Dice index) on the correctly detected masses, and overall classifies masses as malignant or benign with sensitivity (Se) of 0.98 and specificity (Sp) of 0.7.

[1]  Gustavo Carneiro,et al.  Automated Detection of Individual Micro-calcifications from Mammograms using a Multi-stage Cascade Approach , 2016, ArXiv.

[2]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[3]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[4]  Lubomir M. Hadjiiski,et al.  Characterization of mammographic masses based on level set segmentation with new image features and patient information. , 2007, Medical physics.

[5]  Gustavo Carneiro,et al.  Tree RE-weighted belief propagation using deep learning potentials for mass segmentation from mammograms , 2015, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[6]  Asoke K. Nandi,et al.  Toward breast cancer diagnosis based on automated segmentation of masses in mammograms , 2009, Pattern Recognit..

[7]  Ling Guan,et al.  A CAD System for the Automatic Detection of Clustered Microcalcification in Digitized Mammogram Films , 2000, IEEE Trans. Medical Imaging.

[8]  N. Karssemeijer,et al.  A new 2D segmentation method based on dynamic programming applied to computer aided detection in mammography. , 2004, Medical physics.

[9]  Gustavo Carneiro,et al.  Deep structured learning for mass segmentation from mammograms , 2014, 2015 IEEE International Conference on Image Processing (ICIP).

[10]  M L Giger,et al.  Computerized analysis of breast lesions in three dimensions using dynamic magnetic-resonance imaging. , 1998, Medical physics.

[11]  Maryellen L. Giger.  Medical imaging and computers in the diagnosis of breast cancer , 2014, Optics & Photonics - Optical Engineering + Applications.

[12]  Lubomir M. Hadjiiski,et al.  Improvement of mammographic mass characterization using spiculation meausures and morphological features. , 2001, Medical physics.

[13]  Gustavo Carneiro,et al.  Automated Mass Detection in Mammograms Using Cascaded Deep Learning and Random Forests , 2015, 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[14]  James A. Sethian,et al.  Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid , 2012 .

[15]  Berkman Sahiner,et al.  Computer-aided detection of breast masses on full field digital mammograms. , 2005, Medical physics.

[16]  C. Balleyguier,et al.  Computed-aided diagnosis (CAD) in the detection of breast cancer. , 2013, European journal of radiology.

[17]  Jaime S. Cardoso,et al.  Closed Shortest Path in the Original Coordinates with an Application to Breast Cancer , 2015, Int. J. Pattern Recognit. Artif. Intell..

[18]  Qiang Li,et al.  Breast mass segmentation in mammography using plane fitting and dynamic programming. , 2009, Academic radiology.

[19]  Pascal Fua,et al.  Refining Mitochondria Segmentation in Electron Microscopy Imagery with Active Surfaces , 2014, ECCV Workshops.

[20]  Behrouz Minaei,et al.  Assessment of a novel mass detection algorithm in mammograms. , 2013, Journal of cancer research and therapeutics.

[21]  Gustavo Carneiro,et al.  Lung segmentation in chest radiographs using distance regularized level set and deep-structured learning and inference , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[22]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[23]  S Tangaro,et al.  A completely automated CAD system for mass detection in a large mammographic database. , 2006, Medical physics.

[24]  Gustavo Carneiro,et al.  Unregistered Multiview Mammogram Analysis with Pre-trained Deep Learning Models , 2015, MICCAI.

[25]  Sebastian Nowozin,et al.  Structured Learning and Prediction in Computer Vision , 2011, Found. Trends Comput. Graph. Vis..

[26]  A. Jemal,et al.  Cancer Statistics, 2008 , 2008, CA: a cancer journal for clinicians.

[27]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[29]  J. Elmore,et al.  Variability in interpretive performance at screening mammography and radiologists' characteristics associated with accuracy. , 2009, Radiology.

[30]  L.M. Bruce,et al.  Digital Mammographic Computer Aided Diagnosis (CAD) using Adaptive Level Set Segmentation , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[31]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[32]  Gustavo Carneiro,et al.  The Automated Learning of Deep Features for Breast Mass Classification from Mammograms , 2016, MICCAI.

[33]  Hong Liu,et al.  A novel segmentation method for convex lesions based on dynamic programming with local intra-class variance , 2012, SAC '12.

[34]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[35]  Amir Globerson,et al.  Convergent message passing algorithms - a unifying view , 2009, UAI.

[36]  N Karssemeijer,et al.  Use of border information in the classification of mammographic masses , 2006, Physics in medicine and biology.

[37]  Rainer Stotzka,et al.  An Example-Based System to Support the Segmentation of Stellate Lesions , 2005, Bildverarbeitung für die Medizin.

[38]  Justin Domke,et al.  Learning Graphical Model Parameters with Approximate Marginal Inference , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  C. D'Orsi,et al.  Influence of computer-aided detection on performance of screening mammography. , 2007, The New England journal of medicine.

[40]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[41]  Arnau Oliver,et al.  A review of automatic mass detection and segmentation in mammographic images , 2010, Medical Image Anal..

[42]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[43]  M. Masotti,et al.  A novel featureless approach to mass detection in digital mammograms based on support vector machines. , 2004, Physics in medicine and biology.

[44]  Martin J. Wainwright,et al.  Tree-reweighted belief propagation algorithms and approximate ML estimation by pseudo-moment matching , 2003, AISTATS.

[45]  Mia K Markey,et al.  A model-based framework for the detection of spiculated masses on mammography. , 2008, Medical physics.

[46]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[47]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[48]  Ghassan Hamarneh,et al.  Mammography Segmentation with Maximum Likelihood Active Contours , 2022 .

[49]  Yongyi Yang,et al.  Computer-Aided Detection and Diagnosis of Breast Cancer With Mammography: Recent Advances , 2009, IEEE Transactions on Information Technology in Biomedicine.

[50]  Neil A. Dodgson,et al.  Proceedings Ninth IEEE International Conference on Computer Vision , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[51]  Gustavo Carneiro,et al.  Fully Automated Non-rigid Segmentation with Distance Regularized Level Set Evolution Initialized and Constrained by Deep-Structured Inference , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[54]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[55]  N. Karssemeijer,et al.  An automatic method to discriminate malignant masses from normal tissue in digital mammograms1 , 2000, Physics in medicine and biology.

[56]  Jaime S. Cardoso,et al.  INbreast: toward a full-field digital mammographic database. , 2012, Academic radiology.

[57]  Georgia D. Tourassi,et al.  A Concentric Morphology Model for the Detection of Masses in Mammography , 2007, IEEE Transactions on Medical Imaging.

[58]  Gustavo Carneiro,et al.  Deep Learning and Structured Prediction for the Segmentation of Mass in Mammograms , 2015, MICCAI.

[59]  Yuting Zhang,et al.  Improving object detection with deep convolutional networks via Bayesian optimization and structured prediction , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Max A. Viergever,et al.  Computer-aided diagnosis in chest radiography: a survey , 2001, IEEE Transactions on Medical Imaging.