Digital breast tomosynthesis versus digital mammography: integration of image modalities enhances deep learning-based breast mass classification

Objective To evaluate the impact of utilizing digital breast tomosynthesis (DBT) or/and full-field digital mammography (FFDM), and different transfer learning strategies on deep convolutional neural network (DCNN)-based mass classification for breast cancer. Methods We retrospectively collected 441 patients with both DBT and FFDM on which regions of interest (ROIs) covering the malignant, benign and normal tissues were extracted for DCNN training and validation. Experiments were conducted for tasks in distinguishing malignant/benign/normal: (1) classification capabilities of DBT vs FFDM and the role of transfer learning were validated on 2D-DCNN; (2) different strategies of combining DBT and FFDM and the associated impacts on classification were explored; (3) 2D-DCNN and 3D-DCNN trained from scratch with volumetric DBT were compared. Results 2D-DCNN with transfer learning outperformed that without for DBT in distinguishing malignant (ΔAUC = 0.059 ± 0.009, p  < 0.001), benign (ΔAUC = 0.095 ± 0.010, p  < 0.001) and normal tissue (ΔAUC = 0.042 ± 0.004, p  < 0.001) (paired samples t  test). 2D-DCNN trained on DBT (with transfer learning) achieved higher accuracy than those on FFDM (malignant: ΔAUC = 0.014 ± 0.014, p  = 0.037; benign: ΔAUC = 0.031 ± 0.006, p  < 0.001; normal: ΔAUC = 0.017 ± 0.004, p  < 0.001) (independent samples t  test). The 2D-DCNN employing both DBT and FFDM for training achieved better performances in benign (FFDM: ΔAUC = 0.010 ± 0.008, p <  0.001; DBT: ΔAUC = 0.009 ± 0.005, p  < 0.001) and normal (FFDM: ΔAUC = 0.005 ± 0.003, p  < 0.001; DBT: ΔAUC = 0.002 ± 0.002, p  < 0.001) (related samples Friedman test). The 3D-DCNN and 2D-DCNN trained from scratch with DBT only produced moderate classification. Conclusions Transfer learning facilitates mass classification for both DBT and FFDM, and DBT outperforms FFDM when equipped with transfer learning. Integrating DBT and FFDM in DCNN training enhances mass classification accuracy for breast cancer. Key Points • Transfer learning facilitates mass classification for both DBT and FFDM, and the DBT-based DCNN outperforms the FFDM-based DCNN when equipped with transfer learning. • Integrating DBT and FFDM in DCNN training enhances breast mass classification accuracy. • 3D-DCNN/2D-DCNN trained from scratch with volumetric DBT but without transfer learning only produce moderate mass classification result.

[1]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[2]  Susanna M Cramb,et al.  The descriptive epidemiology of female breast cancer: an international comparison of screening, incidence, survival and mortality. , 2012, Cancer epidemiology.

[3]  Hui Li,et al.  Transfer Learning From Convolutional Neural Networks for Computer-Aided Diagnosis: A Comparison of Digital Breast Tomosynthesis and Full-Field Digital Mammography. , 2019, Academic radiology.

[4]  P. Brennan,et al.  Can digital breast tomosynthesis perform better than standard digital mammography work-up in breast cancer assessment clinic? , 2018, European Radiology.

[5]  Karla Kerlikowske,et al.  National Performance Benchmarks for Modern Diagnostic Digital Mammography: Update from the Breast Cancer Surveillance Consortium. , 2017, Radiology.

[6]  Yong Man Ro,et al.  Feature extraction from bilateral dissimilarity in digital breast tomosynthesis reconstructed volume , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[7]  A. Jemal,et al.  Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries , 2018, CA: a cancer journal for clinicians.

[8]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  Milan Sonka,et al.  "Handbook of Medical Imaging, Volume 2. Medical Image Processing and Analysis " , 2000 .

[10]  Lubomir M. Hadjiiski,et al.  Mass detection in digital breast tomosynthesis: Deep convolutional neural network with transfer learning from mammography. , 2016, Medical physics.

[11]  Yong Man Ro,et al.  Improving mass detection using combined feature representations from projection views and reconstructed volume of DBT and boosting based classification with feature selection. , 2015, Physics in medicine and biology.

[12]  G. Baird,et al.  Changes in recall type and patient treatment following implementation of screening digital breast tomosynthesis. , 2015, Radiology.

[13]  Berkman Sahiner,et al.  Characterization of masses in digital breast tomosynthesis: comparison of machine learning in projection views and reconstructed slices. , 2010, Medical physics.

[14]  Berkman Sahiner,et al.  Computer-aided detection system for breast masses on digital tomosynthesis mammograms: preliminary experience. , 2005, Radiology.

[15]  Andriy I. Bandos,et al.  Comparison of digital mammography alone and digital mammography plus tomosynthesis in a population-based screening program. , 2013, Radiology.

[16]  S. Ciatto,et al.  Integration of 3D digital mammography with tomosynthesis for population breast-cancer screening (STORM): a prospective comparison study. , 2013, The Lancet. Oncology.

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Alberto Tagliafico,et al.  Digital breast tomosynthesis: A practical approach , 2016 .

[19]  L. Brinton,et al.  Global trends in breast cancer incidence and mortality 1973-1997. , 2005, International journal of epidemiology.

[20]  Pragya A. Dang,et al.  Addition of tomosynthesis to conventional digital mammography: effect on image interpretation time of screening examinations. , 2014, Radiology.

[21]  Marcus A. Badgeley,et al.  Confounding variables can degrade generalization performance of radiological deep learning models , 2018, ArXiv.

[22]  D. Berry,et al.  Effect of screening and adjuvant therapy on mortality from breast cancer. , 2006, The New England journal of medicine.

[23]  J. Bian,et al.  Comparison of reconstruction algorithms for digital breast tomosynthesis , 2009, 0908.2610.

[24]  Yin Yin,et al.  Detection of soft tissue densities from digital breast tomosynthesis: comparison of conventional and deep learning approaches , 2016, SPIE Medical Imaging.

[25]  X. Zou.  Epidemic trend, screening, and early detection and treatment of cancer in Chinese population , 2017, Cancer biology & medicine.

[26]  D. Berry,et al.  Effect of screening and adjuvant therapy on mortality from breast cancer , 2005 .

[27]  Isabelle Bloch,et al.  Detection of masses and architectural distortions in digital breast tomosynthesis images using fuzzy and a contrario approaches , 2014, Pattern Recognit..

[28]  Daniel F Heitjan,et al.  Screening outcomes following implementation of digital breast tomosynthesis in a general-population screening program. , 2014, Journal of the National Cancer Institute.

[29]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[30]  SchmidhuberJürgen.  Deep learning in neural networks , 2015 .

[31]  Emily F Conant,et al.  Breast cancer screening using tomosynthesis in combination with digital mammography. , 2014, JAMA.

[32]  Lubomir M. Hadjiiski,et al.  Evolutionary pruning of transfer learned deep convolutional neural network for breast cancer diagnosis in digital breast tomosynthesis , 2018, Physics in medicine and biology.

[33]  The positive predictive value for diagnosis of breast cancer full-field digital mammography versus film-screen mammography in the diagnostic mammographic population. , 2006, Academic radiology.

[34]  L. Liberman,et al.  Breast imaging reporting and data system (BI-RADS). , 2002, Radiologic clinics of North America.

[35]  Oguzhan Alagoz,et al.  Effects of screening and systemic adjuvant therapy on ER-specific US breast cancer mortality. , 2014, Journal of the National Cancer Institute.

[36]  Tao Wu,et al.  A comparison of reconstruction algorithms for breast tomosynthesis. , 2004, Medical physics.

[37]  L. Liberman,et al.  The breast imaging reporting and data system: positive predictive value of mammographic features and final assessment categories. , 1998, AJR. American journal of roentgenology.

[38]  Marcus A. Badgeley,et al.  Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study , 2018, PLoS medicine.

[39]  Andriy I. Bandos,et al.  Prospective trial comparing full-field digital mammography (FFDM) versus combined FFDM and tomosynthesis in a population-based screening programme using independent double reading with arbitration , 2013, European Radiology.

[40]  Anders Tingberg,et al.  Performance of one-view breast tomosynthesis as a stand-alone breast cancer screening modality: results from the Malmö Breast Tomosynthesis Screening Trial, a population-based study , 2015, European Radiology.

[41]  Yong Man Ro,et al.  Breast mass detection using slice conspicuity in 3D reconstructed digital breast volumes , 2014, Physics in medicine and biology.

[42]  Pedagógia,et al.  Cross Sectional Study , 2019 .

[43]  Yong Man Ro,et al.  Latent feature representation with 3-D multi-view deep convolutional neural network for bilateral analysis in digital breast tomosynthesis , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[44]  D. Kopans,et al.  Digital tomosynthesis in breast imaging. , 1997, Radiology.

[45]  C. Lehman,et al.  National Performance Benchmarks for Modern Screening Digital Mammography: Update from the Breast Cancer Surveillance Consortium. , 2017, Radiology.

[46]  Lubomir M. Hadjiiski,et al.  Breast Cancer Diagnosis in Digital Breast Tomosynthesis: Effects of Training Sample Size on Multi-Stage Transfer Learning Using Deep Neural Nets , 2019, IEEE Transactions on Medical Imaging.

[47]  Berkman Sahiner,et al.  Computer-aided detection of breast masses in digital breast tomosynthesis (DBT): improvement of false positive reduction by optimization of object segmentation , 2011, Medical Imaging.

[48]  Yong Man Ro,et al.  Latent feature representation with depth directional long-term recurrent learning for breast masses in digital breast tomosynthesis , 2017, Physics in medicine and biology.

[49]  Madhavi Raghu,et al.  Early clinical experience with digital breast tomosynthesis for screening mammography. , 2015, Radiology.

[50]  Luis Perez,et al.  The Effectiveness of Data Augmentation in Image Classification using Deep Learning , 2017, ArXiv.

[51]  N Houssami,et al.  Application of breast tomosynthesis in screening: incremental effect on mammography acquisition and reading time. , 2012, The British journal of radiology.

[52]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[53]  Madhavi Raghu,et al.  Comparison of tomosynthesis plus digital mammography and digital mammography alone for breast cancer screening. , 2013, Radiology.