Pre and Post-hoc Diagnosis and Interpretation of Malignancy from Breast DCE-MRI

We propose a new method for breast cancer screening from DCE-MRI based on a post-hoc approach that is trained using weakly annotated data (i.e., labels are available only at the image level without any lesion delineation). Our proposed post-hoc method automatically diagnosis the whole volume and, for positive cases, it localizes the malignant lesions that led to such diagnosis. Conversely, traditional approaches follow a pre-hoc approach that initially localises suspicious areas that are subsequently classified to establish the breast malignancy - this approach is trained using strongly annotated data (i.e., it needs a delineation and classification of all lesions in an image). We also aim to establish the advantages and disadvantages of both approaches when applied to breast screening from DCE-MRI. Relying on experiments on a breast DCE-MRI dataset that contains scans of 117 patients, our results show that the post-hoc method is more accurate for diagnosing the whole volume per patient, achieving an AUC of 0.91, while the pre-hoc method achieves an AUC of 0.81. However, the performance for localising the malignant lesions remains challenging for the post-hoc method due to the weakly labelled dataset employed during training.

[1]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[2]  Gustavo Carneiro,et al.  Deep Reinforcement Learning for Active Breast Lesion Detection from DCE-MRI , 2017, MICCAI.

[3]  Johannes Gehrke,et al.  Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission , 2015, KDD.

[4]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[5]  Nico Karssemeijer,et al.  Automated detection of breast cancer in false-negative screening MRI studies from women at increased risk. , 2016, European journal of radiology.

[6]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  J. Hanley,et al.  A method of comparing the areas under receiver operating characteristic curves derived from the same cases. , 1983, Radiology.

[8]  C. Douglas Maynard,et al.  ACR Appropriateness Criteria , 2000 .

[9]  Nico Karssemeijer,et al.  Fully automated detection of breast cancer in screening MRI using convolutional neural networks , 2018, Journal of medical imaging.

[10]  Nico Karssemeijer,et al.  Time to enhancement derived from ultrafast breast MRI as a novel parameter to discriminate benign from malignant breast lesions. , 2017, European journal of radiology.

[11]  Lina Arbash Meinel,et al.  Breast MRI lesion classification: Improved performance of human readers with a backpropagation neural network computer‐aided diagnosis (CAD) system , 2007, Journal of magnetic resonance imaging : JMRI.

[12]  Anne L. Martel,et al.  Effect of the enhancement threshold on the computer-aided detection of breast cancer using MRI. , 2009, Academic radiology.

[13]  Elsa D. Angelini,et al.  Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules , 2017, MICCAI.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Chuang Chen,et al.  Progress in the clinical detection of heterogeneity in breast cancer , 2016, Cancer medicine.

[16]  Guy Amit,et al.  Hybrid Mass Detection in Breast MRI Combining Unsupervised Saliency Analysis and Deep Learning , 2017, MICCAI.

[17]  Maryellen L. Giger,et al.  A Fuzzy C-Means (FCM)-Based Approach for Computerized Segmentation of Breast Lesions in Dynamic Contrast-Enhanced MR Images1 , 2006 .

[18]  C. Wood,et al.  Computer Aided Detection (CAD) for Breast MRI , 2005, Technology in cancer research & treatment.

[19]  Gerard J Fitzsimmons,et al.  From the Australian Institute of Health and Welfare , 2014, Communicable diseases intelligence quarterly report.

[20]  Anne L. Martel,et al.  Improving the Accuracy of Computer-aided Diagnosis for Breast MR Imaging by Differentiating between Mass and Nonmass Lesions. , 2016, Radiology.

[21]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[23]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[24]  H. D. de Koning,et al.  Efficacy of MRI and mammography for breast-cancer screening in women with a familial or genetic predisposition. , 2004, The New England journal of medicine.

[25]  N. Karssemeijer,et al.  A computer-aided diagnosis system for breast DCE-MRI at high spatiotemporal resolution. , 2015, Medical physics.

[26]  Bernd Hamm,et al.  Detection and classification of contrast‐enhancing masses by a fully automatic computer‐assisted diagnosis system for breast MRI , 2012, Journal of magnetic resonance imaging : JMRI.

[27]  Gustavo Carneiro,et al.  Lesion Saliency for Weakly Supervised Lesion Detection from Breast DCE-MRI , 2018 .

[28]  A. Jemal,et al.  Global cancer statistics , 2011, CA: a cancer journal for clinicians.

[29]  A. Jemal,et al.  Global cancer statistics, 2012 , 2015, CA: a cancer journal for clinicians.

[30]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  A. Jemal,et al.  Global Cancer Statistics , 2011 .

[32]  N. Dubrawsky Cancer statistics , 1989, CA: a cancer journal for clinicians.

[33]  John Schulman,et al.  Teacher–Student Curriculum Learning , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[35]  Evanthia Kousi,et al.  Quality assurance in MRI breast screening: comparing signal-to-noise ratio in dynamic contrast-enhanced imaging protocols , 2015, Physics in medicine and biology.

[36]  Bram Platel,et al.  Textural analysis of early‐phase spatiotemporal changes in contrast enhancement of breast lesions imaged with an ultrafast DCE‐MRI protocol , 2017, Medical physics.

[37]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[38]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[39]  Mohammad Teshnehlab,et al.  Breast cancer diagnosis in DCE-MRI using mixture ensemble of convolutional neural networks , 2017, Pattern Recognit..

[40]  Svetlana Lazebnik,et al.  Active Object Localization with Deep Reinforcement Learning , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[41]  Pantelis A Asvestas,et al.  Computer-based automated estimation of breast vascularity and correlation with breast cancer in DCE-MRI images. , 2017, Magnetic resonance imaging.

[42]  Nico Karssemeijer,et al.  Automated Characterization of Breast Lesions Imaged With an Ultrafast DCE-MR Protocol , 2014, IEEE Transactions on Medical Imaging.

[43]  M. Su,et al.  Evaluation of Clinical Breast MR Imaging Performed with Prototype Computer-aided Diagnosis Breast MR Imaging Workstation: Reader Study , 2012 .

[44]  Manuela Pereira,et al.  3D Lacunarity in Multifractal Analysis of Breast Tumor Lesions in Dynamic Contrast-Enhanced Magnetic Resonance Imaging , 2013, IEEE Transactions on Image Processing.

[45]  Darryl McClymont,et al.  Computer assisted detection and characterisation of breast cancer in MRI , 2015 .

[46]  Gustavo Carneiro,et al.  Globally optimal breast mass segmentation from DCE-MRI using deep semantic segmentation as shape prior , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[47]  Michael R Hamblin,et al.  CA : A Cancer Journal for Clinicians , 2011 .

[48]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[49]  Ronald M. Summers,et al.  ChestX-ray: Hospital-Scale Chest X-ray Database and Benchmarks on Weakly Supervised Classification and Localization of Common Thorax Diseases , 2019, Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics.

[50]  Sharon Steinberger,et al.  BI-RADS 3: Current and Future Use of Probably Benign , 2018, Current Radiology Reports.

[51]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[52]  Gustavo Carneiro,et al.  Automated Mass Detection in Mammograms Using Cascaded Deep Learning and Random Forests , 2015, 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[53]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[54]  Stuart Crozier,et al.  Fully automatic lesion segmentation in breast MRI using mean‐shift and graph‐cuts on a region adjacency graph , 2014, Journal of magnetic resonance imaging : JMRI.

[55]  Guy Amit,et al.  Classification of breast MRI lesions using small-size training sets: comparison of deep learning approaches , 2017, Medical Imaging.

[56]  István Csabai,et al.  Detecting and classifying lesions in mammograms with Deep Learning , 2017, Scientific Reports.

[57]  Yuanjie Zheng,et al.  Total variation based DCE‐MRI decomposition by separating lesion from background for time‐intensity curve estimation , 2017, Medical physics.

[58]  Elizabeth A Morris,et al.  Abbreviated protocol for breast MRI: are multiple sequences needed for cancer detection? , 2015, European journal of radiology.

[59]  David A Mankoff,et al.  ACR Appropriateness Criteria Breast Cancer Screening. , 2013, Journal of the American College of Radiology : JACR.

[60]  Wei Wei,et al.  Thoracic Disease Identification and Localization with Limited Supervision , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[61]  Jeffrey D Blume,et al.  Accuracy and interpretation time of computer-aided detection among novice and experienced breast MRI readers. , 2013, AJR. American journal of roentgenology.

[62]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  S. Fedewa,et al.  Cancer screening in the United States, 2017: A review of current American Cancer Society guidelines and current issues in cancer screening , 2017, CA: a cancer journal for clinicians.

[64]  Heinz-Otto Peitgen,et al.  Computer assistance for MR based diagnosis of breast cancer: Present and future challenges , 2007, Comput. Medical Imaging Graph..

[65]  A. Jemal,et al.  Cancer statistics, 2017 , 2017, CA: a cancer journal for clinicians.

[66]  Jacques Ferlay,et al.  International Variation in Female Breast Cancer Incidence and Mortality Rates , 2015, Cancer Epidemiology, Biomarkers & Prevention.

[67]  Lei Wang,et al.  A robust and extendable framework towards fully automated diagnosis of nonmass lesions in breast DCE-MRI , 2014, 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI).

[68]  Wufeng Xue,et al.  Full left ventricle quantification via deep multitask relationships learning , 2018, Medical Image Anal..

[69]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[70]  Sora C. Yoon,et al.  Interobserver Variability Between Breast Imagers Using the Fifth Edition of the BI-RADS MRI Lexicon. , 2015, AJR. American journal of roentgenology.

[71]  Gustavo Carneiro,et al.  Model Agnostic Saliency For Weakly Supervised Lesion Detection From Breast DCE-MRI , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[72]  Madeleine M A Tilanus-Linthorst,et al.  Influence of tumour stage at breast cancer detection on survival in modern times: population based study in 173 797 patients , 2015, BMJ : British Medical Journal.

[73]  Kilian Q. Weinberger,et al.  Deep Networks with Stochastic Depth , 2016, ECCV.

[74]  F J Gilbert,et al.  Personalised screening: is this the way forward? , 2017, Clinical radiology.

[75]  Nima Tajbakhsh,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE Transactions on Medical Imaging.

[76]  P. Prorok,et al.  Breast-Cancer Tumor Size, Overdiagnosis, and Mammography Screening Effectiveness. , 2016, The New England journal of medicine.

[77]  M. Giger,et al.  A fuzzy c-means (FCM)-based approach for computerized segmentation of breast lesions in dynamic contrast-enhanced MR images. , 2006, Academic radiology.

[78]  Gustavo Carneiro,et al.  Training Medical Image Analysis Systems like Radiologists , 2018, MICCAI.

[79]  Xiaogang Wang,et al.  Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection , 2017, MICCAI.

[80]  Nico Karssemeijer,et al.  Automated localization of breast cancer in DCE-MRI , 2015, Medical Image Anal..

[81]  Xin Yang,et al.  Joint Detection and Diagnosis of Prostate Cancer in Multi-parametric MRI Based on Multimodal Convolutional Neural Networks , 2017, MICCAI.

[82]  A. Madabhushi,et al.  Computerized image analysis for identifying triple-negative breast cancers and differentiating them from other molecular subtypes of breast cancer on dynamic contrast-enhanced MR images: a feasibility study. , 2014, Radiology.

[83]  Xiaohui Xie,et al.  Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification , 2016, bioRxiv.

[84]  Michael Brady,et al.  Analysis of dynamic MR breast images using a model of contrast enhancement , 1997, Medical Image Anal..

[85]  Alejandro F. Frangi,et al.  Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 , 2018, Lecture Notes in Computer Science.

[86]  Marleen de Bruijne,et al.  GP-Unet: Lesion Detection from Weak Labels with a 3D Regression Network , 2017, MICCAI.

[87]  N Karssemeijer,et al.  The frequency of missed breast cancers in women participating in a high-risk MRI screening program , 2018, Breast Cancer Research and Treatment.