Deep Multi-Magnification Networks for Multi-Class Breast Cancer Image Segmentation

Pathologic analysis of surgical excision specimens for breast carcinoma is important to evaluate the completeness of surgical excision and has implications for future treatment. This analysis is performed manually by pathologists reviewing histologic slides prepared from formalin-fixed tissue. In this paper, we present Deep Multi-Magnification Network trained by partial annotation for automated multi-class tissue segmentation by a set of patches from multiple magnifications in digitized whole slide images. Our proposed architecture with multi-encoder, multi-decoder, and multi-concatenation outperforms other single and multi-magnification-based architectures by achieving the highest mean intersection-over-union, and can be used to facilitate pathologists' assessments of breast cancer.

[1]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  I. Gage,et al.  Pathologic margin involvement and the risk of recurrence in patients treated with breast‐conserving therapy , 1996 .

[3]  Nasir M. Rajpoot,et al.  SAMS-NET: Stain-aware multi-scale network for instance-based nuclei segmentation in histology images , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[4]  Thomas J. Fuchs,et al.  Clinical-grade computational pathology using weakly supervised deep learning on whole slide images , 2019, Nature Medicine.

[5]  Nasir M. Rajpoot,et al.  Representation-Aggregation Networks for Segmentation of Multi-Gigapixel Histology Images , 2017, ArXiv.

[6]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[7]  Max A. Viergever,et al.  Breast Cancer Histopathology Image Analysis: A Review , 2014, IEEE Transactions on Biomedical Engineering.

[8]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.

[10]  A. Jemal,et al.  Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries , 2018, CA: a cancer journal for clinicians.

[11]  Edward J. Delp,et al.  Nuclei segmentation of fluorescence microscopy images using convolutional neural networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[12]  Shaoqun Zeng,et al.  From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge , 2019, IEEE Transactions on Medical Imaging.

[13]  Joel H. Saltz,et al.  Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  A. Jemal,et al.  Breast cancer statistics, 2017, racial disparity in mortality by state , 2017, CA: a cancer journal for clinicians.

[15]  David B. A. Epstein,et al.  MIMO-Net: A multi-input multi-output convolutional neural network for cell segmentation in fluorescence microscopy images , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[16]  Ismaël Koné,et al.  Hierarchical ResNeXt Models for Breast Cancer Histology Image Classification , 2018, ICIAR.

[17]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[18]  Constantine Katsinis,et al.  Large-scale computations on histology images reveal grade-differentiating parameters for breast cancer , 2006, BMC Medical Imaging.

[19]  Anant Madabhushi,et al.  An Integrated Region-, Boundary-, Shape-Based Active Contour for Multiple Object Overlap Resolution in Histological Imagery , 2012, IEEE Transactions on Medical Imaging.

[20]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[22]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[23]  Jeroen A. W. M. van der Laak,et al.  Learning from sparsely annotated data for semantic segmentation in histopathology images , 2018, MIDL.

[24]  Nico Karssemeijer,et al.  Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies , 2018, Modern Pathology.

[25]  Kevin Smith,et al.  Digital image analysis in breast pathology-from image processing techniques to artificial intelligence. , 2017, Translational research : the journal of laboratory and clinical medicine.

[26]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[27]  Anant Madabhushi,et al.  Automated gland and nuclei segmentation for grading of prostate and breast cancer histopathology , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[28]  Ahmedin Jemal,et al.  Breast cancer statistics, 2019 , 2019, CA: a cancer journal for clinicians.

[29]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[30]  Mats Andersson,et al.  Multi-Resolution Networks for Semantic Segmentation in Whole Slide Images , 2018, COMPAY/OMIA@MICCAI.

[31]  Dayong Wang,et al.  Deep Learning for Identifying Metastatic Breast Cancer , 2016, ArXiv.

[32]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[33]  Anant Madabhushi,et al.  Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent , 2017, Scientific Reports.

[34]  Louis B. Rall,et al.  Automatic differentiation , 1981 .

[35]  Atsuto Maki,et al.  A systematic study of the class imbalance problem in convolutional neural networks , 2017, Neural Networks.

[36]  Maximilian Baust,et al.  Assessment of Breast Cancer Histology using Densely Connected Convolutional Networks , 2018, ICIAR.

[37]  Linda G. Shapiro,et al.  Learning to Segment Breast Biopsy Whole Slide Images , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[38]  Anindya Sarkar,et al.  Structure and Context in Prostatic Gland Segmentation and Classification , 2012, MICCAI.

[39]  Kyunghyun Paeng,et al.  A Robust and Effective Approach Towards Accurate Metastasis Detection and pN-stage Classification in Breast Cancer , 2018, MICCAI.

[40]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[41]  Nasir M. Rajpoot,et al.  Context-Aware Convolutional Neural Network for Grading of Colorectal Cancer Histology Images , 2019, IEEE Transactions on Medical Imaging.

[42]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[43]  Ryoma Bise,et al.  Adaptive Weighting Multi-Field-Of-View CNN for Semantic Segmentation in Pathology , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Joachim M. Buhmann,et al.  Computational Pathology: Challenges and Promises for Tissue Analysis , 2015, Comput. Medical Imaging Graph..

[45]  Edi Brogi,et al.  Impact of Margin Assessment Method on Positive Margin Rate and Total Volume Excised , 2013, Annals of Surgical Oncology.