A deep learning approach for mitosis detection: Application in tumor proliferation prediction from whole slide images

The tumor proliferation, which is correlated with tumor grade, is a crucial biomarker indicative of breast cancer patients' prognosis. The most commonly used method in predicting tumor proliferation speed is the counting of mitotic figures in Hematoxylin and Eosin (H&E) histological slides. Manual mitosis counting is known to suffer from reproducibility problems. This paper presents a fully automated system for tumor proliferation prediction from whole slide images via mitosis counting. First, by considering the epithelial tissue as mitosis activity regions, we build a deep-learning-based region of interest detection method to select the high mitosis activity regions from whole slide images. Second, we learned a set of deep neural networks to detect mitosis detection from selected areas. The proposed mitosis detection system is designed to effectively overcome the mitosis detection challenges by two novel deep preprocessing and two-step hard negative mining approaches. Third, we trained a Support Vector Machine (SVM) classifier to predict the final tumor proliferation score. The proposed method was evaluated on the dataset of the Tumor Proliferation Assessment Challenge (TUPAC16) and achieved a 73.81 % F-measure and 0.612 weighted kappa score, respectively, outperforming all previous approaches significantly. Experimental results demonstrate that the proposed system considerably improves the tumor proliferation prediction accuracy and provides a reliable automated tool to support health care make-decisions.

[1]  I. Ellis,et al.  Pathological prognostic factors in breast cancer. I. The value of histological grade in breast cancer: experience from a large study with long-term follow-up. , 2002, Histopathology.

[2]  Maria S. Kulikova,et al.  Mitosis detection in breast cancer histological images An ICPR 2012 contest , 2013, Journal of pathology informatics.

[3]  Nilanjan Ray,et al.  Training Convolutional Neural Networks and Compressed Sensing End-to-End for Microscopy Cell Detection , 2018, IEEE Transactions on Medical Imaging.

[4]  Matheus Palhares Viana,et al.  Wide residual networks for mitosis detection , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[5]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Max A. Viergever,et al.  Breast Cancer Histopathology Image Analysis: A Review , 2014, IEEE Transactions on Biomedical Engineering.

[7]  Mohammad Sadegh Helfroush,et al.  Maximized Inter-Class Weighted Mean for Fast and Accurate Mitosis Cells Detection in Breast Cancer Histopathology Images , 2017, Journal of Medical Systems.

[8]  Wenyu Liu,et al.  DeepMitosis: Mitosis detection via deep detection, verification and segmentation networks , 2018, Medical Image Anal..

[9]  Ognjen Arandjelovic,et al.  Identification of Immunological Features Enables Survival Prediction of Muscle-Invasive Bladder Cancer Patients Using Machine Learning , 2020, bioRxiv.

[10]  Orcun Goksel,et al.  A deep learning framework for context-aware mitotic activity estimation in whole slide images , 2019, Medical Imaging.

[11]  Gilles Louppe,et al.  Collaborative analysis of multi-gigapixel imaging data using Cytomine , 2016, Bioinform..

[12]  Daniel Racoceanu,et al.  Towards semantic-driven high-content image analysis: An operational instantiation for mitosis detection in digital histopathology , 2015, Comput. Medical Imaging Graph..

[13]  Karl Rohr,et al.  Predicting breast tumor proliferation from whole‐slide images: The TUPAC16 challenge , 2018, Medical Image Anal..

[14]  Siddhant Rao,et al.  MITOS-RCNN: A Novel Approach to Mitotic Figure Detection in Breast Cancer Histopathology Images using Region Based Convolutional Neural Networks , 2018, ArXiv.

[15]  Minsoo Kim,et al.  A Unified Framework for Tumor Proliferation Score Prediction in Breast Histopathology , 2016, DLMIA/ML-CDS@MICCAI.

[16]  Francesco Ciompi,et al.  Deep learning assisted mitotic counting for breast cancer , 2019, Laboratory Investigation.

[17]  Bo Wang,et al.  Weakly supervised mitosis detection in breast histopathology images using concentric loss , 2019, Medical Image Anal..

[18]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[19]  Dipti Prasad Mukherjee,et al.  Mitosis Detection for Invasive Breast Cancer Grading in Histopathological Images , 2015, IEEE Transactions on Image Processing.

[20]  Luca Maria Gambardella,et al.  Assessment of algorithms for mitosis detection in breast cancer histopathology images , 2014, Medical Image Anal..

[21]  Manuel Stritt,et al.  Orbit Image Analysis: An open-source whole slide image analysis tool , 2020, PLoS computational biology.

[22]  Shadi Albarqouni,et al.  AggNet : Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images , 2016 .

[23]  Mohammad Sadegh Helfroush,et al.  Intelligent CAD System for Automatic Detection of Mitotic Cells from Breast Cancer Histology Slide Images Based on Teaching-Learning-Based Optimization , 2014 .

[24]  H. Bloom,et al.  Histological Grading and Prognosis in Breast Cancer , 1957, British Journal of Cancer.

[25]  Nasir M. Rajpoot,et al.  Leveraging Unlabeled Whole-Slide-Images for Mitosis Detection , 2018, COMPAY/OMIA@MICCAI.

[26]  Santiago López-Tapia,et al.  A Fast Pyramidal Bayesian Model for Mitosis Detection in Whole-Slide Images , 2019, ECDP.

[27]  Ognjen Arandjelovic,et al.  Colorectal Cancer Outcome Prediction from H&E Whole Slide Images using Machine Learning and Automatically Inferred Phenotype Profiles , 2019, ArXiv.

[28]  H. Danyali,et al.  Automatic detection of breast cancer mitotic cells based on the combination of textural, statistical and innovative mathematical features , 2015 .

[29]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[30]  A. Camargo,et al.  Epithelial cells captured from ductal carcinoma in situ reveal a gene expression signature associated with progression to invasive breast cancer , 2016, Oncotarget.

[31]  Peter Bankhead,et al.  QuPath: Open source software for digital pathology image analysis , 2017, Scientific Reports.

[32]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[33]  Aboul Ella Hassanien,et al.  Moth-flame swarm optimization with neutrosophic sets for automatic mitosis detection in breast cancer histology images , 2017, Applied Intelligence.

[34]  Anne E Carpenter,et al.  Improved structure, function and compatibility for CellProfiler: modular high-throughput image analysis software , 2011, Bioinform..

[35]  Nico Karssemeijer,et al.  Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks , 2018, IEEE Transactions on Medical Imaging.

[36]  Nasir M. Rajpoot,et al.  A Nonlinear Mapping Approach to Stain Normalization in Digital Histopathology Images Using Image-Specific Color Deconvolution , 2014, IEEE Transactions on Biomedical Engineering.

[37]  Chandan Chakraborty,et al.  Efficient deep learning model for mitosis detection using breast histopathology images , 2017, Comput. Medical Imaging Graph..

[38]  Mats Andersson,et al.  Tumor proliferation assessment of whole slide images , 2018, Medical Imaging.

[39]  J. S. Marron,et al.  A method for normalizing histology slides for quantitative analysis , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[40]  Neofytos Dimitriou,et al.  A principled machine learning framework improves accuracy of stage II colorectal cancer prognosis , 2018, npj Digital Medicine.

[41]  Neofytos Dimitriou,et al.  Deep Learning for Whole Slide Image Analysis: An Overview , 2019, Front. Med..