Multifaceted fused-CNN based scoring of breast cancer whole-slide histopathology images

Abstract Automating the scoring of Whole-Slide Images (WSIs) is a challenging task because the search space for selecting region of interest (ROI) is huge due to the very large sizes of WSIs. A Multifaceted Fused-CNN (MF-CNN) and a Hybrid-Descriptor are proposed to develop an integrated scoring system for Breast Cancer histopathology WSIs. Suitable color and textural features are identified to help mitotic count based selection of ROIs at lower resolution. To recognize complex patterns, the MF-CNN considers multiple facets of the input image. It counts mitoses, extracts handcrafted features from ROIs and utilizes global texture of the images to form a Hybrid-Descriptor for training a classifier assigning scores to WSIs. The proposed system is evaluated on a publicly available benchmark (TUPAC16) and produced the highest score of 0.582 in terms of Cohen’s Kappa. It surpassed human experts’ level accuracy of ROI selection and can therefore reduce the burden of manual ROI selection for WSIs.

[1]  Anne E Carpenter,et al.  CellProfiler: image analysis software for identifying and quantifying cell phenotypes , 2006, Genome Biology.

[2]  Ognjen Arandjelovic,et al.  Using Machine Learning for Automatic Estimation of M. Smegmatis Cell Count from Fluorescence Microscopy Images , 2019, Precision Health and Medicine.

[3]  Asifullah Khan,et al.  A New Channel Boosted Convolution Neural Network using Transfer Learning , 2018, ArXiv.

[4]  Nico Karssemeijer,et al.  Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks , 2018, IEEE Transactions on Medical Imaging.

[5]  Anant Madabhushi,et al.  Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent , 2017, Scientific Reports.

[6]  Muhammad Hanif Durad,et al.  Intrusion detection using deep sparse auto-encoder and self-taught learning , 2019, Neural Computing and Applications.

[7]  Yukako Yagi,et al.  Whole slide imaging for educational purposes , 2012, Journal of pathology informatics.

[8]  Linda G. Shapiro,et al.  Detection and classification of cancer in whole slide breast histopathology images using deep convolutional networks , 2018, Pattern Recognit..

[9]  Arun Ross,et al.  On automated source selection for transfer learning in convolutional neural networks , 2018, Pattern Recognit..

[10]  Karl Rohr,et al.  Predicting breast tumor proliferation from whole‐slide images: The TUPAC16 challenge , 2018, Medical Image Anal..

[11]  Shuicheng Yan,et al.  Hybrid CNN and Dictionary-Based Models for Scene Recognition and Domain Adaptation , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Mats Andersson,et al.  Tumor proliferation assessment of whole slide images , 2018, Medical Imaging.

[13]  Luca Maria Gambardella,et al.  Assessment of algorithms for mitosis detection in breast cancer histopathology images , 2014, Medical Image Anal..

[14]  Karl Rohr,et al.  Automatic Grading of Breast Cancer Whole-Slide Histopathology Images , 2017, Bildverarbeitung für die Medizin.

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Minsoo Kim,et al.  A Unified Framework for Tumor Proliferation Score Prediction in Breast Histopathology , 2016, DLMIA/ML-CDS@MICCAI.

[17]  Shuicheng Yan,et al.  LG-CNN: From local parts to global discrimination for fine-grained recognition , 2017, Pattern Recognit..

[18]  Asifullah Khan,et al.  A survey of the recent architectures of deep convolutional neural networks , 2019, Artificial Intelligence Review.

[19]  Shuicheng Yan,et al.  SDE: A Novel Selective, Discriminative and Equalizing Feature Representation for Visual Recognition , 2017, International Journal of Computer Vision.

[20]  Anant Madabhushi,et al.  A Deep Convolutional Neural Network for segmenting and classifying epithelial and stromal regions in histopathological images , 2016, Neurocomputing.

[21]  Neofytos Dimitriou,et al.  A principled machine learning framework improves accuracy of stage II colorectal cancer prognosis , 2018, npj Digital Medicine.

[22]  Fabio A. González,et al.  Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks , 2014, Medical Imaging.

[23]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Asifullah Khan,et al.  Two-phase deep convolutional neural network for reducing class skewness in histopathological images based breast cancer detection , 2017, Comput. Biol. Medicine.

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  J. S. Marron,et al.  A method for normalizing histology slides for quantitative analysis , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[27]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Loris Nanni,et al.  Handcrafted vs. non-handcrafted features for computer vision classification , 2017, Pattern Recognit..

[29]  Chandan Chakraborty,et al.  Her2Net: A Deep Framework for Semantic Segmentation and Classification of Cell Membranes and Nuclei in Breast Cancer Evaluation , 2018, IEEE Transactions on Image Processing.

[30]  A. Fischer,et al.  Hematoxylin and eosin staining of tissue and cell sections. , 2008, CSH protocols.

[31]  Ognjen Arandjelovic,et al.  Colorectal Cancer Outcome Prediction from H&E Whole Slide Images using Machine Learning and Automatically Inferred Phenotype Profiles , 2019, ArXiv.

[32]  Shuicheng Yan,et al.  Task-Driven Feature Pooling for Image Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).