WeGleNet: A weakly-supervised convolutional neural network for the semantic segmentation of Gleason grades in prostate histology images

BACKGROUND AND OBJECTIVE Prostate cancer is one of the main diseases affecting men worldwide. The Gleason scoring system is the primary diagnostic tool for prostate cancer. This is obtained via the visual analysis of cancerous patterns in prostate biopsies performed by expert pathologists, and the aggregation of the main Gleason grades in a combined score. Computer-aided diagnosis systems allow to reduce the workload of pathologists and increase the objectivity. Nevertheless, those require a large number of labeled samples, with pixel-level annotations performed by expert pathologists, to be developed. Recently, efforts have been made in the literature to develop algorithms aiming the direct estimation of the global Gleason score at biopsy/core level with global labels. However, these algorithms do not cover the accurate localization of the Gleason patterns into the tissue. These location maps are the basis to provide a reliable computer-aided diagnosis system to the experts to be used in clinical practice by pathologists. In this work, we propose a deep-learning-based system able to detect local cancerous patterns in the prostate tissue using only the global-level Gleason score obtained from clinical records during training. METHODS The methodological core of this work is the proposed weakly-supervised-trained convolutional neural network, WeGleNet, based on a multi-class segmentation layer after the feature extraction module, a global-aggregation, and the slicing of the background class for the model loss estimation during training. RESULTS Using a public dataset of prostate tissue-micro arrays, we obtained a Cohen's quadratic kappa (κ) of 0.67 for the pixel-level prediction of cancerous patterns in the validation cohort. We compared the model performance for semantic segmentation of Gleason grades with supervised state-of-the-art architectures in the test cohort. We obtained a pixel-level κ of 0.61 and a macro-averaged f1-score of 0.58, at the same level as fully-supervised methods. Regarding the estimation of the core-level Gleason score, we obtained a κ of 0.76 and 0.67 between the model and two different pathologists. CONCLUSIONS WeGleNet is capable of performing the semantic segmentation of Gleason grades similarly to fully-supervised methods without requiring pixel-level annotations. Moreover, the model reached a performance at the same level as inter-pathologist agreement for the global Gleason scoring of the cores.

[1]  Konstantinos N. Plataniotis,et al.  A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains , 2019, International Journal of Computer Vision.

[2]  Daisuke Komura,et al.  Machine Learning Methods for Histopathological Image Analysis , 2017, Computational and structural biotechnology journal.

[3]  Thomas J. Fuchs,et al.  Clinical-grade computational pathology using weakly supervised deep learning on whole slide images , 2019, Nature Medicine.

[4]  Suha Kwak,et al.  Weakly Supervised Learning of Instance Segmentation With Inter-Pixel Relations , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Qitao Huang,et al.  Weakly Supervised Deep Learning for Whole Slide Lung Cancer Image Analysis , 2020, IEEE Transactions on Cybernetics.

[6]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[7]  Gang Wang,et al.  Automatic grading of prostate cancer in digitized histopathology images: Learning from multiple experts , 2018, Medical Image Anal..

[8]  D. Gleason,et al.  Histologic grading of prostate cancer: a perspective. , 1992, Human pathology.

[9]  Eric Granger,et al.  Multiple instance learning: A survey of problem characteristics and applications , 2016, Pattern Recognit..

[10]  Septimiu E. Salcudean,et al.  Comparison of Artificial Intelligence Techniques to Evaluate Performance of a Classifier for Automatic Grading of Prostate Cancer From Digitized Histopathologic Images , 2019, JAMA network open.

[11]  Vladlen Koltun,et al.  Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials , 2011, NIPS.

[12]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[13]  Ronan Collobert,et al.  From image-level to pixel-level labeling with Convolutional Networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Zhuowen Tu,et al.  Weakly supervised histopathology cancer image segmentation and classification , 2014, Medical Image Anal..

[15]  Konstantinos N. Plataniotis,et al.  HistoSegNet: Semantic Segmentation of Histological Tissue Type in Whole Slide Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  M. Rubin,et al.  Interobserver reproducibility of Gleason grading: evaluation using prostate cancer tissue microarrays , 2008, Journal of Cancer Research and Clinical Oncology.

[17]  Valery Naranjo,et al.  Gleason Grading of Histology Prostate Images Through Semantic Segmentation via Residual U-Net , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[18]  Mats Andersson,et al.  Convolutional neural networks for an automatic classification of prostate tissue slides with high-grade Gleason score , 2017, Medical Imaging.

[19]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[20]  B. van Ginneken,et al.  Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. , 2020, The Lancet. Oncology.

[21]  Henning Müller,et al.  Staining Invariant Features for Improving Generalization of Deep Convolutional Neural Networks in Computational Pathology , 2019, Front. Bioeng. Biotechnol..

[22]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Zhipeng Jia,et al.  Constrained Deep Weak Supervision for Histopathology Image Segmentation , 2017, IEEE Transactions on Medical Imaging.

[24]  Matthieu Cord,et al.  WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Sungroh Yoon,et al.  FickleNet: Weakly and Semi-Supervised Semantic Image Segmentation Using Stochastic Inference , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Yong Jae Lee,et al.  Hide-and-Seek: Forcing a Network to be Meticulous for Weakly-Supervised Object and Action Localization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  Eric W. Tramel,et al.  Classification and Disease Localization in Histopathology Using Only Global Labels: A Weakly-Supervised Approach , 2018, ArXiv.

[29]  Arkadiusz Gertych,et al.  An attention-based multi-resolution model for prostate whole slide imageclassification and localization , 2019, ArXiv.

[30]  Max Welling,et al.  Attention-based Deep Multiple Instance Learning , 2018, ICML.

[31]  Arkadiusz Gertych,et al.  An EM-based semi-supervised deep learning approach for semantic segmentation of histopathological images from radical prostatectomies , 2018, Comput. Medical Imaging Graph..

[32]  Manfred Claassen,et al.  Coupling weak and strong supervision for classification of prostate cancer histopathology images , 2018, ArXiv.

[33]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[34]  George Papandreou,et al.  Weakly-and Semi-Supervised Learning of a Deep Convolutional Network for Semantic Image Segmentation , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[35]  B. S. Manjunath,et al.  Weakly Supervised Localization Using Deep Feature Maps , 2016, ECCV.

[36]  Kimmo Kartasalo,et al.  Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. , 2020, The Lancet. Oncology.

[37]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[38]  T. Hermanns,et al.  Automated Gleason grading of prostate cancer tissue microarrays via deep learning , 2018, Scientific Reports.

[39]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).