Bayesian Collaborative Learning for Whole-Slide Image Classification

Whole-slide image (WSI) classification is fundamental to computational pathology, which is challenging in extra-high resolution, expensive manual annotation, data heterogeneity, etc. Multiple instance learning (MIL) provides a promising way towards WSI classification, which nevertheless suffers from the memory bottleneck issue inherently, due to the gigapixel high resolution. To avoid this issue, the overwhelming majority of existing approaches have to decouple the feature encoder and the MIL aggregator in MIL networks, which may largely degrade the performance. Towards this end, this paper presents a Bayesian Collaborative Learning (BCL) framework to address the memory bottleneck issue with WSI classification. Our basic idea is to introduce an auxiliary patch classifier to interact with the target MIL classifier to be learned, so that the feature encoder and the MIL aggregator in the MIL classifier can be learned collaboratively while preventing the memory bottleneck issue. Such a collaborative learning procedure is formulated under a unified Bayesian probabilistic framework and a principled Expectation-Maximization algorithm is developed to infer the optimal model parameters iteratively. As an implementation of the E-step, an effective quality-aware pseudo labeling strategy is also suggested. The proposed BCL is extensively evaluated on three publicly available WSI datasets, i.e., CAMELYON16, TCGA-NSCLC and TCGA-RCC, achieving an AUC of 95.6%, 96.0% and 97.5% respectively, which consistently outperforms all the methods compared. Comprehensive analysis and discussion will also be presented for in-depth understanding of the method. To promote future work, our source code is released at: https://github.com/Zero-We/BCL.

[1]  Manning Wang,et al.  Bi-directional Weakly Supervised Knowledge Distillation for Whole Slide Image Classification , 2022, NeurIPS.

[2]  R. G. Krishnan,et al.  Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Junzhou Huang,et al.  Node-aligned Graph Convolutional Network for Whole-slide Image Representation and Classification , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Margrit Betke,et al.  A Graph-Transformer for Whole Slide Image Classification , 2022, IEEE Transactions on Medical Imaging.

[5]  Lequan Yu,et al.  Lymph Node Metastasis Prediction From Whole Slide Images With Transformer-Guided Multiinstance Learning and Knowledge Transfer , 2022, IEEE Transactions on Medical Imaging.

[6]  Linda G. Shapiro,et al.  End-to-End diagnosis of breast biopsy images with transformers , 2022, Medical Image Anal..

[7]  Mohammed Bennamoun,et al.  Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Ben Glocker,et al.  Multiple Instance Learning with Auxiliary Task Weighting for Multiple Myeloma Classification , 2021, MICCAI.

[9]  Dimitris N. Metaxas,et al.  Hybrid Supervision Learning for Pathology Whole Slide Image Classification , 2021, MICCAI.

[10]  Lars Petersson,et al.  A Survey on Graph-Based Deep Learning for Computational Histopathology , 2021, Comput. Medical Imaging Graph..

[11]  Xiangyang Ji,et al.  TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classication , 2021, NeurIPS.

[12]  Lubaina Ehsan,et al.  Cluster-to-Conquer: A Framework for End-to-End Multi-Instance Learning for Whole Slide Image Classification , 2021, MIDL.

[13]  M. Hsiao,et al.  An annotation-free whole-slide training approach to pathological classification of lung cancer types using deep learning , 2021, Nature Communications.

[14]  K. Eliceiri,et al.  Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Liang Sun,et al.  Multi-task multi-modal learning for joint diagnosis and prognosis of human cancers , 2020, Medical Image Anal..

[16]  Jitendra Jonnagaddala,et al.  Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks , 2020, Medical Image Anal..

[17]  Jeroen van der Laak,et al.  Detection of Prostate Cancer in Whole-Slide Images Through End-to-End Training With Image-Level Labels , 2020, IEEE Transactions on Medical Imaging.

[18]  Fan Yang,et al.  Predicting Lymph Node Metastasis Using Histopathological Images Based on Multiple Instance Learning With Deep Graph Convolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Xiaolin Hu,et al.  Online Knowledge Distillation via Collaborative Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Ming Y. Lu,et al.  Data-efficient and weakly supervised computational pathology on whole-slide images , 2020, Nature Biomedical Engineering.

[21]  I. Takeuchi,et al.  Multi-scale Domain-adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Anne L. Martel,et al.  Deep neural network models for computational histopathology: A survey , 2019, Medical Image Anal..

[23]  Ming Y. Lu,et al.  Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosis , 2019, IEEE Transactions on Medical Imaging.

[24]  B. van Ginneken,et al.  Streaming Convolutional Neural Networks for End-to-End Learning With Multi-Megapixel Images , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Tatsuya Harada,et al.  Multi-Stage Pathological Image Classification Using Semantic Segmentation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  Andrew H. Beck,et al.  Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the Digital Pathology Association , 2019, The Journal of pathology.

[27]  Nasir M. Rajpoot,et al.  Context-Aware Convolutional Neural Network for Grading of Colorectal Cancer Histology Images , 2019, IEEE Transactions on Medical Imaging.

[28]  Thomas J. Fuchs,et al.  Clinical-grade computational pathology using weakly supervised deep learning on whole slide images , 2019, Nature Medicine.

[29]  Hai Su,et al.  Pathologist-level interpretable whole-slide cancer diagnosis with deep learning , 2019, Nat. Mach. Intell..

[30]  Zizhao Zhang,et al.  Pathologist-level interpretable whole-slide cancer diagnosis with deep learning , 2019, Nature Machine Intelligence.

[31]  Francesco Ciompi,et al.  Neural Image Compression for Gigapixel Histopathology Image Analysis , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Guocong Song,et al.  Collaborative Learning for Deep Neural Networks , 2018, NeurIPS.

[33]  Max Welling,et al.  Attention-based Deep Multiple Instance Learning , 2018, ICML.

[34]  Rui Zhang,et al.  Collaborative Learning for Weakly Supervised Object Detection , 2018, IJCAI.

[35]  Andrew H. Beck,et al.  Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer , 2017, JAMA.

[36]  Qianjin Feng,et al.  Identification of topological features in renal tumor microenvironment associated with patient survival , 2017, Bioinform..

[37]  Guy Cazuguel,et al.  Multiple-Instance Learning for Medical Image and Video Analysis , 2017, IEEE Reviews in Biomedical Engineering.

[38]  Eric Granger,et al.  Multiple instance learning: A survey of problem characteristics and applications , 2016, Pattern Recognit..

[39]  Wenyu Liu,et al.  Revisiting multiple instance neural networks , 2016, Pattern Recognit..

[40]  Andrew Janowczyk,et al.  Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases , 2016, Journal of pathology informatics.

[41]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Joel H. Saltz,et al.  Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Jaume Amores,et al.  Multiple instance classification: Review, taxonomy and comparative study , 2013, Artif. Intell..

[45]  Alexis B. Carter,et al.  Computational Pathology: A Path Ahead. , 2016, Archives of pathology & laboratory medicine.