Self supervised learning improves dMMR/MSI detection from histology slides across multiple cancers

Microsatellite instability (MSI) is a tumor phenotype whose diagnosis largely impacts patient care in colorectal cancers (CRC), and is associated with response to immunotherapy in all solid tumors. Deep learning models detecting MSI tumors directly from H&E stained slides have shown promise in improving diagnosis of MSI patients. Prior deep learning models for MSI detection have relied on neural networks pretrained on ImageNet dataset, which does not contain any medical image. In this study, we leverage recent advances in self-supervised learning by training neural networks on histology images from the TCGA dataset using MoCo V2. We show that these networks consistently outperform their counterparts pretrained using ImageNet and obtain state-of-the-art results for MSI detection with AUCs of 0.92 and 0.83 for CRC and gastric tumors, respectively. These models generalize well on an external CRC cohort (0.97 AUC on PAIP) and improve transfer from one organ to another. Finally we show that predictive image regions exhibit meaningful histological patterns, and that the use of MoCo features highlighted more relevant patterns according to an expert pathologist.

[1]  Anne L. Martel,et al.  Self-supervised driven consistency training for annotation efficient histopathology image analysis , 2021, Medical Image Anal..

[2]  M. Dennis,et al.  Artificial Intelligence for Histology-Based Detection of Microsatellite Instability and Prediction of Response to Immunotherapy in Colorectal Cancer , 2021, Cancers.

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Georg Heigold,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2021, ICLR.

[6]  Jakob Nikolas Kather,et al.  Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer , 2019, Nature Medicine.

[7]  N. Razavian,et al.  Predicting endometrial cancer subtypes and molecular features from histopathology images using multi-resolution deep learning models , 2020, bioRxiv.

[8]  Kaiming He,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  S. Gruber,et al.  Pathologic Predictors of Microsatellite Instability in Colorectal Cancer , 2009, The American journal of surgical pathology.

[10]  H. Sakamoto,et al.  Predictive model for high‐frequency microsatellite instability in colorectal cancer patients over 50 years of age , 2017, Cancer medicine.

[11]  Vinay Prasad,et al.  Cancer Drugs Approved Based on Biomarkers and Not Tumor Type-FDA Approval of Pembrolizumab for Mismatch Repair-Deficient Solid Cancers. , 2017, JAMA oncology.

[12]  Kaiming He,et al.  Improved Baselines with Momentum Contrastive Learning , 2020, ArXiv.

[13]  Ming Y. Lu,et al.  Semi-Supervised Histology Classification using Deep Multiple Instance Learning and Contrastive Predictive Coding , 2019, ArXiv.

[14]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Payal Sipahimalani,et al.  A Histology-Based Model for Predicting Microsatellite Instability in Colorectal Cancers , 2010, The American journal of surgical pathology.

[16]  Alexander T. Pearson,et al.  Clinical-grade Detection of Microsatellite Instability in Colorectal Tumors by Deep Learning. , 2020, Gastroenterology.

[17]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[18]  Jakob Nikolas Kather,et al.  Artificial Intelligence Can Cut Costs While Maintaining Accuracy in Colorectal Cancer Genotyping , 2021, Frontiers in Oncology.

[19]  Michal Valko,et al.  Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning , 2020, NeurIPS.

[20]  Bin Li,et al.  Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning , 2020, ArXiv.

[21]  Tang,et al.  Development and interpretation of a pathomics-based model for the prediction of microsatellite instability in Colorectal Cancer , 2020, Theranostics.

[22]  H. Jang,et al.  Feasibility of deep learning‐based fully automated classification of microsatellite instability in tissue slides of colorectal cancer , 2021, International journal of cancer.

[23]  Julien Mairal,et al.  Emerging Properties in Self-Supervised Vision Transformers , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[24]  X. Puig,et al.  Microsatellite instability of the colorectal carcinoma can be predicted in the conventional pathologic examination. A prospective multicentric study and the statistical analysis of 615 cases consolidate our previously proposed logistic regression model , 2010, Virchows Archiv.

[25]  Max Welling,et al.  Attention-based Deep Multiple Instance Learning , 2018, ICML.

[26]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  N. Rajpoot,et al.  Novel deep learning algorithm predicts the status of molecular pathways and key mutations in colorectal cancer from routine histology images , 2021, medRxiv.

[29]  Eric W. Tramel,et al.  Classification and Disease Localization in Histopathology Using Only Global Labels: A Weakly-Supervised Approach , 2018, ArXiv.

[30]  B. Martin,et al.  Deep learning model for the prediction of microsatellite instability in colorectal cancer: a diagnostic study. , 2021, The Lancet. Oncology.

[31]  Saining Xie,et al.  An Empirical Study of Training Self-Supervised Vision Transformers , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[32]  Lu Yuan,et al.  Efficient Self-supervised Vision Transformers for Representation Learning , 2021, ArXiv.

[33]  Jean-Philippe Thiran,et al.  Self-Rule to Adapt: Learning Generalized Features from Sparsely-Labeled Data Using Unsupervised Domain Adaptation for Colorectal Cancer Tissue Phenotyping , 2021, MIDL.

[34]  R. Labianca,et al.  Defective mismatch repair as a predictive marker for lack of efficacy of fluorouracil-based adjuvant therapy in colon cancer. , 2010, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[35]  Pavitra Krishnaswamy,et al.  Self-Path: Self-Supervision for Classification of Pathology Images With Limited Annotations , 2020, IEEE Transactions on Medical Imaging.

[36]  Gianluca Bontempi,et al.  TCGAbiolinks: an R/Bioconductor package for integrative analysis of TCGA data , 2015, Nucleic acids research.

[37]  Jakob Nikolas Kather,et al.  Pan-cancer image-based detection of clinically actionable genetic alterations , 2019, Nature Cancer.

[38]  Ludmila V. Danilova,et al.  Mismatch repair deficiency predicts response of solid tumors to PD-1 blockade , 2017, Science.

[39]  Pierre Courtiol,et al.  Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology , 2020, ArXiv.

[40]  A. Scarpa,et al.  ESMO recommendations on microsatellite instability testing for immunotherapy in cancer, and its relationship with PD-1/PD-L1 expression and tumour mutational burden: a systematic review-based approach. , 2019, Annals of oncology : official journal of the European Society for Medical Oncology.

[41]  Geoffrey E. Hinton,et al.  Big Self-Supervised Models are Strong Semi-Supervised Learners , 2020, NeurIPS.

[42]  A. Duval,et al.  MSI/MMR-deficient tumor diagnosis: Which standard for screening and for diagnosis? Diagnostic modalities for the colon and other sites: Differences between tumors. , 2019, Bulletin du cancer.

[43]  John D Potter,et al.  Pathology features in Bethesda guidelines predict colorectal cancer microsatellite instability: a population-based study. , 2007, Gastroenterology.

[44]  Jacob Gildenblat,et al.  Self-Supervised Similarity Learning for Digital Pathology , 2019, ArXiv.