WSSS4LUAD: Grand Challenge on Weakly-supervised Tissue Semantic Segmentation for Lung Adenocarcinoma

Lung cancer is the leading cause of cancer death worldwide, and adenocarcinoma (LUAD) is the most common subtype. Exploiting the potential value of the histopathology images can promote precision medicine in oncology. Tissue segmentation is the basic upstream task of histopathology image analysis. Existing deep learning models have achieved superior segmentation performance but require sufficient pixel-level annotations, which is time-consuming and expensive. To enrich the label resources of LUAD and to alleviate the annotation efforts, we organize this challenge WSSS4LUAD to call for the outstanding weakly-supervised semantic segmentation (WSSS) techniques for histopathology images of LUAD. Participants have to design the algorithm to segment tumor epithelial, tumor-associated stroma and normal tissue with only patch-level labels. This challenge includes 10,091 patch-level annotations (the training set) and over 130 million labeled pixels (the validation and test sets), from 87 WSIs (67 from GDPH, 20 from TCGA). All the labels were generated by a pathologist-in-the-loop pipeline with the help of AI models and checked by the label review board. Among 532 registrations, 28 teams submitted the results in the test phase with over 1,000 submissions. Finally, the first place team achieved mIoU of 0.8413 (tumor: 0.8389, stroma: 0.7931, normal: 0.8919). According to the technical reports of the top-tier teams, CAM is still the most popular approach in WSSS. Cutmix data augmentation has been widely adopted to generate more reliable samples. With the success of this challenge, we believe that WSSS approaches with patch-level annotations can be a complement to the traditional pixel annotations while reducing the annotation efforts. The entire dataset has been released to encourage more researches on computational pathology in LUAD and more novel WSSS techniques.

[1]  Zhineng Chen,et al.  Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification , 2022, ArXiv.

[2]  Donggeun Yoo,et al.  Artificial Intelligence–Powered Spatial Analysis of Tumor-Infiltrating Lymphocytes as Complementary Biomarker for Immune Checkpoint Inhibition in Non–Small-Cell Lung Cancer , 2022, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[3]  Kwang-Ting Cheng,et al.  Label Propagation for Annotation-Efficient Nuclei Segmentation from Pathology Images , 2022, ArXiv.

[4]  D. Hanahan.  Hallmarks of Cancer: New Dimensions. , 2022, Cancer discovery.

[5]  Ross B. Girshick,et al.  Masked Autoencoders Are Scalable Vision Learners , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Jiatai Lin,et al.  PDBL: Improving Histopathological Tissue Classification With Plug-and-Play Pyramidal Deep-Broad Learning , 2021, IEEE Transactions on Medical Imaging.

[7]  Yi Wang,et al.  Multi-Layer Pseudo-Supervision for Histopathology Tissue Semantic Segmentation using Patch-level Classification Labels , 2021, Medical Image Anal..

[8]  R. Duwairi,et al.  Self-supervised learning methods and applications in medical imaging analysis: a survey , 2021, PeerJ Comput. Sci..

[9]  Nikos Paragios,et al.  Self-Supervised Representation Learning using Visual Field Expansion on Digital Pathology , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[10]  Nasir Rajpoot,et al.  Robust Interactive Semantic Segmentation of Pathology Images with Minimal User Input , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[11]  Eryun Liu,et al.  Joint fully convolutional and graph convolutional networks for weakly-supervised segmentation of pathology images , 2021, Medical Image Anal..

[12]  Manfredo Atzori,et al.  Semi-supervised training of deep convolutional neural networks with heterogeneous data and few local annotations: An experiment on prostate histopathology image classification , 2021, Medical Image Anal..

[13]  Xin Chen,et al.  Curriculum Self-supervised Learning for Weakly-supervised Histopathological Image Segmentation , 2021, 2021 10th International Conference on Internet Computing for Science and Engineering.

[14]  Hao Chen,et al.  Deep Semi-Supervised Metric Learning with Dual Alignment for Cervical Cancer Cell Detection , 2021, 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI).

[15]  Tyler T. Risom,et al.  Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning , 2021, Nature Biotechnology.

[16]  Anne L. Martel,et al.  Self-supervised driven consistency training for annotation efficient histopathology image analysis , 2021, Medical Image Anal..

[17]  A. Jemal,et al.  Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries , 2021, CA: a cancer journal for clinicians.

[18]  A. Madabhushi,et al.  A prognostic model for overall survival of patients with early-stage non-small cell lung cancer: a multicentre, retrospective study , 2020, The Lancet. Digital health.

[19]  Ismail Ben Ayed,et al.  Deep Active Learning for Joint Classification & Segmentation with Weak Annotator , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[20]  Jun Zhang,et al.  Deep Active Learning for Breast Cancer Segmentation on Immunohistochemistry Images , 2020, MICCAI.

[21]  L. Cooper,et al.  A narrative review of digital pathology and artificial intelligence: focusing on lung cancer , 2020, Translational lung cancer research.

[22]  Xichao Teng,et al.  Deep Learning Methods for Lung Cancer Segmentation in Whole-Slide Histopathology Images—The ACDC@LungHP Challenge 2019 , 2020, IEEE Journal of Biomedical and Health Informatics.

[23]  Zhibin Liao,et al.  Pairwise Relation Learning for Semi-supervised Gland Segmentation , 2020, MICCAI.

[24]  W. Seeger,et al.  Spatial Density and Distribution of Tumor-Associated Macrophages Predict Survival in Non–Small Cell Lung Carcinoma , 2020, Cancer Research.

[25]  Kai Ma,et al.  Self-Loop Uncertainty: A Novel Pseudo-Label for Semi-Supervised Medical Image Segmentation , 2020, MICCAI.

[26]  Zhiwen Yu,et al.  Triple U-net: Hematoxylin-aware nuclei segmentation with progressive dense feature aggregation , 2020, Medical Image Anal..

[27]  Brian Kenji Iwana,et al.  Negative Pseudo Labeling using Class Proportion for Semantic Segmentation in Pathology , 2020, ECCV.

[28]  Won-Ki Jeong,et al.  Scribble2Label: Scribble-Supervised Cell Segmentation via Self-Generating Pseudo-Labels with Consistency , 2020, MICCAI.

[29]  Jeroen van der Laak,et al.  HookNet: multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images , 2020, Medical Image Anal..

[30]  Dimitris N. Metaxas,et al.  Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images , 2020, IEEE Transactions on Medical Imaging.

[31]  Pierre H. Richemond,et al.  Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning , 2020, NeurIPS.

[32]  Kai Ma,et al.  Rubik's Cube+: A self-supervised feature learning framework for 3D medical image analysis , 2020, Medical Image Anal..

[33]  Navid Alemi Koohbanani,et al.  NuClick: A Deep Learning Framework for Interactive Segmentation of Microscopy Images , 2020, Medical Image Anal..

[34]  Nicolai J. Birkbak,et al.  Geospatial immune variability illuminates differential evolution of lung adenocarcinoma , 2020, Nature Medicine.

[35]  Nikos Paragios,et al.  Weakly supervised multiple instance learning histopathological tumor segmentation , 2020, MICCAI.

[36]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[37]  Anne L. Martel,et al.  Deep neural network models for computational histopathology: A survey , 2019, Medical Image Anal..

[38]  Ross B. Girshick,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Yang Xie,et al.  Artificial Intelligence in Lung Cancer Pathology Image Analysis , 2019, Cancers.

[40]  Bernhard Kainz,et al.  A Survey on Active Learning and Human-in-the-Loop Deep Learning for Medical Image Analysis , 2019, Medical Image Anal..

[41]  Yujiu Yang,et al.  Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik's Cube , 2019, MICCAI.

[42]  Konstantinos N. Plataniotis,et al.  HistoSegNet: Semantic Segmentation of Histological Tissue Type in Whole Slide Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[43]  Zhiqiang Hu,et al.  Signet Ring Cell Detection with a Semi-supervised Learning Framework , 2019, IPMI.

[44]  Dimitris N. Metaxas,et al.  Weakly Supervised Deep Nuclei Segmentation using Points Annotation in Histopathology Images , 2019, MIDL.

[45]  Jakob Nikolas Kather,et al.  Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer , 2019, Nature Medicine.

[46]  Jonathan D. Beezley,et al.  Structured crowdsourcing enables convolutional segmentation of histology images , 2019, Bioinform..

[47]  Constantino Carlos Reyes-Aldasoro,et al.  Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study , 2019, PLoS medicine.

[48]  Jin Tae Kwak,et al.  Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images , 2018, Medical Image Anal..

[49]  Gang Wang,et al.  Automatic grading of prostate cancer in digitized histopathology images: Learning from multiple experts , 2018, Medical Image Anal..

[50]  Isaac S Kohane,et al.  Artificial Intelligence in Healthcare , 2019, Artificial Intelligence and Machine Learning for Business for Non-Engineers.

[51]  N. Razavian,et al.  Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning , 2018, Nature Medicine.

[52]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[53]  Joel H. Saltz,et al.  Comparison of Different Classifiers with Active Learning to Support Quality Control in Nucleus Segmentation in Pathology Images , 2018, AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science.

[54]  Josien P. W. Pluim,et al.  Not‐so‐supervised: A survey of semi‐supervised, multi‐instance, and transfer learning in medical image analysis , 2018, Medical Image Anal..

[55]  Jae Y. Shin,et al.  Active, continual fine tuning of convolutional neural networks for reducing annotation efforts , 2018, Medical Image Anal..

[56]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[57]  Zhipeng Jia,et al.  Constrained Deep Weak Supervision for Histopathology Image Segmentation , 2017, IEEE Transactions on Medical Imaging.

[58]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Anton van den Hengel,et al.  Wider or Deeper: Revisiting the ResNet Model for Visual Recognition , 2016, Pattern Recognit..

[60]  Ce Zhang,et al.  Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features , 2016, Nature Communications.

[61]  Alexei A. Efros,et al.  Context Encoders: Feature Learning by Inpainting , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Hao Chen,et al.  DCAN: Deep Contour-Aware Networks for Accurate Gland Segmentation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[63]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[66]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[67]  Anant Madabhushi,et al.  An active learning based classification strategy for the minority class problem: application to histopathology annotation , 2011, BMC Bioinformatics.

[68]  Rafael Sirera,et al.  The Role of Tumor Stroma in Cancer Progression and Prognosis: Emphasis on Carcinoma-Associated Fibroblasts and Non-small Cell Lung Cancer , 2011, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[69]  Junzhou Huang,et al.  TransPath: Transformer-Based Self-supervised Learning for Histopathological Image Classification , 2021, MICCAI.

[70]  H. Hansen,et al.  Lung cancer. , 1990, Cancer chemotherapy and biological response modifiers.

[71]  A. Krizhevsky.  ImageNet Classification with Deep Convolutional Neural Networks , 2022 .