Teacher-Student chain for efficient semi-supervised histology image classification

Deep learning shows great potential for the domain of digital pathology. An automated digital pathology system could serve as a second reader, perform initial triage in large screening studies, or assist in reporting. However, it is expensive to exhaustively annotate large histology image databases, since medical specialists are a scarce resource. In this paper, we apply the semi-supervised teacher-student knowledge distillation technique proposed by Yalniz et al. (2019) to the task of quantifying prognostic features in colorectal cancer. We obtain accuracy improvements through extending this approach to a chain of students, where each student's predictions are used to train the next student i.e. the student becomes the teacher. Using the chain approach, and only 0.5% labelled data (the remaining 99.5% in the unlabelled pool), we match the accuracy of training on 100% labelled data. At lower percentages of labelled data, similar gains in accuracy are seen, allowing some recovery of accuracy even from a poor initial choice of labelled training set. In conclusion, this approach shows promise for reducing the annotation burden, thus increasing the affordability of automated digital pathology systems.

[1]  Michael L Wilson,et al.  Access to pathology and laboratory medicine services: a crucial gap , 2018, The Lancet.

[2]  Tahsin Kurc,et al.  Twenty Years of Digital Pathology: An Overview of the Road Travelled, What is on the Horizon, and the Emergence of Vendor-Neutral Archives , 2018, Journal of pathology informatics.

[3]  Quoc V. Le,et al.  Self-Training With Noisy Student Improves ImageNet Classification , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Hammad Qureshi,et al.  Translational AI and Deep Learning in Diagnostic Pathology , 2019, Front. Med..

[5]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[6]  Mita Nasipuri,et al.  Patch-based system for Classification of Breast Histology images using deep learning , 2019, Comput. Medical Imaging Graph..

[7]  Constantino Carlos Reyes-Aldasoro,et al.  Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study , 2019, PLoS medicine.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Quoc V. Le,et al.  Unsupervised Data Augmentation for Consistency Training , 2019, NeurIPS.

[10]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Quoc V. Le,et al.  Unsupervised Data Augmentation , 2019, ArXiv.

[12]  M. Salto‐Tellez,et al.  Artificial intelligence—the third revolution in pathology , 2019, Histopathology.

[13]  Kan Chen,et al.  Billion-scale semi-supervised learning for image classification , 2019, ArXiv.

[14]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.