Resolution-Based Distillation for Efficient Histology Image Classification

Developing deep learning models to analyze histology images has been computationally challenging, as the massive size of the images causes excessive strain on all parts of the computing pipeline. This paper proposes a novel deep learning-based methodology for improving the computational efficiency of histology image classification. The proposed approach is robust when used with images that have reduced input resolution and can be trained effectively with limited labeled data. Pre-trained on the original high-resolution (HR) images, our method uses knowledge distillation (KD) to transfer learned knowledge from a teacher model to a student model trained on the same images at a much lower resolution. To address the lack of large-scale labeled histology image datasets, we perform KD in a self-supervised manner. We evaluate our approach on two histology image datasets associated with celiac disease (CD) and lung adenocarcinoma (LUAD). Our results show that a combination of KD and self-supervision allows the student model to approach, and in some cases, surpass the classification accuracy of the teacher, while being much more efficient. Additionally, we observe an increase in student classification performance as the size of the unlabeled dataset increases, indicating that there is potential to scale further. For the CD data, our model outperforms the HR teacher model, while needing 4 times fewer computations. For the LUAD data, our student model results at 1.25x magnification are within 3% of the teacher model at 10x magnification, with a 64 times computational cost reduction. Moreover, our CD outcomes benefit from performance scaling with the use of more unlabeled data. For 0.625x magnification, using unlabeled data improves accuracy by 4% over the baseline. Thus, our method can improve the feasibility of deep learning solutions for digital pathology with standard computational hardware.

[1]  Saeed Hassanpour,et al.  Deep Learning for Classification of Colorectal Polyps on Whole-slide Images , 2017, Journal of pathology informatics.

[2]  Geoffrey E. Hinton,et al.  Distilling the Knowledge in a Neural Network , 2015, ArXiv.

[3]  Liron Pantanowitz,et al.  The Landscape of Digital Pathology in Transplantation: From the Beginning to the Virtual E-Slide , 2019, Journal of pathology informatics.

[4]  Mahadev Satyanarayanan,et al.  OpenSlide: A vendor-neutral software foundation for digital pathology , 2013, Journal of pathology informatics.

[5]  Ken Turkowski,et al.  Filters for common resampling tasks , 1990 .

[6]  J. Visakorpi,et al.  The diagnosis of coeliac disease. A commentary on the current practices of members of the European Society for Paediatric Gastroenterology and Nutrition (ESPGAN). , 1979, Archives of disease in childhood.

[7]  L. Elli,et al.  Celiac disease: From pathophysiology to treatment , 2017, World journal of gastrointestinal pathophysiology.

[8]  Rich Caruana,et al.  Do Deep Nets Really Need to be Deep? , 2013, NIPS.

[9]  J. Austin,et al.  The 2015 World Health Organization Classification of Lung Tumors: Impact of Genetic, Clinical and Radiologic Advances Since the 2004 Classification. , 2015, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[10]  Max Welling,et al.  Rotation Equivariant CNNs for Digital Pathology , 2018, MICCAI.

[11]  Jacob Gildenblat,et al.  Self-Supervised Similarity Learning for Digital Pathology , 2019, ArXiv.

[12]  Lin Yang,et al.  tissueloc: Whole slide digital pathology image tissue localization , 2019, J. Open Source Softw..

[13]  Paolo Favaro,et al.  Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles , 2016, ECCV.

[14]  Mengjiao Wang,et al.  Improved Knowledge Distillation for Training Fast Low Resolution Face Recognition Model , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[15]  Dacheng Tao,et al.  Self-Supervised Representation Learning by Rotation Feature Decoupling , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Saeed Hassanpour,et al.  Finding a Needle in the Haystack: Attention-Based Classification of High Resolution Microscopy Images , 2018, ArXiv.

[17]  Deepak Anand,et al.  Breast Cancer Histopathology Image Classification and Localization using Multiple Instance Learning , 2019, 2019 IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE).

[18]  Saeed Hassanpour,et al.  Automated Detection of Celiac Disease on Duodenal Biopsy Slides: A Deep Learning Approach , 2019, Journal of pathology informatics.

[19]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[20]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[21]  Franccois Fleuret,et al.  Processing Megapixel Images with Deep Attention-Sampling Models , 2019, ICML.

[22]  Jeonghwan Gwak,et al.  Utilizing Knowledge Distillation in Deep Learning for Classification of Chest X-Ray Abnormalities , 2020, IEEE Access.

[23]  Yoshua Bengio,et al.  FitNets: Hints for Thin Deep Nets , 2014, ICLR.

[24]  Shiming Ge,et al.  Low-Resolution Face Recognition in the Wild via Selective Knowledge Distillation , 2018, IEEE Transactions on Image Processing.

[25]  Masahiro Tsuboi,et al.  International Association for the Study of Lung Cancer/American Thoracic Society/European Respiratory Society International Multidisciplinary Classification of Lung Adenocarcinoma , 2011, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[26]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.

[27]  Tony X. Han,et al.  Learning Efficient Object Detection Models with Knowledge Distillation , 2017, NIPS.

[28]  W. Caspary,et al.  Celiac disease , 2006, Orphanet journal of rare diseases.

[29]  William Pao,et al.  Comprehensive Histologic Assessment Helps to Differentiate Multiple Lung Primary Nonsmall Cell Carcinomas From Metastases , 2009, The American journal of surgical pathology.

[30]  Saeed Hassanpour,et al.  Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks , 2019, Scientific Reports.

[31]  Rich Caruana,et al.  Model compression , 2006, KDD '06.

[32]  Gregory Shakhnarovich,et al.  Learning Representations for Automatic Colorization , 2016, ECCV.

[33]  Paolo Favaro,et al.  Representation Learning by Learning to Count , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[34]  David A. Forsyth,et al.  Learning Large-Scale Automatic Image Colorization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[35]  Ghassan Hamarneh,et al.  Predicting Cancer with a Recurrent Visual Attention Model for Histopathology Images , 2018, MICCAI.

[36]  Peter Bankhead,et al.  QuPath: Open source software for digital pathology image analysis , 2017, Scientific Reports.

[37]  Fan Yang,et al.  Predicting Lymph Node Metastasis Using Histopathological Images Based on Multiple Instance Learning With Deep Graph Convolution , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Elisabeth Brambilla,et al.  The 2004 World Health Organization classification of lung tumors. , 2005, Seminars in roentgenology.

[39]  David B. A. Epstein,et al.  Dense Steerable Filter CNNs for Exploiting Rotational Symmetry in Histology Images , 2020, IEEE Transactions on Medical Imaging.

[40]  Linda G. Shapiro,et al.  Multi-Instance Multi-Label Learning for Multi-Class Classification of Whole Slide Breast Histopathology Images , 2018, IEEE Transactions on Medical Imaging.

[41]  Pavitra Krishnaswamy,et al.  Self-Path: Self-Supervision for Classification of Pathology Images With Limited Annotations , 2020, IEEE Transactions on Medical Imaging.

[42]  Tahsin Kurc,et al.  Twenty Years of Digital Pathology: An Overview of the Road Travelled, What is on the Horizon, and the Emergence of Vendor-Neutral Archives , 2018, Journal of pathology informatics.

[43]  Geert J. S. Litjens,et al.  Training convolutional neural networks with megapixel images , 2018, ArXiv.

[44]  Changming Sun,et al.  Knowledge Adaptation for Efficient Semantic Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  A. Jemal,et al.  Lung Cancer Statistics. , 2016, Advances in experimental medicine and biology.

[46]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  E. Berg,et al.  World Health Organization Classification of Tumours , 2002 .

[48]  Liu Yan-hui,et al.  Interpretation of Pathological Perspective——International Association for the Study of Lung Cancer/American Thoracic Society/European Respiratory Society International Multidisciplinary Classification of Lung Adenocarcinoma , 2011 .

[49]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[50]  Jeroen van der Laak,et al.  Detection of Prostate Cancer in Whole-Slide Images Through End-to-End Training With Image-Level Labels , 2020, IEEE Transactions on Medical Imaging.

[51]  W. Travis,et al.  The new World Health Organization classification of lung tumours , 2001, European Respiratory Journal.

[52]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[53]  Nikos Paragios,et al.  Weakly supervised multiple instance learning histopathological tumor segmentation , 2020, MICCAI.

[54]  Nikos Komodakis,et al.  Unsupervised Representation Learning by Predicting Image Rotations , 2018, ICLR.

[55]  Max Welling,et al.  Attention-based Deep Multiple Instance Learning , 2018, ICML.

[56]  Che-Rung Lee,et al.  Knowledge Distillation with Feature Maps for Image Classification , 2018, ACCV.

[57]  Jihyoun Jeon,et al.  Lung Cancer Incidence Trends by Gender, Race and Histology in the United States, 1973–2010 , 2015, PloS one.