A Closer Look at Domain Shift for Deep Learning in Histopathology

Domain shift is a significant problem in histopathology. There can be large differences in data characteristics of whole-slide images between medical centers and scanners, making generalization of deep learning to unseen data difficult. To gain a better understanding of the problem, we present a study on convolutional neural networks trained for tumor classification of H&E stained whole-slide images. We analyze how augmentation and normalization strategies affect performance and learned representations, and what features a trained model respond to. Most centrally, we present a novel measure for evaluating the distance between domains in the context of the learned representation of a particular model. This measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. The results show how learning is heavily influenced by the preparation of training data, and that the latent representation used to do classification is sensitive to changes in data distribution, especially when training without augmentation or normalization.

[1]  Patrick D. McDaniel,et al.  Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning , 2018, ArXiv.

[2]  Pascal Vincent,et al.  Visualizing Higher-Layer Features of a Deep Network , 2009 .

[3]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[4]  Geert J. S. Litjens,et al.  Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology , 2019, Medical Image Anal..

[5]  Deborah Silver,et al.  Feature Visualization , 1994, Scientific Visualization.

[6]  Joachim Denzler,et al.  Finding the Unknown: Novelty Detection with Extreme Value Signatures of Deep Neural Activations , 2017, GCPR.

[7]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Meyke Hermsen,et al.  1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset , 2018, GigaScience.

[9]  Nico Karssemeijer,et al.  H and E stain augmentation improves generalization of convolutional networks for histopathological mitosis detection , 2018, Medical Imaging.

[10]  Yukako Yagi,et al.  Color standardization and optimization in Whole Slide Imaging , 2011, Diagnostic pathology.

[11]  Bram van Ginneken,et al.  The importance of stain normalization in colorectal tissue classification with convolutional networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[12]  Samy Bengio,et al.  Transfusion: Understanding Transfer Learning with Applications to Medical Imaging , 2019, ArXiv.

[13]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Jon Kleinberg,et al.  Transfusion: Understanding Transfer Learning for Medical Imaging , 2019, NeurIPS.

[15]  Nassir Navab,et al.  Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images , 2016, IEEE Transactions on Medical Imaging.

[16]  Geoffrey E. Hinton,et al.  Analyzing and Improving Representations with the Soft Nearest Neighbor Loss , 2019, ICML.

[17]  Ghassan Hamarneh,et al.  Adversarial Stain Transfer for Histopathology Image Analysis , 2018, IEEE Transactions on Medical Imaging.

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Anders Heyden,et al.  Generalization of prostate cancer classification for multiple sites using deep learning , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[20]  Jorge Nocedal,et al.  On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima , 2016, ICLR.

[21]  Matthias Bethge,et al.  ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness , 2018, ICLR.

[22]  Samy Bengio,et al.  Adversarial examples in the physical world , 2016, ICLR.

[23]  Alexei A. Efros,et al.  Unbiased look at dataset bias , 2011, CVPR 2011.

[24]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.