Unsupervised Domain Adaptation for Classification of Histopathology Whole-Slide Images

Computational image analysis is one means for evaluating digitized histopathology specimens that can increase the reproducibility and reliability with which cancer diagnoses are rendered while simultaneously providing insight as to the underlying mechanisms of disease onset and progression. A major challenge that is confronted when analyzing samples that have been prepared at disparate laboratories and institutions is that the algorithms used to assess the digitized specimens often exhibit heterogeneous staining characteristics because of slight differences in incubation times and the protocols used to prepare the samples. Unfortunately, such variations can render a prediction model learned from one batch of specimens ineffective for characterizing an ensemble originating from another site. In this work, we propose to adopt unsupervised domain adaptation to effectively transfer the discriminative knowledge obtained from any given source domain to the target domain without requiring any additional labeling or annotation of images at the target site. In this paper, our team investigates the use of two approaches for performing the adaptation: (1) color normalization and (2) adversarial training. The adversarial training strategy is implemented through the use of convolutional neural networks to find an invariant feature space and Siamese architecture within the target domain to add a regularization that is appropriate for the entire set of whole-slide images. The adversarial adaptation results in significant classification improvement compared with the baseline models under a wide range of experimental settings.

[1]  Joakim Lindblad,et al.  Blind Color Decomposition of Histological Images , 2013, IEEE Transactions on Medical Imaging.

[2]  Peng Zhao,et al.  Supervised learning-based cell image segmentation for P53 immunohistochemistry , 2006, IEEE Transactions on Biomedical Engineering.

[3]  Metin Nafi Gürcan,et al.  Adaptive Discriminant Wavelet Packet Transform and Local Binary Patterns for Meningioma Subtype Classification , 2008, MICCAI.

[4]  Fei-Fei Li,et al.  Label Efficient Learning of Transferable Representations acrosss Domains and Tasks , 2017, NIPS.

[5]  Michael Gadermayr,et al.  Which Way Round? A Study on the Performance of Stain-Translation for Segmenting Arbitrarily Dyed Histological Images , 2018, MICCAI.

[6]  Nassir Navab,et al.  Staingan: Stain Style Transfer for Digital Histological Images , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[7]  Anil K. Jain,et al.  Prostate cancer grading: Gland segmentation and structural features , 2012, Pattern Recognit. Lett..

[8]  Jaime S. Cardoso,et al.  Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support , 2017, Lecture Notes in Computer Science.

[9]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[10]  Todd H. Stokes,et al.  Automatic batch-invariant color segmentation of histological cancer images , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Nico Karssemeijer,et al.  Stain Specific Standardization of Whole-Slide Histopathological Images , 2016, IEEE Transactions on Medical Imaging.

[13]  Ewert Bengtsson,et al.  A New Method for Segmentation of Colour Images Applied to Immunohistochemically Stained Cell Nuclei , 1997, Analytical cellular pathology : the journal of the European Society for Analytical Cellular Pathology.

[14]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Dumitru Erhan,et al.  Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Ming-Yu Liu,et al.  Coupled Generative Adversarial Networks , 2016, NIPS.

[17]  Svitlana Zinger,et al.  Histopathology stain-color normalization using deep generative models , 2018 .

[18]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.

[19]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[20]  Joel H. Saltz,et al.  Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Jocelyn Barker,et al.  A method for normalizing pathology images to improve feature extraction for quantitative pathology. , 2016, Medical physics.

[22]  Hamid Soltanian-Zadeh,et al.  Multiwavelet grading of pathological images of prostate , 2003, IEEE Transactions on Biomedical Engineering.

[23]  Nassir Navab,et al.  Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images , 2016, IEEE Transactions on Medical Imaging.

[24]  Yiran Chen,et al.  A Compact DNN: Approaching GoogLeNet-Level Accuracy of Classification and Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Deborah B. Thompson,et al.  AUTOMATED LOCATION OF DYSPLASTIC FIELDS IN COLORECTAL HISTOLOGY USING IMAGE TEXTURE ANALYSIS , 1997, The Journal of pathology.

[26]  Kim L. Boyer,et al.  Computer-Aided Grading of Neuroblastic Differentiation: Multi-Resolution and Multi-Classifier Approach , 2007, 2007 IEEE International Conference on Image Processing.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Michael Titford,et al.  What May the Future Hold for Histotechnologists , 2012 .

[29]  Carlos D. Castillo,et al.  Generate to Adapt: Aligning Domains Using Generative Adversarial Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  L. Egevad,et al.  A Contemporary Prostate Cancer Grading System: A Validated Alternative to the Gleason Score. , 2016, European urology.

[31]  Peter H. N. de With,et al.  Stain normalization of histopathology images using generative adversarial networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[32]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[33]  Mark D Zarella,et al.  An alternative reference space for H&E color normalization , 2017, PloS one.

[34]  Gyan Bhanot,et al.  Computerized Image-Based Detection and Grading of Lymphocytic Infiltration in HER2+ Breast Cancer Histopathology , 2010, IEEE Transactions on Biomedical Engineering.

[35]  C. Mathers,et al.  Cancer incidence and mortality worldwide: Sources, methods and major patterns in GLOBOCAN 2012 , 2015, International journal of cancer.

[36]  Aaron Fenster,et al.  Prostate Histopathology: Learning Tissue Component Histograms for Cancer Detection and Classification , 2013, IEEE Transactions on Medical Imaging.

[37]  Nicolas Papadakis,et al.  A Variational Model for Histogram Transfer of Color Images , 2011, IEEE Transactions on Image Processing.

[38]  Bram van Ginneken,et al.  The importance of stain normalization in colorectal tissue classification with convolutional networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[39]  Chandan Chakraborty,et al.  Statistical Analysis of Textural Features for Improved Classification of Oral Histopathological Images , 2012, Journal of Medical Systems.

[40]  A. Madabhushi,et al.  Histopathological Image Analysis: A Review , 2009, IEEE Reviews in Biomedical Engineering.

[41]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[42]  Kamyar Nazeri,et al.  Two-Stage Convolutional Neural Network for Breast Cancer Histology Image Classification , 2018 .

[43]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Deendayal Dinakarpandian,et al.  A New Metric to Measure Gene Product Similarity , 2007, BIBM.

[45]  J. S. Marron,et al.  Appearance normalization of histology slides , 2015, Comput. Medical Imaging Graph..

[46]  Anant Madabhushi,et al.  EM-based segmentation-driven color standardization of digitized histopathology , 2013, Medical Imaging.

[47]  Qilong Wang,et al.  Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  O. Lézoray,et al.  A comparison of supervised pixel-based color image segmentation methods. Application in cancerology , 2003 .

[49]  Konstantinos N. Plataniotis,et al.  A Complete Color Normalization Approach to Histopathology Images Using Color Cues Computed From Saturation-Weighted Statistics , 2015, IEEE Transactions on Biomedical Engineering.

[50]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[51]  Joel H. Saltz,et al.  Pathological Image Analysis Using the GPU: Stroma Classification for Neuroblastoma , 2007, 2007 IEEE International Conference on Bioinformatics and Biomedicine (BIBM 2007).

[52]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[53]  Anant Madabhushi,et al.  AUTOMATED GRADING OF PROSTATE CANCER USING ARCHITECTURAL AND TEXTURAL IMAGE FEATURES , 2007, 2007 4th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[54]  Xin Qi,et al.  Computer aided analysis of prostate histopathology images Gleason grading especially for Gleason score 7 , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[55]  Nasir M. Rajpoot,et al.  Stain Deconvolution Using Statistical Analysis of Multi-Resolution Stain Colour Representation , 2017, PloS one.

[56]  Mehrtash Tafazzoli Harandi,et al.  Learning an Invariant Hilbert Space for Domain Adaptation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Alexander Rakhlin,et al.  Deep Convolutional Neural Networks for Breast Cancer Histology Image Analysis , 2018, bioRxiv.

[58]  Mikhail Teverovskiy,et al.  Multifeature Prostate Cancer Diagnosis and Gleason Grading of Histological Images , 2007, IEEE Transactions on Medical Imaging.

[59]  Fabio A. González,et al.  Combining Unsupervised Feature Learning and Riesz Wavelets for Histopathology Image Representation: Application to Identifying Anaplastic Medulloblastoma , 2015, MICCAI.

[60]  Nasir M. Rajpoot,et al.  A Nonlinear Mapping Approach to Stain Normalization in Digital Histopathology Images Using Image-Specific Color Deconvolution , 2014, IEEE Transactions on Biomedical Engineering.

[61]  Ghassan Hamarneh,et al.  Adversarial Stain Transfer for Histopathology Image Analysis , 2018, IEEE Transactions on Medical Imaging.

[62]  Xin Qi,et al.  Computer aided analysis of prostate histopathology images to support a refined Gleason grading system , 2017, Medical Imaging.

[63]  Benjamin J. Raphael,et al.  Mutational landscape and significance across 12 major cancer types , 2013, Nature.

[64]  Elizabeth Genega,et al.  Network cycle features: Application to computer-aided Gleason grading of prostate cancer histopathological images , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[65]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[66]  Xin Qi,et al.  Adversarial Domain Adaptation for Classification of Prostate Histopathology Whole-Slide Images , 2018, MICCAI.

[67]  J. S. Marron,et al.  A method for normalizing histology slides for quantitative analysis , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[68]  Anil K. Jain Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[69]  Mehmet Toner,et al.  Spatially Invariant Vector Quantization: A pattern matching algorithm for multiple classes of image subject matter including pathology , 2011, Journal of pathology informatics.

[70]  Shyam Lal,et al.  A study about color normalization methods for histopathology images. , 2018, Micron.

[71]  Derek R. Magee,et al.  Colour Normalisation in Digital Histopathology Images , 2009 .

[72]  Andrew Janowczyk,et al.  Stain Normalization using Sparse AutoEncoders (StaNoSA): Application to digital pathology , 2017, Comput. Medical Imaging Graph..

[73]  B. van Ginneken,et al.  Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis , 2016, Scientific Reports.

[74]  Hyunseok Min,et al.  Neural Stain-Style Transfer Learning using GAN for Histopathological Images , 2017, ArXiv.

[75]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[76]  Mats Andersson,et al.  Convolutional neural networks for an automatic classification of prostate tissue slides with high-grade Gleason score , 2017, Medical Imaging.

[77]  Morten Wang Fagerland,et al.  The McNemar test for binary matched-pairs data: mid-p and asymptotic are better than exact conditional , 2013, BMC Medical Research Methodology.