Stain normalization of histopathology images using generative adversarial networks

Computational histopathology involves CAD for microscopic analysis of stained histopathological slides to study presence, localization or grading of disease. An important stage in a CAD system, stain color normalization, has been broadly studied. The existing approaches are mainly defined in the context of stain deconvolution and template matching. In this paper, we propose a novel approach to this problem by introducing a parametric, fully unsupervised generative model. Our model is based on end-to-end machine learning in the framework of generative adversarial networks. It can learn a nonlinear transformation of a set of latent variables, which are forced to have a prior Dirichlet distribution and control the color of staining hematoxylin and eosin (H&E) images. By replacing the latent variables of a source image with those extracted from a template image in the trained model, it can generate a new color copy of the source image while preserving the important tissue structures resembling the chromatic information of the template image. Our proposed method can instantly be applied to new unseen images, which is different from previous methods that need to compute some statistical properties on input test data. This is potentially problematic when the test sample sizes are limited. Experiments on H&E images from different laboratories show that the proposed model outperforms most state-of-the-art methods.

[1]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[2]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[3]  J. S. Marron,et al.  A method for normalizing histology slides for quantitative analysis , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[4]  Anil K. Jain Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[5]  Konstantinos N. Plataniotis,et al.  Circular Mixture Modeling of Color Distribution for Blind Stain Separation in Pathology Images , 2017, IEEE Journal of Biomedical and Health Informatics.

[6]  Yi-Ying Wang,et al.  A Color-Based Approach for Automated Segmentation in Tumor Tissue Classification , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[7]  Alexander Zelinsky,et al.  Fast Radial Symmetry for Detecting Points of Interest , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Nasir M. Rajpoot,et al.  Stain Deconvolution Using Statistical Analysis of Multi-Resolution Stain Colour Representation , 2017, PloS one.

[9]  Anant Madabhushi,et al.  EM-based segmentation-driven color standardization of digitized histopathology , 2013, Medical Imaging.

[10]  Nazri Mohd Nawi,et al.  The Effect of Data Pre-processing on Optimized Training of Artificial Neural Networks , 2013 .

[11]  Oscar Déniz,et al.  Colour Model Analysis for Histopathology Image Processing , 2013 .

[12]  S. Mohamed,et al.  Statistical Normalization and Back Propagation for Classification , 2022 .

[13]  Nasir M. Rajpoot,et al.  A Nonlinear Mapping Approach to Stain Normalization in Digital Histopathology Images Using Image-Specific Color Deconvolution , 2014, IEEE Transactions on Biomedical Engineering.

[14]  A. Ruifrok,et al.  Quantification of histochemical staining by color deconvolution. , 2001, Analytical and quantitative cytology and histology.

[15]  J. S. Marron,et al.  Appearance normalization of histology slides , 2015, Comput. Medical Imaging Graph..

[16]  Ewert Bengtsson,et al.  Automated segmentation of free-lying cell nuclei in Pap smears for malignancy-associated change analysis , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[17]  Bram van Ginneken,et al.  The importance of stain normalization in colorectal tissue classification with convolutional networks , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).

[18]  Neeraj Kumar,et al.  Empirical comparison of color normalization methods for epithelial-stromal classification in H and E images , 2016, Journal of pathology informatics.

[19]  Derek R. Magee,et al.  Colour Normalisation in Digital Histopathology Images , 2009 .

[20]  Yukako Yagi,et al.  Color standardization in whole slide imaging using a color calibration slide , 2014, Journal of pathology informatics.

[21]  Todd H. Stokes,et al.  Automatic batch-invariant color segmentation of histological cancer images , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[22]  Nico Karssemeijer,et al.  Stain Specific Standardization of Whole-Slide Histopathological Images , 2016, IEEE Transactions on Medical Imaging.

[23]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[24]  Jie Wu,et al.  Covert Video Classification by Codebook Growing Pattern , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  Dan Schonfeld,et al.  Color normalization of histology slides using graph regularized sparse NMF , 2017, Medical Imaging.

[26]  Konstantinos N. Plataniotis,et al.  A Complete Color Normalization Approach to Histopathology Images Using Color Cues Computed From Saturation-Weighted Statistics , 2015, IEEE Transactions on Biomedical Engineering.