Computational Histological Staining and Destaining of Prostate Core Biopsy RGB Images with Generative Adversarial Neural Networks

Histopathology tissue samples are widely available in two states: paraffin-embedded unstained and non-paraffin-embedded stained whole slide RGB images (WSRI). Hematoxylin and eosin stain (H&E) is one of the principal stains in histology but suffers from several shortcomings related to tissue preparation, staining protocols, slowness and human error. We report two novel approaches for training machine learning models for the computational H&E staining and destaining of prostate core biopsy RGB images. The staining model uses a conditional generative adversarial network that learns hierarchical non-linear mappings between whole slide RGB image (WSRI) pairs of prostate core biopsy before and after H&E staining. The trained staining model can then generate computationally H&E-stained prostate core WSRIs using previously unseen non-stained biopsy images as input. The destaining model, by learning mappings between an H&E stained WSRI and a non-stained WSRI of the same biopsy, can computationally destain previously unseen H&E-stained images. Structural and anatomical details of prostate tissue and colors, shapes, geometries, locations of nuclei, stroma, vessels, glands and other cellular components were generated by both models with structural similarity indices of 0.68 (staining) and 0.84 (destaining). The proposed staining and destaining models can engender computational H&E staining and destaining of WSRI biopsies without additional equipment and devices.

[1]  Peter H. N. de With,et al.  Stain normalization of histopathology images using generative adversarial networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[2]  Bahram Parvin,et al.  Automated Histology Analysis: Opportunities for signal processing , 2015, IEEE Signal Processing Magazine.

[3]  Clive R. Taylor,et al.  Whole Slide Imaging Versus Microscopy for Primary Diagnosis in Surgical Pathology , 2017, The American journal of surgical pathology.

[4]  David Mayerich,et al.  Stain-less staining for computed histopathology. , 2015, Technology.

[5]  Navid Farahani,et al.  whole slide imaging in pathology: advantages, limitations, and emerging perspectives , 2015 .

[6]  Chris Phillips,et al.  Digistain: a digital staining instrument for histopathology. , 2012, Optics express.

[7]  Janne Heikkilä,et al.  Towards Virtual H&E Staining of Hyperspectral Lung Histology Images Using Conditional Generative Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[8]  Dorit Merhof,et al.  Context-Based Normalization of Histological Stains Using Deep Convolutional Features , 2017, DLMIA/ML-CDS@MICCAI.

[9]  Ada T Feldman,et al.  Tissue processing and hematoxylin and eosin staining. , 2014, Methods in molecular biology.

[10]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Andrew A. Renshaw,et al.  Rubin??s Pathology. Clinicopathologic Foundations of Medicine , 2008 .

[12]  Hyunseok Min,et al.  Neural Stain-Style Transfer Learning using GAN for Histopathological Images , 2017, ArXiv.

[13]  I. Kopriva,et al.  Unsupervised segmentation of low-contrast multichannel images: discrimination of tissue components in microscopic images of unstained specimens , 2015, Scientific Reports.

[14]  Thomas Neff,et al.  Generative Adversarial Network based Synthesis for Supervised Medical Image Segmentation , 2017 .

[15]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[16]  Pinky A. Bautista,et al.  Digital Staining of Unstained Pathological Tissue Samples through Spectral Transmittance Classification , 2005 .

[17]  J. S. Marron,et al.  A method for normalizing histology slides for quantitative analysis , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[18]  Rita Noumeir,et al.  Limitations of the SSIM quality metric in the context of diagnostic imaging , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[19]  M. Rajadhyaksha,et al.  Confocal mosaicing microscopy of human skin ex vivo: spectral analysis for digital staining to simulate histology-like appearance. , 2011, Journal of biomedical optics.