Deep learning-based transformation of the H&E stain into special stains improves kidney disease diagnosis

Pathology is practiced by visual inspection of histochemically stained slides. Most commonly, the hematoxylin and eosin (H&E) stain is used in the diagnostic workflow and it is the gold standard for cancer diagnosis. However, in many cases, especially for non-neoplastic diseases, additional "special stains" are used to provide different levels of contrast and color to tissue components and allow pathologists to get a clearer diagnostic picture. In this study, we demonstrate the utility of supervised learning-based computational stain transformation from H&E to different special stains (Masson's Trichrome, periodic acid-Schiff and Jones silver stain) using tissue sections from kidney needle core biopsies. Based on evaluation by three renal pathologists, followed by adjudication by a fourth renal pathologist, we show that the generation of virtual special stains from existing H&E images improves the diagnosis in several non-neoplastic kidney diseases, sampled from 16 unique subjects. Adjudication of N=48 diagnoses from the three pathologists revealed that the virtually generated special stains yielded 22 improvements (45.8%), 23 concordances (47.9%) and 3 discordances (6.3%), when compared against the use of H&E stained tissue only. As the virtual transformation of H&E images into special stains can be achieved in less than 1 min per patient core specimen slide, this stain-to-stain transformation framework can improve the quality of the preliminary diagnosis when additional special stains are needed, along with significant savings in time and cost, reducing the burden on healthcare system and patients.

[1]  Aman Rana,et al.  Use of Deep Learning to Develop and Analyze Computational Hematoxylin and Eosin Staining of Prostate Core Biopsy Images for Tumor Diagnosis , 2020, JAMA network open.

[2]  Janne Heikkilä,et al.  Towards Virtual H&E Staining of Hyperspectral Lung Histology Images Using Conditional Generative Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[3]  A. Ozcan,et al.  Deep learning enables cross-modality super-resolution in fluorescence microscopy , 2018, Nature Methods.

[4]  James G. Fujimoto,et al.  Assessment of breast pathologies using nonlinear microscopy , 2014, Proceedings of the National Academy of Sciences.

[5]  Sina Honari,et al.  Distribution Matching Losses Can Hallucinate Features in Medical Image Translation , 2018, MICCAI.

[6]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Hidekata Hontani,et al.  Re-staining Pathology Images by FCNN , 2019, 2019 16th International Conference on Machine Vision Applications (MVA).

[8]  Brooke L. Billman,et al.  Collection and Handling of Thoracic Small Biopsy and Cytology Specimens for Ancillary Studies: Guideline From the College of American Pathologists in Collaboration With the American College of Chest Physicians, Association for Molecular Pathology, American Society of Cytopathology, American Thoracic , 2020, Archives of pathology & laboratory medicine.

[9]  S. Bonsib,et al.  Practice guidelines for the renal biopsy , 2004, Modern Pathology.

[10]  Navid Borhani,et al.  Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy. , 2019, Biomedical optics express.

[11]  Nassir Navab,et al.  Seamless Virtual Whole Slide Image Synthesis and Validation Using Perceptual Embedding Consistency , 2020, IEEE Journal of Biomedical and Health Informatics.

[12]  Michael Gadermayr,et al.  Which Way Round? A Study on the Performance of Stain-Translation for Segmenting Arbitrarily Dyed Histological Images , 2018, MICCAI.

[13]  Nassir Navab,et al.  Staingan: Stain Style Transfer for Digital Histological Images , 2018, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[14]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[15]  Yibo Zhang,et al.  PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning , 2018, Light: Science & Applications.

[16]  Zachary T. Harmany,et al.  Microscopy with ultraviolet surface excitation for rapid slide-free histology , 2017, Nature Biomedical Engineering.

[17]  Apostolos Delis,et al.  Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue , 2020, Light, science & applications.

[18]  R. Erlandson,et al.  Role of Electron Microscopy in Modern Diagnostic Surgical Pathology , 2009, Modern Surgical Pathology.

[19]  Joshua J. Levy,et al.  Preliminary Evaluation of the Utility of Deep Generative Histopathology Image Translation at a Mid-sized NCI Cancer Center , 2020, BIOINFORMATICS.

[20]  Marcel van Gerven,et al.  Virtual Staining for Mitosis Detection in Breast Histopathology , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[21]  Christophe Leterrier,et al.  NanoJ-SQUIRREL: quantitative mapping and minimisation of super-resolution optical imaging artefacts , 2018, Nature Methods.

[22]  Walter H Henricks,et al.  Validation of whole slide imaging for primary diagnosis in surgical pathology. , 2013, Archives of pathology & laboratory medicine.

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  A. Ozcan,et al.  Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning , 2018, Nature Biomedical Engineering.

[25]  Nicolas Brieu,et al.  DASGAN - Joint Domain Adaptation and Segmentation for the Analysis of Epithelial Regions in Histopathology PD-L1 Images , 2019, ArXiv.

[26]  Ke Yan,et al.  Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks , 2019, Scientific Reports.

[27]  Lawrence D. True,et al.  Light-sheet microscopy for slide-free non-destructive pathology of large clinical specimens , 2017, Nature Biomedical Engineering.

[28]  Zuhair M Mohammedsaleh,et al.  Histological Stains: A Literature Review and Case Study , 2015, Global journal of health science.