Use of Deep Learning to Develop and Analyze Computational Hematoxylin and Eosin Staining of Prostate Core Biopsy Images for Tumor Diagnosis

Key Points Question Can deep learning systems perform hematoxylin and eosin (H&E) staining and destaining and are the virtual core biopsy samples generated by them as valid and interpretable as their real-life unstained and H&E dye–stained counterparts? Findings In this cross-sectional study, deep learning models were trained using nonstained prostate core biopsy images to generate computationally H&E stained images, and core biopsy images were extracted from each whole slide, consisting of approximately 87 000 registered patch pairs of 1024 × 1024 × 3 pixels each. Comprehensive analyses of virtually stained images vs H&E dye–stained images confirmed successful computational staining. Meaning The findings of this study suggest that whole slide nonstained microscopic images of prostate core biopsy, instead of tissue samples, could be integrated with deep learning algorithms to perform computational H&E staining and destaining for rapid and accurate tumor diagnosis.

[1]  Jerome Mertz,et al.  Optical sectioning microscopy with planar or structured illumination , 2011, Nature Methods.

[2]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Stephen A. Boppart,et al.  Stain-free histopathology by programmable supercontinuum pulses , 2016, Nature Photonics.

[4]  Jiaoti Huang,et al.  Identification of a Cell of Origin for Human Prostate Cancer , 2010, Science.

[5]  Joe W. Gray,et al.  SHIFT: speedy histopathological-to-immunofluorescent translation of whole slide images using conditional generative adversarial networks , 2018, Medical Imaging.

[6]  Bahram Parvin,et al.  Automated Histology Analysis: Opportunities for signal processing , 2015, IEEE Signal Processing Magazine.

[7]  Warren S. Grundfest,et al.  Advanced Biomedical and Clinical Diagnostic Systems IX , 2011 .

[8]  Ada T Feldman,et al.  Tissue processing and hematoxylin and eosin staining. , 2014, Methods in molecular biology.

[9]  Zachary T. Harmany,et al.  Microscopy with ultraviolet surface excitation for rapid slide-free histology , 2017, Nature Biomedical Engineering.

[10]  Jianfei Cai,et al.  Robust Transmission of JPEG2000 Encoded Images Over Packet Loss Channels , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[11]  Masahiro Yamaguchi,et al.  Digital staining for multispectral images of pathological tissue specimens based on combined classification of spectral transmittance. , 2005, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[12]  Tai Sing Lee,et al.  Statistical correlations between two-dimensional images and three-dimensional structures in natural scenes. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[13]  K. Gerwert,et al.  Virtual staining of colon cancer tissue by label-free Raman micro-spectroscopy. , 2017, The Analyst.

[14]  Todd C. Hollon,et al.  Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy , 2017, Nature Biomedical Engineering.

[15]  Jianying Hu,et al.  Artificial intelligence and machine learning in clinical development: a translational perspective , 2019, npj Digital Medicine.

[16]  James G. Fujimoto,et al.  Assessment of breast pathologies using nonlinear microscopy , 2014, Proceedings of the National Academy of Sciences.

[17]  M. Gurcan,et al.  Digital pathology and artificial intelligence. , 2019, The Lancet. Oncology.

[18]  Gerard Lozanski,et al.  DeepFocus: Detection of out-of-focus regions in whole slide digital images using deep learning , 2018, PloS one.

[19]  Michael G. Strintzis,et al.  Optimized transmission of JPEG2000 streams over wireless channels , 2006, IEEE Transactions on Image Processing.

[20]  Pratik Shah,et al.  Convolutional Neural Network for Combined Classification of Fluorescent Biomarkers and Expert Annotations using White Light Images , 2017, 2017 IEEE 17th International Conference on Bioinformatics and Bioengineering (BIBE).

[21]  Aman Rana,et al.  Computational Histological Staining and Destaining of Prostate Core Biopsy RGB Images with Generative Adversarial Neural Networks , 2018, 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA).

[22]  M. Monici Cell and tissue autofluorescence research and diagnostic applications. , 2005, Biotechnology annual review.

[23]  Liron Pantanowitz,et al.  Current State of the Regulatory Trajectory for Whole Slide Imaging Devices in the USA , 2017, Journal of pathology informatics.

[24]  A. Ozcan,et al.  Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning , 2018, Nature Biomedical Engineering.

[25]  Oliver Grimm,et al.  Enabling Histopathological Annotations on Immunofluorescent Images through Virtualization of Hematoxylin and Eosin , 2018, Journal of pathology informatics.

[26]  Masahiro Yamaguchi,et al.  Digital staining of pathological tissue specimens using spectral transmittance , 2005, SPIE Medical Imaging.

[27]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[28]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[29]  Xavière Panhard,et al.  The 20-core prostate biopsy protocol--a new gold standard? , 2008, The Journal of urology.