Deep learning‐based color holographic microscopy

We report a framework based on a generative adversarial network (GAN) that performs high-fidelity color image reconstruction using a single hologram of a sample that is illuminated simultaneously by light at three different wavelengths. The trained network learns to eliminate missing-phase-related artifacts, and generates an accurate color transformation for the reconstructed image. Our framework is experimentally demonstrated using lung and prostate tissue sections that are labeled with different histological stains. This framework is envisaged to be applicable to point-of-care histopathology, and presents a significant improvement in the throughput of coherent microscopy systems given that only a single hologram of the specimen is required for accurate color imaging. This article is protected by copyright. All rights reserved.

[1]  Yibo Zhang,et al.  Deep learning-based super-resolution in coherent imaging systems , 2018, Scientific Reports.

[2]  Stephan Saalfeld,et al.  Globally optimal stitching of tiled 3D microscopic image acquisitions , 2009, Bioinform..

[3]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[4]  A. Ozcan,et al.  Synthetic aperture-based on-chip microscopy , 2015, Light: Science & Applications.

[5]  Aydogan Ozcan,et al.  Increased space-bandwidth product in pixel super-resolved lensfree on-chip microscopy , 2013, Scientific Reports.

[6]  A. Ozcan,et al.  Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy , 2012, Optics Express.

[7]  Aydogan Ozcan,et al.  Field-Portable Pixel Super-Resolution Colour Microscope , 2013, PloS one.

[8]  Prarthana Shrestha,et al.  Color accuracy and reproducibility in whole slide imaging scanners , 2014, Medical Imaging.

[9]  A. Ozcan,et al.  Accurate color imaging of pathology slides using holography and absorbance spectrum estimation of histochemical stains , 2018, Journal of biophotonics.

[10]  Bernhard Hill,et al.  Comparative analysis of the quantization of color spaces on the basis of the CIELAB color-difference formula , 1997, TOGS.

[11]  Leslie J. Allen,et al.  Phase retrieval from series of images obtained by defocus variation , 2001 .

[12]  W Scott Campbell,et al.  Sixty-five thousand shades of gray: importance of color in surgical pathology diagnoses. , 2015, Human pathology.

[13]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[14]  Aydogan Ozcan,et al.  Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy , 2012, Nature Methods.

[15]  Aydogan Ozcan,et al.  Edge sparsity criterion for robust holographic autofocusing. , 2017, Optics letters.

[16]  Osamu Matoba,et al.  Digital Holography Using Spectral Estimation Technique , 2014, Journal of Display Technology.

[17]  Yukako Yagi,et al.  Color standardization in whole slide imaging using a color calibration slide , 2014, Journal of pathology informatics.

[18]  Yibo Zhang,et al.  Sparsity-based multi-height phase recovery in holographic microscopy , 2016, Scientific Reports.

[19]  A. Ozcan,et al.  Pixel super-resolution using wavelength scanning , 2015, Light: Science & Applications.

[20]  Aydogan Ozcan,et al.  A robust holographic autofocusing criterion based on edge sparsity: comparison of Gini index and Tamura coefficient for holographic autofocusing based on the edge sparsity of the complex optical wavefront , 2017, BiOS.

[21]  Aydogan Ozcan,et al.  Field-Portable Lensless Holographic Microscope using Pixel Super-Resolution , 2011 .

[22]  Aydogan Ozcan,et al.  Field-portable wide-field microscopy of dense samples using multi-height pixel super-resolution based lensfree imaging. , 2012, Lab on a chip.

[23]  Yong Yu,et al.  Unsupervised Diverse Colorization via Generative Adversarial Networks , 2017, ECML/PKDD.

[24]  A. Ozcan,et al.  Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning , 2018, Nature Biomedical Engineering.

[25]  Yibo Zhang,et al.  Demosaiced pixel super-resolution for multiplexed holographic color imaging , 2016, Scientific Reports.

[26]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[27]  Mike Shires,et al.  Color Management in Digital Pathology , 2014, Analytical Cellular Pathology (Amsterdam).

[28]  A. Ozcan,et al.  Deep learning enables cross-modality super-resolution in fluorescence microscopy , 2018, Nature Methods.

[29]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[30]  Aydogan Ozcan,et al.  Bright-field holography: cross-modality deep learning enables snapshot 3D imaging with bright-field contrast using a single hologram , 2018, Light: Science & Applications.

[31]  Aydogan Ozcan,et al.  Wide-field computational color imaging using pixel super-resolved on-chip microscopy. , 2013, Optics express.

[32]  Yibo Zhang,et al.  Phase recovery and holographic image reconstruction using deep learning in neural networks , 2017, Light: Science & Applications.

[33]  Yibo Zhang,et al.  PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning , 2018, Light: Science & Applications.

[34]  Yibo Zhang,et al.  Wide-field computational imaging of pathology slides using lens-free on-chip microscopy , 2014, Science Translational Medicine.

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  M. S. Peercy,et al.  Wavelength selection for true-color holography. , 1994, Applied optics.

[37]  Danny Pascale,et al.  A Review of RGB Color Spaces , 2003 .

[38]  Aydogan Ozcan,et al.  Lensfree on-chip tomographic microscopy employing multi-angle illumination and pixel super-resolution. , 2012, Journal of visualized experiments : JoVE.

[39]  A. Ozcan,et al.  Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution , 2010, Optics express.

[40]  Aydogan Ozcan,et al.  Resolution enhancement in scanning electron microscopy using deep learning , 2019, Scientific Reports.

[41]  Prarthana Shrestha,et al.  Color accuracy and reproducibility in whole slide imaging scanners , 2014, Journal of medical imaging.