PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning

Using a deep neural network, we demonstrate a digital staining technique, which we term PhaseStain, to transform the quantitative phase images (QPI) of label-free tissue sections into images that are equivalent to the brightfield microscopy images of the same samples that are histologically stained. Through pairs of image data (QPI and the corresponding brightfield images, acquired after staining), we train a generative adversarial network and demonstrate the effectiveness of this virtual-staining approach using sections of human skin, kidney, and liver tissue, matching the brightfield microscopy images of the same samples stained with Hematoxylin and Eosin, Jones’ stain, and Masson’s trichrome stain, respectively. This digital-staining framework may further strengthen various uses of label-free QPI techniques in pathology applications and biomedical research in general, by eliminating the need for histological staining, reducing sample preparation related costs and saving time. Our results provide a powerful example of some of the unique opportunities created by data-driven image transformations enabled by deep learning.Microscopy: AI brings color to quantitative phase images of pathology samplesCombining a technique called quantitative phase microscopy with a trained neural network avoids the need to use chemical stains to visualize constituents of biological tissue samples. Quantitative phase microscopy reveals the features of a label-free specimen by analyzing changes in the phase of light waves as they interact with different regions of a sample. Aydogan Ozcan, Yair Rivenson and colleagues at the University of California, Los Angeles, USA, trained a neural network to transform phase microscopy images of human skin, kidney and liver tissues into images that matched those obtained using conventional histological staining techniques used in pathology. This virtual-staining approach, which the authors termed PhaseStain, offers easier, faster and cheaper high quality analysis of tissue samples for use in biology and medicine.

[1]  J. Goodman.  Introduction to Fourier optics , 1969 .

[2]  J. Rogers,et al.  Spatial light interference microscopy (SLIM) , 2010, IEEE Photonic Society 24th Annual Meeting.

[3]  Aleksey Boyko,et al.  Detecting Cancer Metastases on Gigapixel Pathology Images , 2017, ArXiv.

[4]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[5]  Barry R. Masters,et al.  Quantitative Phase Imaging of Cells and Tissues , 2012 .

[6]  H. Greenspan,et al.  Quantitative phase microscopy spatial signatures of cancer cells , 2017, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[7]  Wei Luo,et al.  Propagation phasor approach for holographic image reconstruction , 2016, Scientific Reports.

[8]  Natan T Shaked,et al.  Reflective interferometric chamber for quantitative phase imaging of biological sample dynamics. , 2010, Journal of biomedical optics.

[9]  Tomer Michaeli,et al.  Deep-STORM: super-resolution single-molecule microscopy by deep learning , 2018, 1801.09631.

[10]  A. Ozcan,et al.  Deep learning enables cross-modality super-resolution in fluorescence microscopy , 2018, Nature Methods.

[11]  James G. Fujimoto,et al.  Assessment of breast pathologies using nonlinear microscopy , 2014, Proceedings of the National Academy of Sciences.

[12]  R. Horstmeyer,et al.  Wide-field, high-resolution Fourier ptychographic microscopy , 2013, Nature Photonics.

[13]  Zhuo Wang,et al.  Tissue refractive index as marker of disease. , 2011, Journal of biomedical optics.

[14]  Jonghee Yoon,et al.  Holographic deep learning for rapid optical screening of anthrax spores , 2017, Science Advances.

[15]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[16]  Aydogan Ozcan,et al.  Deep learning achieves super-resolution in fluorescence microscopy , 2018, bioRxiv.

[17]  Yibo Zhang,et al.  Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery , 2018, Optica.

[18]  Bahram Javidi,et al.  High-precision microscopic phase imaging without phase unwrapping , 2013 .

[19]  Bahram Javidi,et al.  Sickle cell disease diagnosis based on spatio-temporal cell dynamics analysis using 3D printed shearing digital holographic microscopy. , 2018, Optics express.

[20]  Aydogan Ozcan,et al.  Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy , 2012, Nature Methods.

[21]  Aydogan Ozcan,et al.  Edge sparsity criterion for robust holographic autofocusing. , 2017, Optics letters.

[22]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  John B. A. Okello,et al.  Preparation of Formalin-fixed Paraffin-embedded Tissue Cores for both RNA and DNA Extraction , 2016, Journal of visualized experiments : JoVE.

[24]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[25]  Yibo Zhang,et al.  Wide-field computational imaging of pathology slides using lens-free on-chip microscopy , 2014, Science Translational Medicine.

[26]  Zhuo Wang,et al.  Scattering-phase theorem. , 2011, Optics letters.

[27]  Lei Tian,et al.  Deep learning approach to Fourier ptychographic microscopy , 2018, 1805.00334.

[28]  Benjamin Recht,et al.  DeepLoco: Fast 3D Localization Microscopy Using Neural Networks , 2018, bioRxiv.

[29]  Joachim Hornegger,et al.  Virtual Hematoxylin and Eosin Transillumination Microscopy Using Epi-Fluorescence Imaging , 2016, PloS one.

[30]  Lei Tian,et al.  A deep-learning approach for high-speed Fourier ptychographic microscopy , 2018 .

[31]  Christophe Zimmer,et al.  Deep learning massively accelerates super-resolution localization microscopy , 2018, Nature Biotechnology.

[32]  Gabriel Popescu,et al.  Hilbert phase microscopy for investigating fast dynamics in transparent systems. , 2005, Optics letters.

[33]  Yibo Zhang,et al.  Deep Learning Microscopy , 2017, ArXiv.

[34]  Shuai Li,et al.  Lensless computational imaging through deep learning , 2017, ArXiv.

[35]  Shie Mannor,et al.  Deep Learning Reconstruction of Ultrashort Pulses , 2018, 2018 Conference on Lasers and Electro-Optics (CLEO).

[36]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 , 2015, Lecture Notes in Computer Science.

[37]  A. Ozcan,et al.  Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution , 2010, Optics express.

[38]  Yibo Zhang,et al.  Deep learning enhanced mobile-phone microscopy , 2017, ACS Photonics.

[39]  Lei Tian,et al.  Convolutional neural network for Fourier ptychography video reconstruction: learning temporal dynamics from spatial ensembles , 2018, ArXiv.

[40]  Ata Mahjoubfar,et al.  Deep Learning in Label-free Cell Classification , 2016, Scientific Reports.

[41]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[42]  Michael Unser,et al.  Learning approach to optical tomography , 2015, 1502.01914.

[43]  Hyun-seok Min,et al.  Quantitative Phase Imaging and Artificial Intelligence: A Review , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[44]  L. Tian,et al.  Quantitative differential phase contrast imaging in an LED array microscope. , 2015, Optics express.

[45]  Yibo Zhang,et al.  Phase recovery and holographic image reconstruction using deep learning in neural networks , 2017, Light: Science & Applications.

[46]  J. Chi,et al.  Automated Detection of P. falciparum Using Machine Learning Algorithms with Quantitative Phase Images of Unstained Cells , 2016, PloS one.

[47]  A. Ozcan,et al.  Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy , 2012, Optics Express.

[48]  A. Ozcan,et al.  Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning , 2018, Nature Biomedical Engineering.

[49]  Etienne Rouleau,et al.  Guidance for laboratories performing molecular pathology for cancer patients , 2014, Journal of Clinical Pathology.

[50]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[51]  E. Cuche,et al.  Simultaneous amplitude-contrast and quantitative phase-contrast microscopy by numerical reconstruction of Fresnel off-axis holograms. , 1999, Applied optics.

[52]  Todd C. Hollon,et al.  Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy , 2017, Nature Biomedical Engineering.

[53]  David A. Jaffray,et al.  Wide-field tissue polarimetry allows efficient localized mass spectrometry imaging of biological tissues , 2015, Chemical science.

[54]  Natan T Shaked,et al.  Dual-interference-channel quantitative-phase microscopy of live cell dynamics. , 2009, Optics letters.