HoloStain: Holographic virtual staining of individual biological cells

Many medical and biological protocols for analyzing individual biological cells involve morphological evaluation based on cell staining, designed to enhance imaging contrast and enable clinicians and biologists to differentiate between various cell organelles. However, cell staining is not always allowed in certain medical procedures. In other cases, staining may be time consuming or expensive to implement. Here, we present a new deep-learning approach, called HoloStain, which converts images of isolated biological cells acquired without staining by holographic microscopy to their virtually stained images. We demonstrate this approach for human sperm cells, as there is a well-established protocol and global standardization for characterizing the morphology of stained human sperm cells for fertility evaluation, but, on the other hand, staining might be cytotoxic and thus is not allowed during human in vitro fertilization (IVF). We use deep convolutional Generative Adversarial Networks (DCGANs) with training that is based on both the quantitative phase images and two gradient phase images, all extracted from the digital holograms of the stain-free cells, with the ground truth of bright-field images of the same cells that subsequently underwent chemical staining. To validate the quality of our virtual staining approach, an experienced embryologist analyzed the unstained cells, the virtually stained cells, and the chemically stained sperm cells several times in a blinded and randomized manner. We obtained a 5-fold recall (sensitivity) improvement in the analysis results. With the introduction of simple holographic imaging methods in clinical settings, the proposed method has a great potential to become a common practice in human IVF procedures, as well as to significantly simplify and facilitate other cell analyses and techniques such as imaging flow cytometry.

[1]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[2]  N. Shaked,et al.  Compact and portable low-coherence interferometer with off-axis geometry for quantitative phase microscopy and nanoscopy. , 2013, Optics express.

[3]  Pinhas Girshovitz,et al.  Fast phase processing in off-axis holography using multiplexing with complex encoding and live-cell fluctuation map calculation in real-time. , 2015, Optics express.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Aydogan Ozcan,et al.  Field-portable reflection and transmission microscopy based on lensless holography , 2011, Biomedical optics express.

[7]  Christian Depeursinge,et al.  Quantitative phase imaging in biomedicine , 2018, Nature Photonics.

[8]  Pinhas Girshovitz,et al.  Generalized cell morphological parameters based on interferometric phase microscopy and their application to cell life cycle characterization , 2012, Biomedical optics express.

[9]  Natan T. Shaked,et al.  TOP-GAN: Label-Free Cancer Cell Classification Using Deep Learning with a Small Training Set , 2018, ArXiv.

[10]  Krzysztof Krawiec,et al.  Segmenting Retinal Blood Vessels With Deep Neural Networks , 2016, IEEE Transactions on Medical Imaging.

[11]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Shu Liao,et al.  Representation Learning: A Unified Deep Learning Framework for Automatic Prostate MR Segmentation , 2013, MICCAI.

[13]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[14]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[15]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[16]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[17]  Christopher Joseph Pal,et al.  Brain tumor segmentation with Deep Neural Networks , 2015, Medical Image Anal..

[18]  J. Goodman Introduction to Fourier optics , 1969 .

[19]  Yibo Zhang,et al.  PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning , 2018, Light: Science & Applications.

[20]  Ata Mahjoubfar,et al.  Deep Learning in Label-free Cell Classification , 2016, Scientific Reports.

[21]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[24]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Ronald M. Summers,et al.  Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique , 2016 .

[26]  Robert C. Wolpert,et al.  A Review of the , 1985 .