Deep learning-based digital in-line holographic microscopy for high resolution with extended field of view

Abstract A digital in-line holographic microscopy (DIHM) with deep learning-based upscaling method is proposed to overcome the trade-off between high resolving power and large field-of-view (FOV). To enhance the spatial resolution of a hologram, a deep neural network was trained with hologram images, which are defocused images with diffraction patterns. The performance of the artificial intelligence-based DIHM method was verified using hologram images obtained by computer simulation and experiments. Upscaled holograms with enhanced image contrast and clear diffraction pattern provided high quality of reconstructed holograms. In addition to the enhancement of reconstructed image at the sample position, details of light scattering pattern could be revealed with the proposed method. The proposed deep learning-based DIHM method is promising for accurate monitoring of many samples and analyzing dynamics of particles or cells in large FOV with detailed 3D information reconstructed from the upscaled holograms.

[1]  Matthew M. Crane,et al.  Automated on-chip rapid microscopy, phenotyping and sorting of C. elegans , 2008, Nature Methods.

[2]  Joseph Katz,et al.  On the refractive index of sodium iodide solutions for index matching in PIV , 2014 .

[3]  D. Grier,et al.  Rotational and translational diffusion of copper oxide nanorods measured with holographic video microscopy. , 2010, Optics express.

[4]  Taesik Go,et al.  Machine learning‐based in‐line holographic sensing of unstained malaria‐infected red blood cells , 2018, Journal of biophotonics.

[5]  Geert Vanmeerbeeck,et al.  Three-part differential of unlabeled leukocytes with a compact lens-free imaging flow cytometer. , 2015, Lab on a chip.

[6]  P Memmolo,et al.  Particle tracking by full-field complex wavefront subtraction in digital holography microscopy. , 2014, Lab on a chip.

[7]  Yibo Zhang,et al.  Phase recovery and holographic image reconstruction using deep learning in neural networks , 2017, Light: Science & Applications.

[8]  Youngchan Kim,et al.  Common-path diffraction optical tomography for investigation of three-dimensional structures and dynamics of biological cells. , 2014, Optics express.

[9]  Mark D Hannel,et al.  Machine-learning techniques for fast and accurate feature localization in holograms of colloidal particles. , 2018, Optics express.

[10]  S. I. Abdel-Khalik,et al.  A simple model for the refractive index of sodium iodide aqueous solutions , 2000 .

[11]  A. Ozcan,et al.  Holographic pixel super-resolution in portable lensless on-chip microscopy using a fiber-optic array. , 2011, Lab on a chip.

[12]  S. Lee,et al.  Hybrid bright-field and hologram imaging of cell dynamics , 2016, Scientific Reports.

[13]  Sang Joon Lee,et al.  Precise measurement of orientations of transparent ellipsoidal particles through digital holographic microscopy. , 2016, Optics express.

[14]  W Xu,et al.  Digital in-line holography for biological applications , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[15]  A. Keller,et al.  Preparation of monodisperse ellipsoidal polystyrene particles , 1993 .

[16]  Yong-Seok Choi,et al.  Advances in digital holographic micro-PTV for analyzing microscale flows , 2012 .

[17]  Lijuan Su,et al.  Multiframe super resolution reconstruction method based on light field angular images , 2017 .

[18]  Peter Klages,et al.  Digital in-line holographic microscopy. , 2006 .

[19]  Xiangnan Wang,et al.  Fast phase retrieval in off-axis digital holographic microscopy through deep learning. , 2018, Optics express.

[20]  J. Katz,et al.  Applications of Holography in Fluid Mechanics and Particle Dynamics , 2010 .

[21]  Yibo Zhang,et al.  Deep Learning Microscopy , 2017, ArXiv.

[22]  Sang Joon Lee,et al.  Precise measurement of three-dimensional positions of transparent ellipsoidal particles using digital holographic microscopy. , 2015, Applied optics.

[23]  YongKeun Park,et al.  Refractive index tomograms and dynamic membrane fluctuations of red blood cells from patients with diabetes mellitus , 2016, Scientific Reports.

[24]  Paul J. Choi,et al.  Quantifying E. coli Proteome and Transcriptome with Single-Molecule Sensitivity in Single Cells , 2010, Science.

[25]  Pasquale Memmolo,et al.  On the holographic 3D tracking of in vitro cells characterized by a highly-morphological change. , 2012, Optics express.

[26]  E. Franses,et al.  Preparation and characterization of monodisperse polymer microspheroids , 1991 .

[27]  Three-dimensional rotational dynamics of prolate particles in a circular tube , 2018 .

[28]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Van Lam,et al.  Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection. , 2017, Optics express.

[30]  Guohai Situ,et al.  eHoloNet: a learning-based end-to-end approach for in-line digital holographic reconstruction. , 2018, Optics express.

[31]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[32]  Eros Comunello,et al.  Portable lensless wide-field microscopy imaging platform based on digital inline holography and multi-frame pixel super-resolution , 2015, Light: Science & Applications.

[33]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[34]  J. Goodman.  Introduction to Fourier optics , 1969 .

[35]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[36]  Edmund Y. Lam,et al.  Learning-based nonparametric autofocusing for digital holography , 2018 .

[37]  Pasquale Memmolo,et al.  Tomographic flow cytometry by digital holography , 2016, Light: Science & Applications.

[38]  Hao Yu,et al.  Machine Learning Based Single-Frame Super-Resolution Processing for Lensless Blood Cell Counting , 2016, Sensors.

[39]  Aydogan Ozcan,et al.  Field-portable reflection and transmission microscopy based on lensless holography , 2011, Biomedical optics express.

[40]  Pasquale Memmolo,et al.  Recent advances in holographic 3D particle tracking , 2015 .

[41]  Franck Nicoud,et al.  Red cells’ dynamic morphologies govern blood shear thinning under microcirculatory flow conditions , 2016, Proceedings of the National Academy of Sciences.

[42]  Aydogan Ozcan,et al.  Lensless digital holographic microscopy and its applications in biomedicine and environmental monitoring. , 2017, Methods.

[43]  Yibo Zhang,et al.  Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery , 2018, Optica.

[44]  W. Ristenpart,et al.  Mechanical response of red blood cells entering a constriction. , 2013, Biomicrofluidics.

[45]  Pilhan Kim,et al.  Holographic intravital microscopy for 2-D and 3-D imaging intact circulating blood cells in microcapillaries of live mice , 2016, Scientific Reports.

[46]  Derek Tseng,et al.  Compact, light-weight and cost-effective microscope based on lensless incoherent holography for telemedicine applications. , 2010, Lab on a chip.

[47]  R. Dasari,et al.  Diffraction phase microscopy for quantifying cell structure and dynamics. , 2006, Optics letters.

[48]  Samir Mitragotri,et al.  Role of target geometry in phagocytosis. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[49]  Jonghee Yoon,et al.  Holographic deep learning for rapid optical screening of anthrax spores , 2017, Science Advances.