Deep learning-based accurate and rapid tracking of 3D positional information of microparticles using digital holographic microscopy

Digital holographic microscopy (DHM) encrypts three-dimensional (3D) volumetric information of a test sample in the form of diffraction patterns. Computationally intensive numerical reconstruction and autofocusing procedures, which require precise determination of some parameters, are essential for the extraction of 3D positional information. In this study, a method that can satisfy these requirements by combining digital in-line holographic microscopy (DIHM) with deep learning was proposed. Deep neural networks were trained with tens of thousands of defocused holograms of microparticles moving in a microtube. The in-plane positions of the spherical microparticles were detected by applying Segnet and circular Hough transform, and the depth positions of the microparticles were estimated using a convolutional neural network. The performance of the proposed method was verified by conducting a planar surface experiment and tracking the 3D motions of the particles in a circular microtube. Results indicated that the trained neural networks precisely determined the 3D positions and 3D trajectories of the particles, and the proposed method outperformed conventional methods in measuring 3D positional information in the in-plane and out-of-plane direction. The process time of the proposed method for 3D tracking was only 1.92% of that of the conventional method. The present deep learning-based DIHM method can facilitate the analysis of the 3D dynamic motions of particles or cells and 3D tracking of numerous samples in motion.

[1]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[2]  Hayit Greenspan,et al.  Automated analysis of individual sperm cells using stain‐free interferometric phase microscopy and machine learning , 2017, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[3]  Yibo Zhang,et al.  Phase recovery and holographic image reconstruction using deep learning in neural networks , 2017, Light: Science & Applications.

[4]  J. Katz,et al.  Applications of Holography in Fluid Mechanics and Particle Dynamics , 2010 .

[5]  Shigeru Murata,et al.  Potential of digital holography in particle measurement , 2000 .

[6]  Yibo Zhang,et al.  Deep Learning Microscopy , 2017, ArXiv.

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Zachary S. Ballard,et al.  Air quality monitoring using mobile microscopy and machine learning , 2017, Light: Science & Applications.

[9]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jonghee Yoon,et al.  Holographic deep learning for rapid optical screening of anthrax spores , 2017, Science Advances.

[11]  Xiangnan Wang,et al.  Fast phase retrieval in off-axis digital holographic microscopy through deep learning. , 2018, Optics express.

[12]  Aydogan Ozcan,et al.  Label-free 3D computational imaging of spermatozoon locomotion, head spin and flagellum beating over a large volume , 2017, Light: Science & Applications.

[13]  Pradipta Kumar Panigrahi,et al.  Three-dimensional investigation of liquid slug Taylor flow inside a micro-capillary using holographic velocimetry , 2015 .

[14]  Yibo Zhang,et al.  Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery , 2018, Optica.

[15]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[16]  Paul Kumar Upputuri,et al.  Super-resolution Photoacoustic Microscopy Using Near-Field Localization by a Plasmonic Metal Nanoaperture: A Simulation Study , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[17]  H. Greenspan,et al.  Quantitative phase microscopy spatial signatures of cancer cells , 2017, Cytometry. Part A : the journal of the International Society for Analytical Cytology.

[18]  Bahram Javidi,et al.  Automatic cell identification and visualization using digital holographic microscopy with head mounted augmented reality devices: An Overview , 2018, 2018 17th Workshop on Information Optics (WIO).

[19]  Sang Joon Lee,et al.  Three-dimensional swimming motility of microorganism in the near-wall region , 2016 .

[20]  Derek K. Tseng,et al.  Detection of waterborne parasites using field-portable and cost-effective lensfree microscopy. , 2010, Lab on a chip.

[21]  Tomi Pitkäaho,et al.  Focus prediction in digital holographic microscopy using deep convolutional neural networks. , 2019, Applied optics.

[22]  Yong-Seok Choi,et al.  Advances in digital holographic micro-PTV for analyzing microscale flows , 2012 .

[23]  Bahram Javidi,et al.  Automated red blood cells extraction from holographic images using fully convolutional neural networks. , 2017, Biomedical optics express.

[24]  Aydogan Ozcan,et al.  Edge sparsity criterion for robust holographic autofocusing. , 2017, Optics letters.

[25]  Sang Joon Lee,et al.  Deep learning-based digital in-line holographic microscopy for high resolution with extended field of view , 2019, Optics & Laser Technology.

[26]  Heung-Il Suk,et al.  Deep Learning in Medical Image Analysis. , 2017, Annual review of biomedical engineering.

[27]  Pasquale Memmolo,et al.  Recent advances in holographic 3D particle tracking , 2015 .

[28]  Van Lam,et al.  Automatic phase aberration compensation for digital holographic microscopy based on deep learning background detection. , 2017, Optics express.

[29]  Bo Sun,et al.  Flow visualization and flow cytometry with holographic video microscopy. , 2009 .

[30]  J. Chi,et al.  Automated Detection of P. falciparum Using Machine Learning Algorithms with Quantitative Phase Images of Unstained Cells , 2016, PloS one.

[31]  Edmund Y. Lam,et al.  Autofocusing in digital holography using deep learning , 2018, BiOS.

[32]  Andrea Vedaldi,et al.  MatConvNet: Convolutional Neural Networks for MATLAB , 2014, ACM Multimedia.

[33]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[34]  Shuai Li,et al.  Lensless computational imaging through deep learning , 2017, ArXiv.

[35]  Joseph Katz,et al.  Digital holographic microscopy reveals prey-induced changes in swimming behavior of predatory dinoflagellates , 2007, Proceedings of the National Academy of Sciences.

[36]  YongKeun Park,et al.  Identification of non-activated lymphocytes using three-dimensional refractive index tomography and machine learning , 2017, Scientific Reports.

[37]  Sang Joon Lee,et al.  Label-free sensor for automatic identification of erythrocytes using digital in-line holographic microscopy and machine learning. , 2018, Biosensors & bioelectronics.

[38]  Sang Joon Lee,et al.  A new two-frame particle tracking algorithm using match probability , 1996 .

[39]  Jun Tanida,et al.  Deep-learning-generated holography. , 2018, Applied optics.

[40]  Myong Hwan Sohn,et al.  Three-dimensional motion measurements of free-swimming microorganisms using digital holographic microscopy , 2011 .

[41]  Yong-Seok Choi,et al.  Three-dimensional volumetric measurement of red blood cell motion using digital holographic microscopy. , 2009, Applied optics.

[42]  Taesik Go,et al.  Machine learning‐based in‐line holographic sensing of unstained malaria‐infected red blood cells , 2018, Journal of biophotonics.

[43]  Edmund Y. Lam,et al.  Learning-based nonparametric autofocusing for digital holography , 2018 .

[44]  Sang Joon Lee,et al.  High-accuracy measurement of depth-displacement using a focus function and its cross- correlation in holographic PTV. , 2014, Optics express.

[45]  Hyun-seok Min,et al.  Quantitative Phase Imaging and Artificial Intelligence: A Review , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[46]  Christian Depeursinge,et al.  Quantitative phase imaging in biomedicine , 2018, Nature Photonics.

[47]  Sang Joon Lee,et al.  Focusing and alignment of erythrocytes in a viscoelastic medium , 2017, Scientific Reports.

[48]  Myung K. Kim,et al.  Wavelength-scanning digital interference holography for tomographic three-dimensional imaging by use of the angular spectrum method. , 2005, Optics letters.

[49]  Roman Stocker,et al.  Failed escape: solid surfaces prevent tumbling of Escherichia coli. , 2014, Physical review letters.

[50]  Julia Lobera,et al.  Holography, tomography and 3D microscopy as linear filtering operations , 2008 .

[51]  P. K. Panigrahi,et al.  Improved digital holographic reconstruction algorithm for depth error reduction and elimination of out-of-focus particles. , 2010, Optics express.

[52]  Bahram Javidi,et al.  Cell morphology-based classification of red blood cells using holographic imaging informatics. , 2016, Biomedical optics express.

[53]  Hui Meng,et al.  Digital holography of particle fields: reconstruction by use of complex amplitude. , 2003, Applied optics.

[54]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.