Deep learning-based super-resolution in coherent imaging systems

We present a deep learning framework based on a generative adversarial network (GAN) to perform super-resolution in coherent imaging systems. We demonstrate that this framework can enhance the resolution of both pixel size-limited and diffraction-limited coherent imaging systems. The capabilities of this approach are experimentally validated by super-resolving complex-valued images acquired using a lensfree on-chip holographic microscope, the resolution of which was pixel size-limited. Using the same GAN-based approach, we also improved the resolution of a lens-based holographic imaging system that was limited in resolution by the numerical aperture of its objective lens. This deep learning-based super-resolution framework can be broadly applied to enhance the space-bandwidth product of coherent imaging systems using image data and convolutional neural networks, and provides a rapid, non-iterative method for solving inverse image reconstruction or enhancement problems in optics.

[1]  Zeev Zalevsky,et al.  Superresolved imaging in digital holography by superposition of tilted wavefronts. , 2006, Applied optics.

[2]  Yibo Zhang,et al.  Wide-field computational imaging of pathology slides using lens-free on-chip microscopy , 2014, Science Translational Medicine.

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  D. Gabor.  A New Microscopic Principle , 1948, Nature.

[5]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[6]  M. Teague.  Deterministic phase retrieval: a Green’s function solution , 1983 .

[7]  A. Ozcan,et al.  Deep learning enables cross-modality super-resolution in fluorescence microscopy , 2018, Nature Methods.

[8]  P. Ferraro,et al.  Super-resolution in digital holography by a two-dimensional dynamic phase grating. , 2008, Optics express.

[9]  A. Ozcan,et al.  Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy , 2012, Optics Express.

[10]  Yibo Zhang,et al.  Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery , 2018, Optica.

[11]  Aydogan Ozcan,et al.  Deep learning achieves super-resolution in fluorescence microscopy , 2018, bioRxiv.

[12]  E. Leith,et al.  Reconstructed Wavefronts and Communication Theory , 1962 .

[13]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[14]  Aydogan Ozcan,et al.  Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy , 2012, Nature Methods.

[15]  Aydogan Ozcan,et al.  Edge sparsity criterion for robust holographic autofocusing. , 2017, Optics letters.

[16]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Tomer Michaeli,et al.  Deep-STORM: super-resolution single-molecule microscopy by deep learning , 2018, 1801.09631.

[18]  Aydogan Ozcan,et al.  Lensless digital holographic microscopy and its applications in biomedicine and environmental monitoring. , 2017, Methods.

[19]  Zeev Zalevsky,et al.  Synthetic aperture superresolution with multiple off-axis holograms. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[20]  Leslie J. Allen,et al.  Phase retrieval from series of images obtained by defocus variation , 2001 .

[21]  A. Ozcan,et al.  Lensfree on-chip microscopy over a wide field-of-view using pixel super-resolution , 2010, Optics express.

[22]  Yibo Zhang,et al.  Deep learning enhanced mobile-phone microscopy , 2017, ACS Photonics.

[23]  Thierry Fournel,et al.  Pixel super-resolution in digital holography by regularized reconstruction , 2017 .

[24]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[25]  A. Ozcan,et al.  Pixel super-resolution using wavelength scanning , 2015, Light: Science & Applications.

[26]  Shuai Li,et al.  Lensless computational imaging through deep learning , 2017, ArXiv.

[27]  Aydogan Ozcan,et al.  Field-portable wide-field microscopy of dense samples using multi-height pixel super-resolution based lensfree imaging. , 2012, Lab on a chip.

[28]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Yibo Zhang,et al.  Deep Learning Microscopy , 2017, ArXiv.

[30]  J. Walkup,et al.  Statistical optics , 1986, IEEE Journal of Quantum Electronics.

[31]  Aydogan Ozcan,et al.  Bright-field holography: cross-modality deep learning enables snapshot 3D imaging with bright-field contrast using a single hologram , 2018, Light: Science & Applications.

[32]  Barry R. Masters,et al.  Quantitative Phase Imaging of Cells and Tissues , 2012 .

[33]  Wei Luo,et al.  Propagation phasor approach for holographic image reconstruction , 2016, Scientific Reports.

[34]  Yibo Zhang,et al.  Sparsity-based multi-height phase recovery in holographic microscopy , 2016, Scientific Reports.

[35]  Zeev Zalevsky,et al.  Compressive Fresnel holography approach for high-resolution viewpoint inference. , 2015, Optics letters.

[36]  Yonina C. Eldar,et al.  Sparsity-based single-shot sub-wavelength coherent diffractive imaging , 2011, 2012 IEEE 27th Convention of Electrical and Electronics Engineers in Israel.

[37]  B. Javidi,et al.  Super Resolved Holographic Configurations , 2014 .

[38]  Christophe Zimmer,et al.  Deep learning massively accelerates super-resolution localization microscopy , 2018, Nature Biotechnology.

[39]  I. Yamaguchi,et al.  Phase-shifting digital holography. , 1997, Optics letters.

[40]  Yibo Zhang,et al.  Phase recovery and holographic image reconstruction using deep learning in neural networks , 2017, Light: Science & Applications.

[41]  Lei Tian,et al.  Compressive holographic two-dimensional localization with 1/30(2) subpixel accuracy. , 2014, Optics express.

[42]  J. Goodman.  Introduction to Fourier optics , 1969 .

[43]  J. Rogers,et al.  Spatial light interference microscopy (SLIM) , 2010, IEEE Photonic Society 24th Annual Meeting.

[44]  R. Horstmeyer,et al.  Wide-field, high-resolution Fourier ptychographic microscopy , 2013, Nature Photonics.