Phase Retrieval with Holography and Untrained Priors: Tackling the Challenges of Low-Photon Nanoscale Imaging

Phase retrieval is the inverse problem of recovering a signal from magnitude-only Fourier measurements, and underlies numerous imaging modalities, such as Coherent Diffraction Imaging (CDI). A variant of this setup, known as holography, includes a reference object that is placed adjacent to the specimen of interest before measurements are collected. The resulting inverse problem, known as holographic phase retrieval, is well-known to have improved problem conditioning relative to the original. This innovation, i.e. Holographic CDI, becomes crucial at the nanoscale, where imaging specimens such as viruses, proteins, and crystals require low-photon measurements. This data is highly corrupted by Poisson shot noise, and often lacks low-frequency content as well. In this work, we introduce a dataset-free deep learning framework for holographic phase retrieval adapted to these challenges. The key ingredients of our approach are the explicit and flexible incorporation of the physical forward model into an automatic differentiation procedure, the Poisson log-likelihood objective function, and an optional untrained deep image prior. We perform extensive evaluation under realistic conditions. Compared to competing classical methods, our method recovers signal from higher noise levels and is more resilient to suboptimal reference design, as well as to large missing regions of low frequencies in the observations. To the best of our knowledge, this is the first work to consider a dataset-free machine learning approach for holographic phase retrieval.

[1]  Reinhard Heckel,et al.  Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks , 2018, ICLR.

[2]  George Barbastathis,et al.  Low Photon Count Phase Retrieval Using Deep Learning. , 2018, Physical review letters.

[3]  Stefan Harmeling,et al.  Phase Retrieval using Conditional Generative Adversarial Networks , 2019, ArXiv.

[4]  M. Hayes The reconstruction of a multidimensional sequence from the phase or magnitude of its Fourier transform , 1982 .

[5]  Oliver Cossairt,et al.  High dynamic range coherent imaging using compressed sensing. , 2015, Optics express.

[6]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[7]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[8]  D. Gabor A New Microscopic Principle , 1948, Nature.

[9]  Youssef S. G. Nashed,et al.  Distributed Automatic Differentiation for Ptychography , 2017, ICCS.

[10]  F. Krzakala,et al.  Precise asymptotics for phase retrieval and compressed sensing with random generative priors , 2019 .

[11]  Yibo Zhang,et al.  Phase recovery and holographic image reconstruction using deep learning in neural networks , 2017, Light: Science & Applications.

[12]  Baoshun Shi,et al.  Deep prior-based sparse representation model for diffraction imaging: A plug-and-play method , 2020, Signal Process..

[13]  Ali Ahmed,et al.  Robust Compressive Phase Retrieval via Deep Generative Priors , 2018, ArXiv.

[14]  Tatiana Latychevskaia,et al.  Reconstruction of missing information in diffraction patterns and holograms by iterative phase retrieval , 2019, Optics Communications.

[15]  Alexandros G. Dimakis,et al.  Deep Learning Techniques for Inverse Problems in Imaging , 2020, IEEE Journal on Selected Areas in Information Theory.

[16]  Charles L. Epstein,et al.  Recovering missing data in coherent diffraction imaging , 2020, SIAM J. Imaging Sci..

[17]  S. Kikuta,et al.  X-ray holography of lensless Fourier-transform type , 1972 .

[18]  Michael Elad,et al.  The Little Engine That Could: Regularization by Denoising (RED) , 2016, SIAM J. Imaging Sci..

[19]  Xiaodong Li,et al.  Phase Retrieval via Wirtinger Flow: Theory and Algorithms , 2014, IEEE Transactions on Information Theory.

[20]  Michael Möller,et al.  Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Jason W. Fleischer,et al.  When deep denoising meets iterative phase retrieval , 2020, ICML.

[22]  James R Fienup,et al.  Applications of algorithmic differentiation to phase retrieval algorithms. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[23]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[24]  Richard G. Baraniuk,et al.  prDeep: Robust Phase Retrieval with a Flexible Deep Network , 2018, ICML.

[25]  Florent Krzakala,et al.  Inferring sparsity: Compressed sensing using generalized restricted Boltzmann machines , 2016, 2016 IEEE Information Theory Workshop (ITW).

[26]  L. Pelkmans,et al.  Ameobal Pathogen Mimivirus Infects Macrophages through Phagocytosis , 2008, PLoS pathogens.

[27]  P. Thibault,et al.  Maximum-likelihood refinement for coherent diffractive imaging , 2012 .

[28]  Joshua W. Shaevitz,et al.  Massively parallel X-ray holography , 2008 .

[29]  Anton Barty,et al.  Femtosecond X-ray Fourier holography imaging of free-flying nanoparticles , 2017, 1707.09424.

[30]  Chinmay Hegde,et al.  Phase Retrieval using Untrained Neural Network Priors , 2019 .

[31]  David A. Barmherzig,et al.  Low-Photon Holographic Phase Retrieval , 2020 .

[32]  Yonina C. Eldar,et al.  Phase Retrieval with Application to Optical Imaging: A contemporary overview , 2015, IEEE Signal Processing Magazine.

[33]  Vladislav Voroninski,et al.  Phase Retrieval Under a Generative Prior , 2018, NeurIPS.

[34]  George Barbastathis,et al.  Phase imaging with an untrained neural network , 2020, Light: Science & Applications.

[35]  Michael Unser,et al.  Convolutional Neural Networks for Inverse Problems in Imaging: A Review , 2017, IEEE Signal Processing Magazine.

[36]  J. Miao,et al.  Extending the methodology of X-ray crystallography to allow imaging of micrometre-sized non-crystalline specimens , 1999, Nature.

[37]  Laura Waller,et al.  Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network , 2020, Optica.

[38]  Chao Yang,et al.  Alternating direction methods for classical and ptychographic phase retrieval , 2012 .

[39]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[40]  T. Latychevskaia,et al.  Fourier Transform Holography: A Lensless Non-Destructive Imaging Technique , 2012, Microscopy and Microanalysis.

[41]  Chris Jacobsen,et al.  Using automatic differentiation as a general framework for ptychographic reconstruction. , 2019, Optics express.

[42]  J R Fienup,et al.  Reconstruction of an object from the modulus of its Fourier transform. , 1978, Optics letters.

[43]  Manuel Guizar-Sicairos,et al.  Holography with extended reference by autocorrelation linear differential operation. , 2007, Optics express.