On the use of deep learning for computational imaging

Deep learning has emerged as a class of optimization algorithms proven to be effective for a variety of inference and decision tasks. Similar algorithms, with appropriate modifications, have also been widely adopted for computational imaging. Here, we review the basic tenets of deep learning and computational imaging, and overview recent progress in two applications: super resolution and phase retrieval.

[1]  A. Ozcan,et al.  On the use of deep learning for computational imaging , 2019, Optica.

[2]  Iksung Kang,et al.  Learning to synthesize: robust phase retrieval at low photon counts , 2019, Light, science & applications.

[3]  Yibo Zhang,et al.  Phase recovery and holographic image reconstruction using deep learning in neural networks , 2017, Light: Science & Applications.

[4]  J R Fienup,et al.  Reconstruction of an object from the modulus of its Fourier transform. , 1978, Optics letters.

[5]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[6]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[7]  Iksung Kang,et al.  On the interplay between physical and content priors in deep learning for computational imaging , 2020, Optics express.

[8]  Yibo Zhang,et al.  Deep Learning Microscopy , 2017, ArXiv.

[9]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[10]  I. Daubechies.  Orthonormal bases of compactly supported wavelets , 1988 .

[11]  A. N. Tikhonov,et al.  The regularization of ill-posed problems , 1963 .

[12]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[13]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[14]  Shuai Li,et al.  Lensless computational imaging through deep learning , 2017, ArXiv.

[15]  Fucai Zhang,et al.  Phase extraction neural network (PhENN) with coherent modulation imaging (CMI) for phase retrieval at low photon counts. , 2020, Optics express.

[16]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[17]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[18]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[19]  Yibo Zhang,et al.  Extended depth-of-field in holographic image reconstruction using deep learning based auto-focusing and phase-recovery , 2018, Optica.

[20]  N. Wiener.  The Wiener RMS (Root Mean Square) Error Criterion in Filter Design and Prediction , 1949 .

[21]  Aydogan Ozcan,et al.  Deep learning achieves super-resolution in fluorescence microscopy , 2018, bioRxiv.

[22]  N. Levinson.  A Heuristic Exposition of Wiener's Mathematical Theory of Prediction and Filtering , 1947 .

[23]  Jonas Adler,et al.  Solving ill-posed inverse problems using iterative deep neural networks , 2017, ArXiv.

[24]  A. N. Tikhonov.  The stability of algorithms for the solution of degenerate systems of linear algebraic equations , 1965 .

[25]  øöö Blockinø.  Phase retrieval, error reduction algorithm, and Fienup variants: A view from convex optimization , 2002 .

[26]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  H. H. Hopkins,et al.  The concept of partial coherence in optics , 1951, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[28]  Veit Elser.  Phase retrieval by iterated projections. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[29]  Ali Ramadhan,et al.  Universal Differential Equations for Scientific Machine Learning , 2020, ArXiv.

[30]  I. David Abrahams,et al.  A brief historical perspective of the Wiener–Hopf technique , 2007 .

[31]  Yibo Zhang,et al.  Deep learning-based super-resolution in coherent imaging systems , 2018, Scientific Reports.

[32]  James R. Fienup,et al.  Phase-retrieval stagnation problems and solutions , 1986 .

[33]  George Barbastathis,et al.  Low Photon Count Phase Retrieval Using Deep Learning. , 2018, Physical review letters.

[34]  Song Han,et al.  Deep Generative Adversarial Networks for Compressed Sensing Automates MRI , 2017, ArXiv.

[35]  Antonin Chambolle,et al.  A l1-Unified Variational Framework for Image Restoration , 2004, ECCV.

[36]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[37]  Navid Borhani,et al.  Learning to see through multimode fibers , 2018, Optica.

[38]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[39]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[40]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[41]  Michael Elad,et al.  Image Denoising Via Learned Dictionaries and Sparse representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[42]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[43]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[44]  D. Donoho,et al.  Recurrent Generative Adversarial Networks for Proximal Learning and Automated Compressive Image Recovery , 2017, ArXiv.

[45]  George Biddell Airy,et al.  On the diffraction of an object-glass with circular aperture , 1835 .

[46]  Shaowei Jiang,et al.  Solving Fourier ptychographic imaging problems via neural network modeling and TensorFlow , 2018, Biomedical optics express.

[47]  R. Gerchberg.  A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[48]  Jong Chul Ye,et al.  Deep residual learning for compressed sensing MRI , 2017, 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017).