Toward Long-Distance Subdiffraction Imaging Using Coherent Camera Arrays

In this work, we propose using camera arrays coupled with coherent illumination as an effective method of improving spatial resolution in long distance images by a factor often and beyond. Recent advances in ptychography have demonstrated that one can image beyond the diffraction limit of the objective lens in a microscope. We demonstrate a similar imaging system to image beyond the diffraction limit in long range imaging. We emulate a camera array with a single camera attached to an XY translation stage. We show that an appropriate phase retrieval based reconstruction algorithm can be used to effectively recover the lost high resolution details from the multiple low resolution acquired images. We analyze the effects of noise, required degree of image overlap, and the effect of increasing synthetic aperture size on the reconstructed image quality. We show that coherent camera arrays have the potential to greatly improve imaging performance. Our simulations show resolution gains of 10× and more are achievable. Furthermore, experimental results from our proof-of-concept systems show resolution gains of 4 × -7× for real scenes. All experimental data and code is made publicly available on the project webpage. Finally, we introduce and analyze in simulation a new strategy to capture macroscopic Fourier Ptychography images in a single snapshot, albeit using a camera array.

[1]  Walter A. Grundmann,et al.  AN IMAGE STABILIZATION HIGH-RESOLUTION CAMERA FOR THE CANADA-FRANCE-HAWAII TELESCOPE , 1989 .

[2]  Christoph Rau,et al.  Sampling in x-ray ptychography , 2013 .

[3]  James R. Fienup,et al.  Phase-retrieval stagnation problems and solutions , 1986 .

[4]  Siyuan Dong,et al.  Spectral multiplexing and coherent-state decomposition in Fourier ptychographic imaging. , 2014, Biomedical optics express.

[5]  Frédo Durand,et al.  Motion-invariant photography , 2008, ACM Trans. Graph..

[6]  Kaikai Guo,et al.  Optimization of sampling pattern and the design of Fourier ptychographic illuminator. , 2015, Optics express.

[7]  S. T. Ridgway,et al.  First Results from the CHARA Array. II. A Description of the Instrument , 2005 .

[8]  Bruce H. Pillman Super-Resolution Imaging , 2013, J. Electronic Imaging.

[9]  Michael F. Cohen,et al.  Seeing Mt. Rainier: Lucky imaging for multi-image denoising, sharpening, and haze removal , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[10]  A W Lohmann,et al.  Scaling laws for lens systems. , 1989, Applied optics.

[11]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[12]  Harry Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[14]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[15]  B. Javidi,et al.  Synthetic aperture single-exposure on-axis digital holography. , 2008, Optics express.

[16]  O. Bunk,et al.  Influence of the overlap parameter on the convergence of the ptychographical iterative engine. , 2008, Ultramicroscopy.

[17]  Klamer Schutte,et al.  Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution , 2006, EURASIP J. Adv. Signal Process..

[18]  J. Rodenburg,et al.  Separation of three-dimensional scattering effects in tilt-series Fourier ptychography. , 2015, Ultramicroscopy.

[19]  M. Schmid Principles Of Optics Electromagnetic Theory Of Propagation Interference And Diffraction Of Light , 2016 .

[20]  Helen Mullaly A Wide Field , 1978 .

[21]  O. Bunk,et al.  High-Resolution Scanning X-ray Diffraction Microscopy , 2008, Science.

[22]  A. Papoulis A new algorithm in spectral analysis and band-limited extrapolation. , 1975 .

[23]  Brendan Ames,et al.  Solving ptychography with a convex relaxation , 2014, New journal of physics.

[24]  Takeo Kanade,et al.  Limits on Super-Resolution and How to Break Them , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Bernhard Schölkopf,et al.  Non-stationary correction of optical aberrations , 2011, 2011 International Conference on Computer Vision.

[26]  L. Tian,et al.  3D intensity and phase imaging from light field measurements in an LED array microscope , 2015 .

[27]  B. Ellerbroek First-order performance evaluation of adaptive optics systems for atmospheric turbulence compensatio , 1994 .

[28]  Shixue Zhang,et al.  An Efficient Lucky Imaging System for Astronomical Image Restoration , 2011 .

[29]  Zeev Zalevsky,et al.  Synthetic aperture superresolution with multiple off-axis holograms. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[30]  Thomas B. Moeslund,et al.  Super-resolution: a comprehensive survey , 2014, Machine Vision and Applications.

[31]  M. Born Principles of Optics : Electromagnetic theory of propagation , 1970 .

[32]  Sundeep Rangan,et al.  Compressive Phase Retrieval via Generalized Approximate Message Passing , 2014, IEEE Transactions on Signal Processing.

[33]  R. Gerchberg Super-resolution through Error Energy Reduction , 1974 .

[34]  Guoan Zheng,et al.  Embedded pupil function recovery for Fourier ptychographic microscopy. , 2014, Optics express.

[35]  J. H. Massig,et al.  Digital off-axis holography with a synthetic aperture. , 2002, Optics letters.

[36]  D. Sampson,et al.  High-resolution, wide-field object reconstruction with synthetic aperture Fourier holographic optical microscopy. , 2009, Optics express.

[37]  M. Murnane,et al.  Tabletop nanometer extreme ultraviolet imaging in an extended reflection mode using coherent Fresnel ptychography , 2013, 1312.2049.

[38]  B. Welsh,et al.  Imaging Through Turbulence , 1996 .

[39]  Siyuan Dong,et al.  Aperture-scanning Fourier ptychography for 3D refocusing and super-resolution macroscopic imaging. , 2014, Optics express.

[40]  A. Lohmann,et al.  Speckle masking in astronomy: triple correlation theory and applications. , 1983, Applied optics.

[41]  Tomasz Orczyk,et al.  Fingerprint ridges frequency , 2011, 2011 Third World Congress on Nature and Biologically Inspired Computing.

[42]  Aswin C. Sankaranarayanan,et al.  Flutter Shutter Video Camera for compressive sensing of videos , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[43]  T. Heinosaari,et al.  Quantum Tomography under Prior Information , 2011, 1109.5478.

[44]  Shmuel Peleg,et al.  Robust super-resolution , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[45]  D. Fried Probability of getting a lucky short-exposure image through turbulence* , 1978 .

[46]  K. Knox,et al.  Recovery of Images from Atmospherically Degraded Short-Exposure Photographs , 1974 .

[47]  R. Gerchberg A practical algorithm for the determination of phase from image and diffraction plane pictures , 1972 .

[48]  M. Booth Adaptive optics in microscopy. , 2003, Philosophical transactions. Series A, Mathematical, physical, and engineering sciences.

[49]  Takeo Watanabe,et al.  Phase Imaging of Extreme-Ultraviolet Mask Using Coherent Extreme-Ultraviolet Scatterometry Microscope , 2013 .

[50]  A. Labeyrie Attainment of diffraction limited resolution in large telescopes by Fourier analysing speckle patterns in star images , 1970 .

[51]  박종민 차량의 가속도를 이용한 Image Stabilization , 2013 .

[52]  Sabine Süsstrunk,et al.  A Frequency Domain Approach to Registration of Aliased Images with Application to Super-resolution , 2006, EURASIP J. Adv. Signal Process..

[53]  K. Nechvíle The High Resolution , 2005 .

[54]  J. Rodenburg,et al.  An improved ptychographical phase retrieval algorithm for diffractive imaging. , 2009, Ultramicroscopy.

[55]  Veit Elser Phase retrieval by iterated projections. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[56]  Guoan Zheng,et al.  Overlapped Fourier coding for optical aberration removal. , 2014, Optics express.

[57]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[58]  Shree K. Nayar,et al.  PiCam , 2013, ACM Trans. Graph..

[59]  Scott W. Teare,et al.  Ground-based High-Resolution Imaging of Mercury , 2000 .

[60]  Heung-Yeung Shum,et al.  Fundamental limits of reconstruction-based superresolution algorithms under local translation , 2004 .

[61]  Andrew R Harvey,et al.  Super-resolution imaging using a camera array. , 2014, Optics letters.

[62]  Basel Salahieh,et al.  Transfer function analysis in epi-illumination Fourier ptychography. , 2015, Optics letters.

[63]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[64]  Sing Bing Kang Automatic Removal of Chromatic Aberration from a Single Image , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  A. S. Fruchter,et al.  Drizzle: A Method for the Linear Reconstruction of Undersampled Images , 1998 .

[66]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[67]  Anil K. Jain,et al.  Handbook of Fingerprint Recognition , 2005, Springer Professional Computing.

[68]  Michal Irani,et al.  Super-resolution from a single image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[69]  Takeo Kanade,et al.  When Is the Shape of a Scene Unique Given Its Light-Field: A Fundamental Theorem of 3D Vision? , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  Robert L. Stevenson,et al.  Spatial Resolution Enhancement of Low-Resolution Image Sequences A Comprehensive Review with Directions for Future Research , 1998 .

[71]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH '06.

[72]  R. Horstmeyer,et al.  Wide-field, high-resolution Fourier ptychographic microscopy , 2013, Nature Photonics.

[73]  J. Davenport Editor , 1960 .

[74]  Kannan Ramchandran,et al.  Multiplexed coded illumination for Fourier Ptychography with an LED array microscope. , 2014, Biomedical optics express.