FlatCam: Thin, Bare-Sensor Cameras using Coded Aperture and Computation

FlatCam is a thin form-factor lensless camera that consists of a coded mask placed on top of a bare, conventional sensor array. Unlike a traditional, lens-based camera where an image of the scene is directly recorded on the sensor pixels, each pixel in FlatCam records a linear combination of light from multiple scene elements. A computational algorithm is then used to demultiplex the recorded measurements and reconstruct an image of the scene. FlatCam is an instance of a coded aperture imaging system; however, unlike the vast majority of related work, we place the coded mask extremely close to the image sensor that can enable a thin system. We employ a separable mask to ensure that both calibration and image reconstruction are scalable in terms of memory requirements and computational complexity. We demonstrate the potential of the FlatCam design using two prototypes: one at visible wavelengths and one at infrared wavelengths.

[1]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[2]  Aydogan Ozcan,et al.  Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy , 2012, Nature Methods.

[3]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[4]  David G. Stork,et al.  Lensless Ultra-Miniature Imagers Using Odd-Symmetry Spiral Phase Gratings , 2013 .

[5]  J. Tanida,et al.  Thin Observation Module by Bound Optics (TOMBO): Concept and Experimental Verification. , 2001, Applied optics.

[6]  Shree K. Nayar,et al.  Lensless Imaging with a Controllable Aperture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  Yvonne Freeh,et al.  Optical Imaging And Spectroscopy , 2016 .

[8]  R. Dicke.  SCATTER-HOLE CAMERAS FOR X-RAYS AND GAMMA RAYS. , 1968 .

[9]  Yibo Zhang,et al.  Wide-field computational imaging of pathology slides using lens-free on-chip microscopy , 2014, Science Translational Medicine.

[10]  David G. Stork,et al.  Lensless Ultra-Miniature CMOS Computational Imagers and Sensors , 2013 .

[11]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[12]  Alyosha Molnar,et al.  A microscale camera using direct Fourier-domain scene capture. , 2011, Optics letters.

[13]  Albert Wang,et al.  Angle sensitive pixels in CMOS for lensless 3D imaging , 2009, 2009 IEEE Custom Integrated Circuits Conference.

[14]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[15]  Michael J. DeWeert,et al.  Lensless coded aperture imaging with separable doubly Toeplitz masks , 2014, Sensing Technologies + Applications.

[16]  K. Perez.  Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment , 2014 .

[17]  Rebecca Willett,et al.  Compressive coded aperture superresolution image reconstruction , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  T. M. Cannon,et al.  Coded aperture imaging with uniformly redundant arrays. , 1978, Applied optics.

[19]  V. Dragoi,et al.  CMOS wafer bonding for back-side illuminated image sensors fabrication , 2010, 2010 11th International Conference on Electronic Packaging Technology & High Density Packaging.

[20]  E E Fenimore,et al.  Fast delta Hadamard transform. , 1981, Applied optics.

[21]  David Ramsden,et al.  The application of pinhole and coded aperture imaging in the nuclear environment , 1999 .

[22]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[23]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[24]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[25]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing - The Sparse Way, 3rd Edition , 2008 .

[26]  Hu Chuan-Gan,et al.  On The Shift Register Sequences , 2004 .

[27]  E E Fenimore,et al.  New family of binary arrays for coded aperture imaging. , 1989, Applied optics.

[28]  Andreas Tünnermann,et al.  Thin wafer-level camera lenses inspired by insect compound eyes. , 2010, Optics express.

[29]  E. E. Fenimore,et al.  Coded Aperture Imaging: Many Holes Make Light Work , 1980 .

[30]  Gordon Wetzstein,et al.  Compressive light field photography using overcomplete dictionaries and optimized projections , 2013, ACM Trans. Graph..

[31]  Paul A. Wilford,et al.  Lensless imaging by compressive sensing , 2013, 2013 IEEE International Conference on Image Processing.

[32]  Joseph E Ford,et al.  Ultrathin cameras using annular folded optics. , 2007, Applied optics.

[33]  F. MacWilliams,et al.  Pseudo-random sequences and arrays , 1976, Proceedings of the IEEE.

[34]  Caihua Chen,et al.  Thin infrared imaging systems through multichannel sampling. , 2008, Applied optics.

[35]  Ashwin A. Wagadarikar,et al.  Single disperser design for coded aperture snapshot spectral imaging. , 2008, Applied optics.

[36]  Guillermo Sapiro,et al.  Coded aperture compressive temporal imaging , 2013, Optics express.

[37]  A. Busboom,et al.  Uniformly Redundant Arrays , 1998 .

[38]  Jie Ding,et al.  Complementary Lattice Arrays for Coded Aperture Imaging , 2015, Journal of the Optical Society of America. A, Optics, image science, and vision.

[39]  Shree K. Nayar,et al.  Efficient Space-Time Sampling with Pixel-Wise Coded Exposure for High-Speed Imaging , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  P. Avouris,et al.  Photodetectors based on graphene, other two-dimensional materials and hybrid systems. , 2014, Nature nanotechnology.