Image and Depth Estimation with Mask-Based Lensless Cameras

Mask-based lensless cameras replace the lens by placing a fixed mask on top of an image sensor. These cameras can potentially be very thin and even flexible. Recently, it has been demonstrated that such mask-based cameras can recover light intensity and depth information of a scene. Existing depth recovery algorithms either assume that the scene consists of a small number of depth planes or solve a sparse recovery problem over a large 3D volume, and lose robustness to complicated scenes consisting of varying depth surface. In this paper, we propose a new approach for depth estimation based on alternating gradient descent algorithm that jointly estimates the continuous depth map and light distribution of a scene. The computational complexity of the algorithm scales linearly with the spatial dimension of the imaging system. We present simulation results on image and depth reconstruction for standard 3D test scenes. The comparison between the proposed algorithm and other method shows that our algorithm is faster and more robust for natural scenes with a large range of depths.

[1]  Benjamin Recht,et al.  The alternating descent conditional gradient method for sparse inverse problems , 2015, 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[2]  Aswin C. Sankaranarayanan,et al.  Lensless Imaging: A computational renaissance , 2016, IEEE Signal Processing Magazine.

[3]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[4]  Christos Thrampoulidis,et al.  Analysis and Optimization of Aperture Design in Computational Imaging , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[6]  Ashok Veeraraghavan,et al.  Single-frame 3D fluorescence microscopy with ultraminiature lensless FlatScope , 2017, Science Advances.

[7]  E. E. Fenimore,et al.  Uniformly redundant arrays , 1977 .

[8]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[9]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[10]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[11]  M. Salman Asif.  Lensless 3D Imaging Using Mask-Based Cameras , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  D. T. Wilson,et al.  Fresnel Zone Plate Imaging in Radiology and Nuclear Medicine , 1973 .

[13]  E. E. Fenimore,et al.  Coded Aperture Imaging: Many Holes Make Light Work , 1980 .

[14]  Cishen Zhang,et al.  Off-Grid Direction of Arrival Estimation Using Sparse Bayesian Inference , 2011, IEEE Transactions on Signal Processing.

[15]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[16]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[17]  Parikshit Shah,et al.  Compressed Sensing Off the Grid , 2012, IEEE Transactions on Information Theory.

[18]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[19]  Aswin C. Sankaranarayanan,et al.  FlatCam: Thin, Lensless Cameras Using Coded Aperture and Computation , 2017, IEEE Transactions on Computational Imaging.

[20]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[21]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[22]  Ashok Veeraraghavan,et al.  PhaseCam3D — Learning Phase Masks for Passive Single View Depth Estimation , 2019, 2019 IEEE International Conference on Computational Photography (ICCP).

[23]  Siam Rfview,et al.  CONVERGENCE CONDITIONS FOR ASCENT METHODS , 2016 .

[24]  Dikpal Reddy,et al.  External Mask Based Depth and Light Field Camera , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[25]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[26]  Shree K. Nayar,et al.  Lensless Imaging with a Controllable Aperture , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[27]  Zhengrong Liang,et al.  Adaptive-weighted total variation minimization for sparse data toward low-dose x-ray computed tomography image reconstruction , 2012, Physics in medicine and biology.

[28]  Yi Luo,et al.  Analysis of Diffractive Optical Neural Networks and Their Integration With Electronic Neural Networks , 2018, IEEE Journal of Selected Topics in Quantum Electronics.

[29]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, ACM Trans. Graph..

[30]  Yi Luo,et al.  All-optical machine learning using diffractive deep neural networks , 2018, Science.

[31]  T. M. Cannon,et al.  Coded aperture imaging with uniformly redundant arrays. , 1978, Applied optics.

[32]  F. MacWilliams,et al.  Pseudo-random sequences and arrays , 1976, Proceedings of the IEEE.

[33]  Arye Nehorai,et al.  Joint Sparse Recovery Method for Compressed Sensing With Structured Dictionary Mismatches , 2013, IEEE Transactions on Signal Processing.

[34]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[35]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[36]  Gordon Wetzstein,et al.  Deep Optics for Monocular Depth Estimation and 3D Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[37]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[38]  Richard G. Baraniuk,et al.  A new compressive imaging camera architecture using optical-domain compression , 2006, Electronic Imaging.

[39]  M. Salman Asif.  Toward depth estimation using mask-based lensless cameras , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[40]  S. Burak Gokturk,et al.  A Time-Of-Flight Depth Sensor - System Description, Issues and Solutions , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[41]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[42]  Ashutosh Saxena,et al.  Learning Depth from Single Monocular Images , 2005, NIPS.