Sub-picosecond photon-efficient 3D imaging using single-photon sensors

Active 3D imaging systems have broad applications across disciplines, including biological imaging, remote sensing and robotics. Applications in these domains require fast acquisition times, high timing accuracy, and high detection sensitivity. Single-photon avalanche diodes (SPADs) have emerged as one of the most promising detector technologies to achieve all of these requirements. However, these detectors are plagued by measurement distortions known as pileup, which fundamentally limit their precision. In this work, we develop a probabilistic image formation model that accurately models pileup. We devise inverse methods to efficiently and robustly estimate scene depth and reflectance from recorded photon counts using the proposed model along with statistical priors. With this algorithm, we not only demonstrate improvements to timing accuracy by more than an order of magnitude compared to the state-of-the-art, but our approach is also the first to facilitate sub-picosecond-accurate, photon-efficient 3D imaging in practical scenarios where widely-varying photon counts are observed.

[1]  Richard A. Fournier,et al.  The structural and radiative consistency of three-dimensional tree reconstructions from terrestrial lidar , 2009 .

[2]  W. Brockherde,et al.  100 000 Frames/s 64 × 32 Single-Photon Detector Array for 2-D Imaging and 3-D Ranging , 2014, IEEE Journal of Selected Topics in Quantum Electronics.

[3]  Vladlen Koltun,et al.  A Simple Model for Intrinsic Image Decomposition with Depth Cues , 2013, 2013 IEEE International Conference on Computer Vision.

[4]  Vivek K. Goyal,et al.  Photon-Efficient Computational 3-D and Reflectivity Imaging With Single-Photon Detectors , 2014, IEEE Transactions on Computational Imaging.

[5]  Alberto Tosi,et al.  Automotive Three-Dimensional Vision Through a Single-Photon Counting SPAD Camera , 2016, IEEE Transactions on Intelligent Transportation Systems.

[6]  Robert W. Boyd,et al.  Imaging with a small number of photons , 2014, Nature Communications.

[7]  Vivek K Goyal,et al.  First-Photon Imaging , 2014, Science.

[8]  Vivek K Goyal,et al.  Photon-efficient imaging with a single-photon camera , 2016, Nature Communications.

[9]  G. Buller,et al.  Kilometer-range, high resolution depth imaging via 1560 nm wavelength single-photon detection. , 2013, Optics express.

[10]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[11]  G. Ripamonti,et al.  Active-Quenching and Gating Circuits for Single-Photon Avalanche Diodes (SPADs) , 1982, IEEE Transactions on Nuclear Science.

[12]  D. Young,et al.  Geiger-Mode Avalanche Photodiodes for Three-Dimensional Imaging , 2002 .

[13]  A. Dalla Mora,et al.  Fast-gated single-photon avalanche diode for extremely wide dynamic-range applications , 2009, BiOS.

[14]  Gordon Wetzstein,et al.  Confocal non-line-of-sight imaging based on the light-cone transform , 2018, Nature.

[15]  Kevin Blankespoor,et al.  BigDog, the Rough-Terrain Quadruped Robot , 2008 .

[16]  Gordon Wetzstein,et al.  Single-photon 3D imaging with deep sensor fusion , 2018, ACM Trans. Graph..

[17]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[18]  J. Busck,et al.  Gated viewing and high-accuracy three-dimensional laser radar. , 2004, Applied optics.

[19]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.

[20]  Sergey V. Polyakov Single-Photon Detector Calibration , 2013 .

[21]  E Gratton,et al.  The photon counting histogram in fluorescence fluctuation spectroscopy. , 1999, Biophysical journal.

[22]  Michael Wahl,et al.  Time-Correlated Single Photon Counting , 2009 .

[23]  W. Brockherde,et al.  CMOS Imager With 1024 SPADs and TDCs for Single-Photon Timing and 3-D Time-of-Flight , 2014, IEEE Journal of Selected Topics in Quantum Electronics.

[24]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[25]  Robert Henderson,et al.  Detection and tracking of moving objects hidden from view , 2015, Nature Photonics.

[26]  Sergey V. Polyakov,et al.  Single-photon generation and detection , 2013 .

[27]  Xi Wang,et al.  High-Resolution Stereo Datasets with Subpixel-Accurate Ground Truth , 2014, GCPR.

[28]  Aswin C. Sankaranarayanan,et al.  Signal Processing Based Pile-up Compensation for Gated Single-Photon Avalanche Diodes , 2018, 1806.07437.

[29]  Vivek K. Goyal,et al.  A Few Photons Among Many: Unmixing Signal and Noise for Photon-Efficient Active Imaging , 2016, IEEE Transactions on Computational Imaging.

[30]  Brent Schwarz,et al.  LIDAR: Mapping the world in 3D , 2010 .

[31]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[32]  Vivek K. Goyal,et al.  Dead Time Compensation for High-Flux Ranging , 2018, IEEE Transactions on Signal Processing.

[33]  P. B. Coates,et al.  The correction for photon `pile-up' in the measurement of radiative lifetimes , 1968 .

[34]  R. Henderson,et al.  Edinburgh Research Explorer A Low Dark Count Single Photon Avalanche Diode Structure Compatible with Standard Nanometer Scale CMOS Technology , 2009 .

[35]  Vivek K Goyal,et al.  Computational multi-depth single-photon imaging. , 2016, Optics express.

[36]  R. Popovic,et al.  First fully integrated 2-D array of single-photon detectors in standard CMOS technology , 2003, IEEE Photonics Technology Letters.

[37]  Gordon Wetzstein,et al.  Confocal non-line-of-sight imaging , 2018, SIGGRAPH Talks.

[38]  Abderrahim Halimi,et al.  Single-photon three-dimensional imaging at up to 10 kilometers range. , 2017, Optics express.

[39]  P.-A. Besse,et al.  Design and characterization of a CMOS 3-D image sensor based on single photon avalanche diodes , 2005, IEEE Journal of Solid-State Circuits.