Comparative study of sampling strategies for sparse photon multispectral lidar imaging: towards mosaic filter arrays

In this paper, we investigate the recovery of range and spectral profiles associated with remote three-dimensional scenes sensed via single-photon multispectral lidar (MSL). We consider two different spatial/spectral sampling strategies and compare their performance for a similar overall number of detected photons. For a regular spatial grid of pixels, the first strategy consists of sampling all the spatial locations of the grid for each of the L wavelengths. The second strategy is consistent with the use of mosaic filter-based arrays and consists of acquiring only one wavelength (out of L) per spatial location. Despite the reduction of spectral content observed in each location, the second strategy has clear potential advantages for fast multispectral imaging using only a single frame read out. We propose a fully automated computational method, adapted for each of the two sampling strategies in order to recover the target range profile, as well as the reflectivity profiles associated with the different wavelengths. These strategies were also assessed with high ambient background. The performance of the two sampling strategies is illustrated using a single-photon MSL system with L = 4 wavelengths (473, 532, 589 and 640 nm). The results presented demonstrate that although the first strategy usually provides more accurate results, the second strategy does not exhibit a significant performance degradation, particularly for sparse photon data (down to 1 photon per pixel on average). These results suggest a way forward for the integration of single-photon detector arrays with mosaic filters for use in a range of emerging photon-starved two-dimensional and three-dimensional imaging applications.

[1]  Ximing Ren,et al.  Design and Evaluation of Multispectral LiDAR for the Recovery of Arboreal Parameters , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Vivek K. Goyal,et al.  Photon-Efficient Computational 3-D and Reflectivity Imaging With Single-Photon Detectors , 2014, IEEE Transactions on Computational Imaging.

[3]  David Tyndall,et al.  A study of pile-up in integrated time-correlated single photon counting systems. , 2013, The Review of scientific instruments.

[4]  S. Pellegrini,et al.  Backside illuminated SPAD image sensor with 7.83μm pitch in 3D-stacked CMOS technology , 2016, 2016 IEEE International Electron Devices Meeting (IEDM).

[5]  G. Buller,et al.  Kilometer-range depth imaging at 1,550 nm wavelength using an InGaAs/InP single-photon avalanche diode detector. , 2013, Optics express.

[6]  Edoardo Charbon,et al.  A 160x128 Single-Photon Image Sensor with On-Pixel 55ps 10b Time-to-Digital Converter , 2011 .

[7]  Edoardo Charbon,et al.  A 160×128 single-photon image sensor with on-pixel 55ps 10b time-to-digital converter , 2011, 2011 IEEE International Solid-State Circuits Conference.

[8]  Aongus McCarthy,et al.  Object Depth Profile and Reflectivity Restoration From Sparse Single-Photon Data Acquired in Underwater Environments , 2016, IEEE Transactions on Computational Imaging.

[9]  J. Tanner,et al.  Low rank matrix completion by alternating steepest descent methods , 2016 .

[10]  Vivek K Goyal,et al.  First-Photon Imaging , 2014, Science.

[11]  Gerald Buller,et al.  Efficient Range Estimation and Material Quantification from Multispectral Lidar Waveforms , 2016, 2016 Sensor Signal Processing for Defence (SSPD).

[12]  G. Buller,et al.  Ranging and Three-Dimensional Imaging Using Time-Correlated Single-Photon Counting and Point-by-Point Acquisition , 2007, IEEE Journal of Selected Topics in Quantum Electronics.

[13]  Andrew M. Wallace,et al.  Bayesian Analysis of Lidar Signals with Multiple Returns , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  José M. Bioucas-Dias,et al.  Restoration of Poissonian Images Using Alternating Direction Optimization , 2010, IEEE Transactions on Image Processing.

[15]  Abderrahim Halimi,et al.  Single-photon three-dimensional imaging at up to 10 kilometers range. , 2017, Optics express.

[16]  M. J. Deprenger,et al.  Imaging with multi-spectral mosaic-array cameras. , 2015, Applied optics.

[17]  R. Raskar,et al.  Erratum: Single-photon sensitive light-in-flight imaging , 2015, Nature communications.

[18]  Alfred Hero,et al.  Robust Spectral Unmixing of Sparse Multispectral Lidar Waveforms Using Gamma Markov Random Fields , 2016, IEEE Transactions on Computational Imaging.

[19]  W. Brockherde,et al.  CMOS Imager With 1024 SPADs and TDCs for Single-Photon Timing and 3-D Time-of-Flight , 2014, IEEE Journal of Selected Topics in Quantum Electronics.

[20]  Aongus McCarthy,et al.  Robust spectral unmixing of multispectral Lidar waveforms , 2016, 2016 8th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[21]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[22]  Nick Whiteley,et al.  Maximum marginal likelihood estimation of the granularity coefficient of a Potts-Markov random field within an MCMC algorithm , 2014, 2014 IEEE Workshop on Statistical Signal Processing (SSP).

[23]  Aongus McCarthy,et al.  Lidar Waveform-Based Analysis of Depth Images Constructed Using Sparse Single-Photon Data , 2015, IEEE Transactions on Image Processing.

[24]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[25]  Robert A. Lamb,et al.  Advanced 3D imaging lidar concepts for long range sensing , 2014, Sensing Technologies + Applications.

[26]  Jeffrey D. Blanchard,et al.  CGIHT: Conjugate Gradient Iterative Hard Thresholding for Compressed Sensing and Matrix Completion , 2015 .

[27]  Mark A. Itzler,et al.  Geiger-mode APD camera system for single-photon 3D LADAR imaging , 2012, Defense, Security, and Sensing.

[28]  Andrew M. Wallace,et al.  Multiple wavelength time-of-flight sensor based on time-correlated single-photon counting , 2004 .

[29]  Pierre Gouton,et al.  Multispectral Filter Arrays: Recent Advances and Practical Implementation , 2014, Sensors.

[30]  Rebecca Willett,et al.  This is SPIRAL-TAP: Sparse Poisson Intensity Reconstruction ALgorithms—Theory and Practice , 2010, IEEE Transactions on Image Processing.

[31]  R. Raskar,et al.  Single-photon sensitive light-in-fight imaging , 2015, Nature Communications.

[32]  John J. Degnan,et al.  Photon-Counting Multikilohertz Microlaser Altimeters for Airborne and Spaceborne Topographic Measurements , 2013 .

[33]  D. G. Kocher,et al.  Three-dimensional imaging laser radar with a photon-counting avalanche photodiode array and microchip laser. , 2002, Applied optics.

[34]  R. Collins,et al.  Long-range time-of-flight scanning sensor based on high-speed time-correlated single-photon counting. , 2009, Applied optics.