Robust Restoration of Sparse Multidimensional Single-Photon LiDAR Images

The challenges of real world applications of the laser detection and ranging (Lidar) three-dimensional (3-D) imaging require specialized algorithms. In this paper, a new reconstruction algorithm for single-photon 3-D Lidar images is presented that can deal with multiple tasks. For example, when the return signal contains multiple peaks due to imaging semitransparent surfaces, or when imaging through obscurants such as scattering media. A generalization to the multidimensional case, including multispectral and multitemporal 3-D images, is also provided. The approach is based on the minimization of a cost function accounting for Poissonian observations of the single-photon data, the nonlocal spatial correlations between pixels and the small number of depth layers inside the observed range window. An alternating direction method of multipliers that offers good convergence properties is used to solve this minimization problem. The resulting algorithm is validated on synthetic and real data and in challenging realistic scenarios including sparse photon regimes for fast imaging, the presence of high background due to obscurants, and the joint processing of multispectral and/or multitemporal data.

[1]  Yao Wang,et al.  Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model , 2014, IEEE Transactions on Image Processing.

[2]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[3]  Ramesh Raskar,et al.  Towards photography through realistic fog , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[4]  Jean-Yves Tourneret,et al.  Bayesian 3D Reconstruction of Complex Scenes from Single-Photon Lidar Data , 2018, SIAM J. Imaging Sci..

[5]  Gerald Buller,et al.  Efficient Range Estimation and Material Quantification from Multispectral Lidar Waveforms , 2016, 2016 Sensor Signal Processing for Defence (SSPD).

[6]  Aongus McCarthy,et al.  Three-dimensional single-photon imaging through obscurants. , 2019, Optics express.

[7]  Aongus McCarthy,et al.  Object Depth Profile and Reflectivity Restoration From Sparse Single-Photon Data Acquired in Underwater Environments , 2016, IEEE Transactions on Computational Imaging.

[8]  Vivek K Goyal,et al.  Computational multi-depth single-photon imaging. , 2016, Optics express.

[9]  Ximing Ren,et al.  High-resolution depth profiling using a range-gated CMOS SPAD quanta image sensor. , 2018, Optics express.

[10]  Alfred Hero,et al.  Robust Spectral Unmixing of Sparse Multispectral Lidar Waveforms Using Gamma Markov Random Fields , 2016, IEEE Transactions on Computational Imaging.

[11]  G. Buller,et al.  Kilometer-range depth imaging at 1,550 nm wavelength using an InGaAs/InP single-photon avalanche diode detector. , 2013, Optics express.

[12]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[13]  Andrew M. Wallace,et al.  Multilayered 3D LiDAR Image Construction Using Spatial Models in a Bayesian Framework , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Antonio J. Plaza,et al.  Total Variation Spatial Regularization for Sparse Hyperspectral Unmixing , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Aongus McCarthy,et al.  Long-range depth profiling of camouflaged targets using single-photon detection , 2017 .

[16]  Aongus McCarthy,et al.  Lidar Waveform-Based Analysis of Depth Images Constructed Using Sparse Single-Photon Data , 2015, IEEE Transactions on Image Processing.

[17]  Abderrahim Halimi,et al.  Single-photon three-dimensional imaging at up to 10 kilometers range. , 2017, Optics express.

[18]  Aongus McCarthy,et al.  Restoration of multilayered single-photon 3D Lidar images , 2017, 2017 25th European Signal Processing Conference (EUSIPCO).

[19]  Gordon Wetzstein,et al.  Single-photon 3D imaging with deep sensor fusion , 2018, ACM Trans. Graph..

[20]  Aongus McCarthy,et al.  Restoration of intensity and depth images constructed using sparse single-photon data , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[21]  Konrad Schindler,et al.  Super-Resolution of Multispectral Multiresolution Images from a Single Sensor , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[23]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[24]  Graham M. Gibson,et al.  3D single-pixel video , 2016 .

[25]  Vivek K. Goyal,et al.  A Few Photons Among Many: Unmixing Signal and Noise for Photon-Efficient Active Imaging , 2016, IEEE Transactions on Computational Imaging.

[26]  José M. Bioucas-Dias,et al.  Restoration of Poissonian Images Using Alternating Direction Optimization , 2010, IEEE Transactions on Image Processing.

[27]  Jocelyn Chanussot,et al.  Dynamical Spectral Unmixing of Multitemporal Hyperspectral Images , 2015, IEEE Transactions on Image Processing.

[28]  Andrew M. Wallace,et al.  Bayesian Analysis of Lidar Signals with Multiple Returns , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Zhao Wei,et al.  Photon-limited depth and reflectivity imaging with sparsity regularization , 2017 .

[30]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Martin Laurenzis,et al.  Influence of gating and of the gate shape on the penetration capacity of range-gated active imaging in scattering environments. , 2015, Optics express.

[32]  Jean-Yves Tourneret,et al.  A Hierarchical Bayesian Model Accounting for Endmember Variability and Abrupt Spectral Changes to Unmix Multitemporal Hyperspectral Images , 2016, IEEE Transactions on Computational Imaging.

[33]  Yonina C. Eldar,et al.  C-HiLasso: A Collaborative Hierarchical Sparse Modeling Framework , 2010, IEEE Transactions on Signal Processing.

[34]  Angshul Majumdar,et al.  Hyperspectral Unmixing in the Presence of Mixed Noise Using Joint-Sparsity and Total Variation , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Rebecca Willett,et al.  Proximal-Gradient methods for poisson image reconstruction with BM3D-Based regularization , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[36]  Rebecca Willett,et al.  Poisson Noise Reduction with Non-local PCA , 2012, Journal of Mathematical Imaging and Vision.

[37]  Antonio J. Plaza,et al.  Collaborative Sparse Regression for Hyperspectral Unmixing , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[38]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[39]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[40]  Paul Honeine,et al.  Unmixing multitemporal hyperspectral images accounting for endmember variability , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[41]  José M. Bioucas-Dias,et al.  Fast Hyperspectral Unmixing in Presence of Nonlinearity or Mismodeling Effects , 2016, IEEE Transactions on Computational Imaging.

[42]  Andrew M. Wallace,et al.  Full Waveform Analysis for Long-Range 3D Imaging Laser Radar , 2010, EURASIP J. Adv. Signal Process..

[43]  Gerald S. Buller,et al.  Depth imaging through obscurants using time-correlated single-photon counting , 2018, Commercial + Scientific Sensing and Imaging.

[44]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[45]  Vivek K. Goyal,et al.  Computational single-photon depth imaging without transverse regularization , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[46]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.