Time-of-Flight Range Measurement in Low-Sensing Environment: Noise Analysis and Complex-Domain Non-Local Denoising

In this paper, we deal with the problem of denoising 3D scene range measurements acquired by time-of-flight (ToF) range sensors and composed in the form of 2D image-like depth maps. We address the specific case of ToF low-sensing environment (LSE). Such environment is set by low-light sensing conditions, low-power hardware requirements, and low-reflectivity scenes. We demonstrate that data captured by a device in such mode can be effectively post-processed in order to reach the same measurement accuracy as if the device was working in normal operating mode. In order to achieve this, we first present an elaborated analysis of noise properties of ToF data sensed in LSE and verify the derived noise models by empirical measurements. Then, we develop a related novel non-local denoising approach working in complex domain and demonstrate its superiority against the state of the art for data acquired by an off-the-shelf ToF device.

[1]  Assen Shulev,et al.  A sliding-window transform-domain technique for denoising of DSPI phase maps , 2014, Electronic Imaging.

[2]  Alessandro Foi,et al.  Optimal Inversion of the Generalized Anscombe Transformation for Poisson-Gaussian Noise , 2013, IEEE Transactions on Image Processing.

[3]  Jacques Froment,et al.  Parameter-Free Fast Pixelwise Non-Local Means Denoising , 2014, Image Process. Line.

[4]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[5]  Kiriakos N. Kutulakos,et al.  A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation , 2005, ICCV.

[6]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[7]  Reinhard Koch,et al.  Technical Foundation and Calibration Methods for Time-of-Flight Cameras , 2013, Time-of-Flight and Depth Imaging.

[8]  Fred A. Hamprecht,et al.  Denoising of continuous-wave time-of-flight depth images using confidence measures , 2009, Optical Engineering.

[9]  S. O. Rice,et al.  Statistical properties of a sine wave plus random noise , 1948, Bell Syst. Tech. J..

[10]  Rahul Nair,et al.  Denoising Strategies for Time-of-Flight Data , 2013, Time-of-Flight and Depth Imaging.

[11]  Jaakko Astola,et al.  Local Approximation Techniques in Signal and Image Processing (SPIE Press Monograph Vol. PM157) , 2006 .

[12]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Jianjun Lei,et al.  Depth Map Super-Resolution Considering View Synthesis Quality , 2017, IEEE Transactions on Image Processing.

[14]  Xuming Luan,et al.  Experimental investigation of photonic mixer device and development of TOF 3D ranging Ssystems based on PMD technology , 2006 .

[15]  Miska M. Hannuksela,et al.  Joint de-noising and fusion of 2D video and depth map sequences sensed by low-powered tof range sensor , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[16]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  A. Mertins,et al.  Time-of-flight depth image denoising using prior noise information , 2010, IEEE 10th INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING PROCEEDINGS.

[18]  Christopher I. Moore,et al.  RF phase-coded random-modulation LIDAR , 2004 .

[19]  Miska M. Hannuksela,et al.  De-noising of distance maps sensed by time-of-flight devices in poor sensing environment , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Alessandro Foi,et al.  Noise estimation and removal in MR imaging: The variance-stabilization approach , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[21]  Timo Schairer,et al.  Fusion of range and color images for denoising and resolution enhancement with a non-local filter , 2010, Comput. Vis. Image Underst..

[22]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[23]  Ju Shen,et al.  Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Yang Li,et al.  A weighted least squares algorithm for time-of-flight depth image denoising , 2014 .

[25]  Sebastian Thrun,et al.  A Noise‐aware Filter for Real‐time Depth Upsampling , 2008 .

[26]  Karen O. Egiazarian,et al.  Methods for depth-map filtering in view-plus-depth 3D video representation , 2012, EURASIP J. Adv. Signal Process..

[27]  G. Buller,et al.  Kilometer-range, high resolution depth imaging via 1560 nm wavelength single-photon detection. , 2013, Optics express.

[28]  Robert Bregovic,et al.  Fixed-Pattern Noise Modeling and Removal in Time-of-Flight Sensing , 2016, IEEE Transactions on Instrumentation and Measurement.

[29]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[30]  Christoph S. Garbe,et al.  Denoising Time-Of-Flight Data with Adaptive Total Variation , 2011, ISVC.

[31]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[32]  Bernd Jähne,et al.  Theoretical and experimental error analysis of continuous-wave time-of-flight range cameras , 2009, Optical Engineering.

[33]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[34]  Vivek K. Goyal,et al.  Phase unwrapping and denoising for time-of-flight imaging using generalized approximate message passing , 2013, 2013 IEEE International Conference on Image Processing.

[35]  Luis Salgado,et al.  Efficient spatio-temporal hole filling strategy for Kinect depth maps , 2012, Electronic Imaging.

[36]  R. Lange,et al.  Solid-state time-of-flight range camera , 2001 .

[37]  Miska M. Hannuksela,et al.  Real-time denoising of ToF measurements by spatio-temporal non-local mean filtering , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[38]  Alessandro Foi,et al.  Joint Removal of Random and Fixed-Pattern Noise Through Spatiotemporal Video Filtering , 2014, IEEE Transactions on Image Processing.

[39]  Torsten Edeler.  Bildverbesserung von Time-Of-Flight-Tiefenkarten , 2012 .

[40]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[41]  Peyman Milanfar,et al.  A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical , 2013, IEEE Signal Processing Magazine.

[42]  Atanas P. Gotchev,et al.  A speed-optimized RGB-Z capture system with improved denoising capabilities , 2014, Electronic Imaging.