What Are Optimal Coding Functions for Time-of-Flight Imaging?

The depth resolution achieved by a continuous wave time-of-flight (C-ToF) imaging system is determined by the coding (modulation and demodulation) functions that it uses. Almost all current C-ToF systems use sinusoid or square coding functions, resulting in a limited depth resolution. In this article, we present a mathematical framework for exploring and characterizing the space of C-ToF coding functions in a geometrically intuitive space. Using this framework, we design families of novel coding functions that are based on Hamiltonian cycles on hypercube graphs. Given a fixed total source power and acquisition time, the new Hamiltonian coding scheme can achieve up to an order of magnitude higher resolution as compared to the current state-of-the-art methods, especially in low signal-to-noise ratio (SNR) settings. We also develop a comprehensive physically-motivated simulator for C-ToF cameras that can be used to evaluate various coding schemes prior to a real hardware implementation. Since most off-the-shelf C-ToF sensors use sinusoid or square functions, we develop a hardware prototype that can implement a wide range of coding functions. Using this prototype and our software simulator, we demonstrate the performance advantages of the proposed Hamiltonian coding functions in a wide range of imaging settings.

[1]  R. Lange,et al.  Solid-state time-of-flight range camera , 2001 .

[2]  Dale Anthony Carnegie,et al.  Analysis of Errors in ToF Range Imaging With Dual-Frequency Modulation , 2011, IEEE Transactions on Instrumentation and Measurement.

[3]  W. H. Mills Some complete cycles on the $n$-cube , 1963 .

[4]  James M. Kang,et al.  Space-Filling Curves , 2017, Encyclopedia of GIS.

[5]  Nahum Kiryati,et al.  Toward optimal structured light patterns , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[6]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[7]  Sebastian Nowozin,et al.  Bayesian Time-of-Flight for Realtime Shape, Illumination and Albedo , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Antonio Torralba,et al.  Statistics of natural image categories , 2003, Network.

[9]  Rudolf Schwarte,et al.  MSM-PMD as correlation receiver in a new 3D-ranging system , 2002, SPIE Remote Sensing.

[10]  A. Hoorfar,et al.  Bandwidth, cross-polarization, and feed-point characteristics of matched Hilbert antennas , 2003, IEEE Antennas and Wireless Propagation Letters.

[11]  Michael Lindenbaum,et al.  On the metric properties of discrete space-filling curves , 1996, IEEE Trans. Image Process..

[12]  Reinhard Koch,et al.  Time‐of‐Flight Cameras in Computer Graphics , 2010, Comput. Graph. Forum.

[13]  W. Koechner Optical Ranging System Employing a High Power Injection Laser Diode , 1968, IEEE Transactions on Aerospace and Electronic Systems.

[14]  Matthew O'Toole,et al.  Homogeneous codes for energy-efficient illumination and imaging , 2015, ACM Trans. Graph..

[15]  B. S. Goldstein,et al.  Gallium arsenide injection laser radar , 1967 .

[16]  Mirko Schmidt,et al.  SRA: Fast Removal of General Multipath for ToF Sensors , 2014, ECCV.

[17]  J. Abshire Optical distance measuring instrument , 1973 .

[18]  Helmut Fischer,et al.  New electro-optical mixing and correlating sensor: facilities and applications of the photonic mixer device (PMD) , 1997, Other Conferences.

[19]  MOHIT GUPTA,et al.  Phasor Imaging , 2015, ACM Trans. Graph..

[20]  Andreas Kolb,et al.  Real-time simulation of time-of-flight sensors , 2009, Simul. Model. Pract. Theory.

[21]  Bernd Jähne,et al.  A Physical Model of Time-of-Flight 3D Imaging Systems, Including Suppression of Ambient Light , 2009, Dyn3D.

[22]  David Stoppa,et al.  3D camera based on linear-mode gain-modulated avalanche photodiodes , 2013, 2013 IEEE International Solid-State Circuits Conference Digest of Technical Papers.

[23]  Matthew O'Toole,et al.  Temporal frequency probing for 5D transient analysis of global light transport , 2014, ACM Trans. Graph..

[24]  Fumio Sato,et al.  Gain-modulated Axi-Vision Camera (high speed high-accuracy depth-mapping camera). , 2004, Optics express.

[25]  Dale A. Carnegie,et al.  Design and Construction of a Configurable Full-Field Range Imaging System for Mobile Robotic Applications , 2011 .

[26]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[27]  S. Inokuchi,et al.  Range-imaging system for 3-D object recognition , 1984 .

[28]  Johann Cussey,et al.  Time-of-flight range detection using low-frequency intensity modulation of a cw laser diode: application to fiber length measurement , 2008 .

[29]  Sven Behnke,et al.  Multi-frequency Phase Unwrapping for Time-of-Flight cameras , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  David Cohen,et al.  A fast global shutter image sensor based on the VOD mechanism , 2014, IEEE SENSORS 2014 Proceedings.

[31]  Michael J. Cree,et al.  Understanding and Ameliorating Mixed Pixels and Multipath Interference in AMCW Lidar , 2013 .

[32]  Alice Biber,et al.  Demodulation pixels in CCD and CMOS technologies for time-of-flight ranging , 2000, Electronic Imaging.

[33]  Frédo Durand,et al.  Noise-optimal capture for high dynamic range photography , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Gordon Wetzstein,et al.  Computational imaging with multi-camera time-of-flight systems , 2016, ACM Trans. Graph..

[35]  Ramesh Raskar,et al.  Coded time of flight cameras , 2013, ACM Trans. Graph..

[36]  John F. Wakerly,et al.  Digital design - principles and practices , 1990, Prentice Hall Series in computer engineering.

[37]  Michael J. Cree,et al.  Illumination waveform optimization for time-of-flight range imaging cameras , 2011, Optical Metrology.

[38]  Rahul Nair,et al.  Simulation of Time-of-Flight Sensors using Global Illumination , 2013, VMV.