Phasor Imaging

In correlation-based time-of-flight (C-ToF) imaging systems, light sources with temporally varying intensities illuminate the scene. Due to global illumination, the temporally varying radiance received at the sensor is a combination of light received along multiple paths. Recovering scene properties (e.g., scene depths) from the received radiance requires separating these contributions, which is challenging due to the complexity of global illumination and the additional temporal dimension of the radiance. We propose phasor imaging, a framework for performing fast inverse light transport analysis using C-ToF sensors. Phasor imaging is based on the idea that, by representing light transport quantities as phasors and light transport events as phasor transformations, light transport analysis can be simplified in the temporal frequency domain. We study the effect of temporal illumination frequencies on light transport and show that, for a broad range of scenes, global radiance (inter-reflections and volumetric scattering) vanishes for frequencies higher than a scene-dependent threshold. We use this observation for developing two novel scene recovery techniques. First, we present micro-ToF imaging, a ToF-based shape recovery technique that is robust to errors due to inter-reflections (multipath interference) and volumetric scattering. Second, we present a technique for separating the direct and global components of radiance. Both techniques require capturing as few as 3--4 images and minimal computations. We demonstrate the validity of the presented techniques via simulations and experiments performed with our hardware prototype.

[1]  Helmut Fischer,et al.  New electro-optical mixing and correlating sensor: facilities and applications of the photonic mixer device (PMD) , 1997, Other Conferences.

[2]  J. Rarity,et al.  High-resolution random-modulation cw lidar. , 2011, Applied optics.

[3]  Henning Heiselberg,et al.  High accuracy 3D laser radar , 2004, SPIE Defense + Commercial Sensing.

[4]  Ramesh Raskar,et al.  Looking around the corner using transient imaging , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  VeeraraghavanAshok,et al.  A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus , 2013 .

[6]  Diego Gutierrez,et al.  Femto-photography , 2013, ACM Trans. Graph..

[7]  Dale A Carnegie,et al.  Improved measurement linearity and precision for AMCW time-of-flight range imaging cameras. , 2010, Applied optics.

[8]  Ramesh Raskar,et al.  Estimating Motion and size of moving non-line-of-sight objects in cluttered environments , 2011, CVPR 2011.

[9]  Shree K. Nayar,et al.  Micro Phase Shifting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Gen-Ming Jin,et al.  Isospin dependence of radial flow in heavy-ion collisions at intermediate energies , 1999 .

[11]  Matthew O'Toole,et al.  Primal-dual coding to probe light transport , 2012, ACM Trans. Graph..

[12]  Peter Seitz,et al.  Pseudonoise Optical Modulation for Real-Time 3-D Imaging With Minimum Interference , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[13]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[14]  Qionghai Dai,et al.  Decomposing Global Light Transport Using Time of Flight Imaging , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Shree K. Nayar,et al.  Structured Light in Sunlight , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Michael J. Cree,et al.  Multiple return separation for a full-field ranger via continuous waveform modelling , 2009, Electronic Imaging.

[17]  Manuel Mazo,et al.  Modelling and correction of multipath interference in time of flight cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  W. Koechner Optical Ranging System Employing a High Power Injection Laser Diode , 1968, IEEE Transactions on Aerospace and Electronic Systems.

[19]  Olaf Hellwich,et al.  Compensation for Multipath in ToF Camera Measurements Supported by Photometric Calibration and Environment Integration , 2013, ICVS.

[20]  Hans-Peter Seidel,et al.  Modulated phase-shifting for 3D scanning , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Michael J. Cree,et al.  Illumination waveform optimization for time-of-flight range imaging cameras , 2011, Optical Metrology.

[22]  Milton Feng,et al.  Tilted-charge high speed (7 GHz) light emitting diode , 2009 .

[23]  Qionghai Dai,et al.  Frequency Analysis of Transient Light Transport with Applications in Bare Sensor Imaging , 2012, ECCV.

[24]  Dale A. Carnegie,et al.  Design and Construction of a Configurable Full-Field Range Imaging System for Mobile Robotic Applications , 2011 .

[25]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[26]  Nicolas Martin,et al.  Unstructured Light Scanning Robust to Indirect Illumination and Depth Discontinuities , 2014, International Journal of Computer Vision.

[27]  Ramesh Raskar,et al.  Single view reflectance capture using multiplexed scattering and time-of-flight imaging , 2011, SA '11.

[28]  L. C. Henyey,et al.  Diffuse radiation in the Galaxy , 1940 .

[29]  Shree K. Nayar,et al.  Multiplexed illumination for scene recovery in the presence of global illumination , 2011, 2011 International Conference on Computer Vision.

[30]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.

[31]  S. Burak Gokturk,et al.  A Time-Of-Flight Depth Sensor - System Description, Issues and Solutions , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[32]  Matthew O'Toole,et al.  Temporal frequency probing for 5D transient analysis of global light transport , 2014, ACM Trans. Graph..

[33]  Kiriakos N. Kutulakos,et al.  A theory of inverse light transport , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[34]  Michael J. Cree,et al.  Separating true range measurements from multi-path and scattering interference in commercial range cameras , 2011, Electronic Imaging.

[35]  Yu. N. Solodkin,et al.  Automatic processing of fringe patterns in integer interferometers , 1991 .

[36]  Yuandong Tian,et al.  (De) focusing on global light transport for active scene recovery , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Ramesh Raskar,et al.  Coded time of flight cameras , 2013, ACM Trans. Graph..

[38]  Eli Yablonovitch,et al.  GHz bandwidth GaAs light-emitting diodes , 1999 .

[39]  Dale A. Carnegie,et al.  Maximizing precision over extended unambiguous range for TOF range imaging systems , 2010, 2010 IEEE Instrumentation & Measurement Technology Conference Proceedings.

[40]  Michael J. Cree,et al.  Closed-form inverses for the mixed pixel/multipath interference problem in AMCW lidar , 2012, Electronic Imaging.

[41]  M. Melloch,et al.  Digital communications above 1 Gb/s using 890-nm surface-emitting light-emitting diodes , 2001, IEEE Photonics Technology Letters.

[42]  Johann Cussey,et al.  Time-of-flight range detection using low-frequency intensity modulation of a cw laser diode: application to fiber length measurement , 2008 .

[43]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, SIGGRAPH 2006.

[44]  Frédo Durand,et al.  Noise-optimal capture for high dynamic range photography , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[45]  Dikpal Reddy,et al.  Frequency-Space Decomposition and Acquisition of Light Transport under Spatially Varying Illumination , 2012, ECCV.

[46]  Shree K. Nayar,et al.  Acquiring scattering properties of participating media by dilution , 2006, ACM Trans. Graph..

[47]  Ashok Veeraraghavan,et al.  A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus , 2013, International Journal of Computer Vision.

[48]  M. Takeda,et al.  Frequency-multiplex Fourier-transform profilometry: a single-shot three-dimensional shape measurement of objects with large height discontinuities and/or surface isolations. , 1997, Applied optics.

[49]  J. Heinen,et al.  Light-emitting diodes with a modulation bandwidth of more than 1 GHz , 1976 .

[50]  Mirko Schmidt,et al.  SRA: Fast Removal of General Multipath for ToF Sensors , 2014, ECCV.

[51]  R. Lange,et al.  Solid-state time-of-flight range camera , 2001 .

[52]  Dale Anthony Carnegie,et al.  Analysis of Errors in ToF Range Imaging With Dual-Frequency Modulation , 2011, IEEE Transactions on Instrumentation and Measurement.

[53]  M. Cree,et al.  Mixed pixel return separation for a full-field ranger , 2008, 2008 23rd International Conference Image and Vision Computing New Zealand.

[54]  Philip A. Chou,et al.  SPUMIC: Simultaneous phase unwrapping and multipath interference cancellation in time-of-flight cameras using spectral methods , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[55]  Rudolf Schwarte,et al.  MSM-PMD as correlation receiver in a new 3D-ranging system , 2002, SPIE Remote Sensing.

[56]  Stefan Fuchs,et al.  Multipath Interference Compensation in Time-of-Flight Camera Images , 2010, 2010 20th International Conference on Pattern Recognition.

[57]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[58]  Milton Feng,et al.  Design and layout of multi ghz operation of light emitting diodes , 2010 .

[59]  Rudolf Schwarte,et al.  Breakthrough in multichannel laser-radar technology providing thousands of high-sensitive lidar receivers on a chip , 2004, SPIE Remote Sensing.