Occluded Imaging with Time-of-Flight Sensors

We explore the question of whether phase-based time-of-flight (TOF) range cameras can be used for looking around corners and through scattering diffusers. By connecting TOF measurements with theory from array signal processing, we conclude that performance depends on two primary factors: camera modulation frequency and the width of the specular lobe (“shininess”) of the wall. For purely Lambertian walls, commodity TOF sensors achieve resolution on the order of meters between targets. For seemingly diffuse walls, such as posterboard, the resolution is drastically reduced, to the order of 10cm. In particular, we find that the relationship between reflectance and resolution is nonlinear—a slight amount of shininess can lead to a dramatic improvement in resolution. Since many realistic scenes exhibit a slight amount of shininess, we believe that off-the-shelf TOF cameras can look around corners.

[1]  Volkan Cevher,et al.  Distributed target localization via spatial sparsity , 2008, 2008 16th European Signal Processing Conference.

[2]  Pat Hanrahan,et al.  A signal-processing framework for inverse rendering , 2001, SIGGRAPH.

[3]  G. Carter.  Time delay estimation for passive sonar signal processing , 1981 .

[4]  Ramesh Raskar,et al.  Single view reflectance capture using multiplexed scattering and time-of-flight imaging , 2011, SA '11.

[5]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[6]  Victor Isakov,et al.  On nearfield acoustic holography , 2004 .

[7]  Kavita Bala,et al.  Proceedings of the Sixteenth Eurographics conference on Rendering Techniques , 2005 .

[8]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[9]  Ivo Ihrke,et al.  Mirrors in Computer Graphics, Computer Vision and Time-of-Flight Imaging , 2013, Time-of-Flight and Depth Imaging.

[10]  J. Gower.  Properties of Euclidean and non-Euclidean distance matrices , 1985 .

[11]  Frédo Durand,et al.  Experimental analysis of BRDF models , 2005, EGSR '05.

[12]  Ramesh Raskar,et al.  Resolving Multi-path Interference in Time-of-Flight Imaging via Modulation Frequency Diversity and Sparse Regularization , 2014, Optics letters.

[13]  Adrian Stern,et al.  Reconstruction guarantees for compressive tomographic holography. , 2013, Optics letters.

[14]  Ramesh Raskar,et al.  Coded time of flight cameras , 2013, ACM Trans. Graph..

[15]  Eero P. Simoncelli,et al.  Recovery of Sparse Translation-Invariant Signals With Continuous Basis Pursuit , 2011, IEEE Transactions on Signal Processing.

[16]  Harry L. Van Trees,et al.  Optimum Array Processing: Part IV of Detection, Estimation, and Modulation Theory , 2002 .

[17]  Stefan Nilsson,et al.  Radar Detection of Moving Targets Behind Corners , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Matthew O'Toole,et al.  Temporal frequency probing for 5D transient analysis of global light transport , 2014, ACM Trans. Graph..

[19]  Rob Miller,et al.  3D Tracking via Body Radio Reflections , 2014, NSDI.

[20]  Diego Gutierrez,et al.  Femto-photography , 2013, ACM Trans. Graph..

[21]  Dmitry M. Malioutov,et al.  A sparse signal reconstruction perspective for source localization with sensor arrays , 2005, IEEE Transactions on Signal Processing.

[22]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[23]  Bhiksha Raj,et al.  Joint sparsity models for wideband array processing , 2011, Optical Engineering + Applications.

[24]  Qionghai Dai,et al.  Ultra-fast Lensless Computational Imaging through 5D Frequency Analysis of Time-resolved Light Transport , 2013, International Journal of Computer Vision.

[25]  MOHIT GUPTA,et al.  Phasor Imaging , 2015, ACM Trans. Graph..

[26]  Qionghai Dai,et al.  Fourier Analysis on Transient Imaging with a Multifrequency Time-of-Flight Camera , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Ashok Veeraraghavan,et al.  Depth Fields: Extending Light Field Techniques to Time-of-Flight Imaging , 2015, 2015 International Conference on 3D Vision.

[28]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[29]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.

[30]  Arye Nehorai,et al.  Adaptive OFDM Radar for Target Detection in Multipath Scenarios , 2011, IEEE Transactions on Signal Processing.

[31]  Diego Gutierrez,et al.  Relativistic ultrafast rendering using time-of-flight imaging , 2012, SIGGRAPH Talks.

[32]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[33]  Piotr Indyk,et al.  Approximation-Tolerant Model-Based Compressive Sensing , 2014, SODA.

[34]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[35]  J. D. Maynard,et al.  Nearfield acoustic holography: I. Theory of generalized holography and the development of NAH , 1985 .

[36]  Frédo Durand,et al.  Micron-scale light transport decomposition using interferometry , 2015, ACM Trans. Graph..

[37]  Wolfgang Heidrich,et al.  Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination Using Inexpensive Time-of-Flight Sensors , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[39]  Michael Wimmer,et al.  O-snap , 2013, ACM Trans. Graph..

[40]  R. Ramamoorthi,et al.  Frequency domain normal map filtering , 2007, SIGGRAPH 2007.

[41]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[42]  Hans-Peter Seidel,et al.  Three-dimensional kaleidoscopic imaging , 2011, CVPR 2011.

[43]  Belén Masiá Corcoy,et al.  Computational imaging: Combining optics, computation and perception , 2013 .