Reconstructing rooms using photon echoes: A plane based model and reconstruction algorithm for looking around the corner

Can we reconstruct the entire internal shape of a room if all we can directly observe is a small portion of one internal wall, presumably through a window in the room? While conventional wisdom may indicate that this is not possible, motivated by recent work on ‘looking around corners’, we show that one can exploit light echoes to reconstruct the internal shape of hidden rooms. Existing techniques for looking around the corner using transient images model the hidden volume using voxels and try to explain the captured transient response as the sum of the transient responses obtained from individual voxels. Such a technique inherently suffers from challenges with regards to low signal to background ratios (SBR) and has difficulty scaling to larger volumes. In contrast, in this paper, we argue for using a plane-based model for the hidden surfaces. We demonstrate that such a plane-based model results in much higher SBR while simultaneously being amenable to larger spatial scales. We build an experimental prototype composed of a pulsed laser source and a single-photon avalanche detector (SPAD) that can achieve a time resolution of about 30ps and demonstrate high-fidelity reconstructions both of individual planes in a hidden volume and for reconstructing entire polygonal rooms composed of multiple planar walls.

[1]  Alberto Tosi,et al.  Time-gated single-photon detection module with 110 ps transition time and up to 80 MHz repetition rate. , 2014, The Review of scientific instruments.

[2]  Ashok Veeraraghavan,et al.  Spatial Phase-Sweep: Increasing temporal resolution of transient imaging using a light source array , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[3]  Ramesh Raskar,et al.  Occluded Imaging with Time-of-Flight Sensors , 2016, ACM Trans. Graph..

[4]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.

[5]  Ashok Veeraraghavan,et al.  Linear systems approach to identifying performance bounds in indirect imaging , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Vivek K. Goyal,et al.  Diffuse Imaging: Creating Optical Images With Unfocused Time-Resolved Illumination and Sensing , 2012, IEEE Signal Processing Letters.

[7]  Jaime Martín,et al.  Tracking objects outside the line of sight using 2D intensity images , 2016, Scientific Reports.

[8]  K. Eliceiri,et al.  Non-line-of-sight imaging using a time-gated single photon avalanche diode. , 2015, Optics express.

[9]  Nils Abramson,et al.  Light-In-Flight Recording By Holography , 1980, Photonics West - Lasers and Applications in Science and Engineering.

[10]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[11]  R. Raskar,et al.  Erratum: Single-photon sensitive light-in-flight imaging , 2015, Nature communications.

[12]  D. O'connor,et al.  Time-Correlated Single Photon Counting , 1984 .

[13]  A. Lacaita,et al.  Avalanche photodiodes and quenching circuits for single-photon detection. , 1996, Applied optics.

[14]  W. Brockherde,et al.  CMOS SPADs with up to 500 μm diameter and 55% detection efficiency at 420 nm , 2014 .

[15]  Vivek K Goyal,et al.  Exploiting sparsity in time-of-flight range acquisition using a single time-resolved sensor. , 2011, Optics express.

[16]  Ramesh Raskar,et al.  Looking around the corner using transient imaging , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[18]  Martin Vetterli,et al.  Can one hear the shape of a room: The 2-D polygonal case , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[20]  Ashok Veeraraghavan,et al.  Depth Selective Camera: A Direct, On-Chip, Programmable Technique for Depth Selectivity in Photography , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Jan-Michael Frahm,et al.  Piecewise planar and non-planar stereo for urban scene reconstruction , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Wolfgang Heidrich,et al.  Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination Using Inexpensive Time-of-Flight Sensors , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Vivek K. Goyal,et al.  Photon-Efficient Computational 3-D and Reflectivity Imaging With Single-Photon Detectors , 2014, IEEE Transactions on Computational Imaging.

[24]  Zihan Zhou,et al.  Robust plane-based structure from motion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Kiriakos N. Kutulakos,et al.  The Geometry of First-Returning Photons for Non-Line-of-Sight Imaging , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Chiye Li,et al.  Single-shot compressed ultrafast photography at one hundred billion frames per second , 2014, Nature.

[27]  Ramesh Raskar,et al.  Slow art with a trillion frames per second camera , 2011, SIGGRAPH '11.

[28]  Jon C. Dattorro,et al.  Convex Optimization & Euclidean Distance Geometry , 2004 .

[29]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[30]  R. A. P. ROGERS,et al.  Perimeter of an Ellipse. , 1920, Nature.

[31]  Diego Gutierrez,et al.  Femto-photography , 2013, ACM Trans. Graph..

[32]  Qionghai Dai,et al.  Fourier Analysis on Transient Imaging with a Multifrequency Time-of-Flight Camera , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Martin Vetterli,et al.  Acoustic echoes reveal room shape , 2013, Proceedings of the National Academy of Sciences.

[34]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[35]  O. Katz,et al.  Looking around corners and through thin turbid layers in real time with scattered incoherent light , 2012, Nature Photonics.

[36]  M. Levandowsky,et al.  Distance between Sets , 1971, Nature.

[37]  Ramesh Raskar,et al.  Coded time of flight cameras , 2013, ACM Trans. Graph..

[38]  Prasanna Rangarajan,et al.  Imaging hidden objects by transforming scattering surfaces into computational holographic sensors , 2016 .

[39]  Vivek K Goyal,et al.  First-Photon Imaging , 2014, Science.

[40]  Robert Henderson,et al.  Detection and tracking of moving objects hidden from view , 2015, Nature Photonics.

[41]  W. Becker Advanced Time-Correlated Single Photon Counting Techniques , 2005 .

[42]  Ramesh Raskar,et al.  Reconstruction of hidden 3D shapes using diffuse reflections , 2012, Optics express.

[43]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.