Tracking objects outside the line of sight using 2D intensity images

The observation of objects located in inaccessible regions is a recurring challenge in a wide variety of important applications. Recent work has shown that using rare and expensive optical setups, indirect diffuse light reflections can be used to reconstruct objects and two-dimensional (2D) patterns around a corner. Here we show that occluded objects can be tracked in real time using much simpler means, namely a standard 2D camera and a laser pointer. Our method fundamentally differs from previous solutions by approaching the problem in an analysis-by-synthesis sense. By repeatedly simulating light transport through the scene, we determine the set of object parameters that most closely fits the measured intensity distribution. We experimentally demonstrate that this approach is capable of following the translation of unknown objects, and translation and orientation of a known object, in real time.

[1]  D. Marquardt An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[2]  Nils Abramson,et al.  Light-In-Flight Recording By Holography , 1980, Photonics West - Lasers and Applications in Science and Engineering.

[3]  N. Abramson Light-in-flight recording: high-speed holographic motion pictures of ultrafast phenomena. , 1983, Applied optics.

[4]  Donald P. Greenberg,et al.  Modeling the interaction of light between diffuse surfaces , 1984, SIGGRAPH.

[5]  F. Quercioli,et al.  White light-in-flight holography. , 1985, Applied optics.

[6]  Y. Çengel Heat and Mass Transfer: Fundamentals and Applications , 2000 .

[7]  Marc Levoy,et al.  Dual photography , 2005, SIGGRAPH 2005.

[8]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[9]  B. Jalali,et al.  Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena , 2009, Nature.

[10]  M. Vannier,et al.  Why do commercial CT scanners still employ traditional, filtered back-projection for image reconstruction? , 2009, Inverse problems.

[11]  R. Raskar,et al.  Picosecond Camera for Time-of-Flight Imaging , 2011 .

[12]  Stefan Nilsson,et al.  Radar Detection of Moving Targets Behind Corners , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Magnus Elmqvist,et al.  See around the corner using active imaging , 2011, Security + Defence.

[14]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.

[15]  O. Katz,et al.  Looking around corners and through thin turbid layers in real time with scattered incoherent light , 2012, Nature Photonics.

[16]  Pedro Arias,et al.  Review of mobile mapping and surveying technologies , 2013 .

[17]  Fadel Adib,et al.  See through walls with WiFi! , 2013, SIGCOMM.

[18]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[19]  Ramesh Raskar,et al.  Coded time of flight cameras , 2013, ACM Trans. Graph..

[20]  Martin Laurenzis,et al.  Nonline-of-sight laser gated viewing of scattered photons , 2014 .

[21]  M. Fink,et al.  Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations , 2014, Nature Photonics.

[22]  Wolfgang Heidrich,et al.  Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination Using Inexpensive Time-of-Flight Sensors , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Frédo Durand,et al.  Capturing the human figure through a wall , 2015, ACM Trans. Graph..

[24]  K. Eliceiri,et al.  Non-line-of-sight imaging using a time-gated single photon avalanche diode. , 2015, Optics express.

[25]  Reinhard Klein,et al.  Solving trigonometric moment problems for fast transient imaging , 2015, ACM Trans. Graph..

[26]  R. Raskar,et al.  Single-photon sensitive light-in-fight imaging , 2015, Nature Communications.

[27]  R. Raskar,et al.  Erratum: Single-photon sensitive light-in-flight imaging , 2015, Nature communications.

[28]  Robert Henderson,et al.  Detection and tracking of moving objects hidden from view , 2015, Nature Photonics.

[29]  Ramesh Raskar,et al.  Occluded Imaging with Time-of-Flight Sensors , 2016, ACM Trans. Graph..