Ellipsoidal path connections for time-gated rendering

During the last decade, we have been witnessing the continued development of new time-of-flight imaging devices, and their increased use in numerous and varied applications. However, physics-based rendering techniques that can accurately simulate these devices are still lacking: while existing algorithms are adequate for certain tasks, such as simulating transient cameras, they are very inefficient for simulating time-gated cameras because of the large number of wasted path samples. We take steps towards addressing these deficiencies, by introducing a procedure for efficiently sampling paths with a predetermined length, and incorporating it within rendering frameworks tailored towards simulating time-gated imaging. We use our open-source implementation of the above to empirically demonstrate improved rendering performance in a variety of applications, including simulating proximity sensors, imaging through occlusions, depth-selective cameras, transient imaging in dynamic scenes, and non-line-of-sight imaging.

[1]  Diego Gutierrez,et al.  Femto-photography , 2013, ACM Trans. Graph..

[2]  Qionghai Dai,et al.  Decomposing Global Light Transport Using Time of Flight Imaging , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Leonidas J. Guibas,et al.  Metropolis light transport , 1997, SIGGRAPH.

[4]  Gordon Wetzstein,et al.  Computational imaging with multi-camera time-of-flight systems , 2016, ACM Trans. Graph..

[5]  Daniele Faccio,et al.  Time-of-Flight Imaging at 10 ps Resolution with an ICCD Camera , 2019, Sensors.

[6]  Qionghai Dai,et al.  Ultra-fast Lensless Computational Imaging through 5D Frequency Analysis of Time-resolved Light Transport , 2013, International Journal of Computer Vision.

[7]  Jaakko Lehtinen,et al.  Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering , 2015, Comput. Graph. Forum.

[8]  Toshiya Hachisuka,et al.  Multiplexed metropolis light transport , 2014, ACM Trans. Graph..

[9]  Klaus C. J. Dietmayer,et al.  Gated2Depth: Real-Time Dense Lidar From Gated Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[10]  Wolfgang Heidrich,et al.  Diffuse Mirrors: 3D Reconstruction from Diffuse Indirect Illumination Using Inexpensive Time-of-Flight Sensors , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  H Szmacinski,et al.  Fluorescence lifetime imaging. , 1992, Analytical biochemistry.

[12]  Steve Marschner,et al.  Manifold exploration , 2012, ACM Trans. Graph..

[13]  H. Jensen,et al.  Progressive photon mapping , 2008, SIGGRAPH 2008.

[14]  Yoav Grauer,et al.  Active gated imaging for automotive safety applications , 2015, Electronic Imaging.

[15]  Sato Imari,et al.  Inverse Rendering for Computer Graphics , 2010 .

[16]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[17]  Jeremy Copley,et al.  A low-noise laser-gated imaging system for long-range target identification , 2004, SPIE Defense + Commercial Sensing.

[18]  Aswin C. Sankaranarayanan,et al.  Beyond Volumetric Albedo — A Surface Optimization Framework for Non-Line-Of-Sight Imaging , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  MOHIT GUPTA,et al.  Phasor Imaging , 2015, ACM Trans. Graph..

[20]  Qionghai Dai,et al.  Fourier Analysis on Transient Imaging with a Multifrequency Time-of-Flight Camera , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Jonathan Leach,et al.  Non-line-of-sight tracking of people at long range , 2017, Optics express.

[22]  Ashok Veeraraghavan,et al.  Reconstructing rooms using photon echoes: A plane based model and reconstruction algorithm for looking around the corner , 2017, 2017 IEEE International Conference on Computational Photography (ICCP).

[23]  Shichao Xu,et al.  Comparison between traditional and heterodyned optical Kerr gated imaging for CS2 and Te glass Kerr media , 2016 .

[24]  Norman S Kopeika,et al.  Range gated active night vision system for automobiles. , 2006, Applied optics.

[25]  Wolfgang Zinth,et al.  A broadband Kerr shutter for femtosecond fluorescence spectroscopy , 2003 .

[26]  Frédo Durand,et al.  Micron-scale light transport decomposition using interferometry , 2015, ACM Trans. Graph..

[27]  S. Marschner,et al.  Inverse Rendering for Computer Graphics , 1998 .

[28]  Jaakko Lehtinen,et al.  Differentiable Monte Carlo ray tracing through edge sampling , 2018, ACM Trans. Graph..

[29]  Diego Gutierrez,et al.  A framework for transient rendering , 2014, ACM Trans. Graph..

[30]  Ashok Veeraraghavan,et al.  SNLOS: Non-line-of-sight Scanning through Temporal Focusing , 2019, 2019 IEEE International Conference on Computational Photography (ICCP).

[31]  Ramesh Raskar,et al.  Towards photography through realistic fog , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[32]  Gordon Wetzstein,et al.  Confocal non-line-of-sight imaging based on the light-cone transform , 2018, Nature.

[33]  Leonidas J. Guibas,et al.  Robust Monte Carlo methods for light transport simulation , 1997 .

[34]  Shree K. Nayar,et al.  What Are Optimal Coding Functions for Time-of-Flight Imaging? , 2018, ACM Trans. Graph..

[35]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[36]  Ashok Veeraraghavan,et al.  Depth Selective Camera: A Direct, On-Chip, Programmable Technique for Depth Selectivity in Photography , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Ramesh Raskar,et al.  Looking around the corner using transient imaging , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[38]  H. Jensen.  Realistic Image Synthesis Using Photon Mapping , 2001 .

[39]  Diego Gutierrez,et al.  Progressive Transient Photon Beams , 2018, Comput. Graph. Forum.

[40]  Philippe Bekaert,et al.  Advanced global illumination , 2006 .

[41]  Gordon Wetzstein,et al.  Non-line-of-sight Imaging with Partial Occluders and Surface Normals , 2017, ACM Trans. Graph..

[42]  R. Raskar,et al.  Single-photon sensitive light-in-fight imaging , 2015, Nature Communications.

[43]  Philippe Bekaert,et al.  Advanced Global Illumination, Second Edition , 2006 .

[44]  G. Rybicki.  Radiative transfer , 2019, Climate Change and Terrestrial Ecosystem Modeling.

[45]  Ashok Veeraraghavan,et al.  Linear systems approach to identifying performance bounds in indirect imaging , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[46]  Ramesh Raskar,et al.  Single view reflectance capture using multiplexed scattering and time-of-flight imaging , 2011, SA '11.

[47]  Jaakko Lehtinen,et al.  Aether , 2017, ACM Trans. Graph..

[48]  Min H. Kim,et al.  DeepToF: off-the-shelf real-time correction of multipath interference in time-of-flight imaging , 2017, ACM Trans. Graph..

[49]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.

[50]  K. Eliceiri,et al.  Non-line-of-sight imaging using a time-gated single photon avalanche diode. , 2015, Optics express.

[51]  Matthew O'Toole,et al.  Temporal frequency probing for 5D transient analysis of global light transport , 2014, ACM Trans. Graph..

[52]  Steve Marschner,et al.  Microfacet Models for Refraction through Rough Surfaces , 2007, Rendering Techniques.

[53]  Diego Gutierrez,et al.  Transient photon beams , 2017, SIGGRAPH Posters.

[54]  Daniele Faccio,et al.  A trillion frames per second: the techniques and applications of light-in-flight photography , 2018, Reports on progress in physics. Physical Society.

[55]  Kiriakos N. Kutulakos,et al.  The Geometry of First-Returning Photons for Non-Line-of-Sight Imaging , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Kiriakos N. Kutulakos,et al.  A Theory of Fermat Paths for Non-Line-Of-Sight Shape Reconstruction , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  Diego Gutierrez,et al.  Fast back-projection for non-line of sight reconstruction , 2017, SIGGRAPH Posters.

[58]  Leonidas J. Guibas,et al.  Optimally combining sampling techniques for Monte Carlo rendering , 1995, SIGGRAPH.

[59]  Daniel Weiskopf,et al.  Refractive radiative transfer equation , 2014, TOGS.

[60]  Jeffrey H. Shapiro,et al.  Exploiting Occlusion in Non-Line-of-Sight Active Imaging , 2017, IEEE Transactions on Computational Imaging.

[61]  Diego Gutierrez,et al.  Recent advances in transient imaging: A computer graphics and vision perspective , 2016, Vis. Informatics.

[62]  Yves D. Willems,et al.  Rendering Participating Media with Bidirectional Path Tracing , 1996, Rendering Techniques.

[63]  Reinhard Klein,et al.  Solving trigonometric moment problems for fast transient imaging , 2015, ACM Trans. Graph..

[64]  Anat Levin,et al.  An Evaluation of Computational Imaging Techniques for Heterogeneous Inverse Scattering , 2016, ECCV.

[65]  R R Alfano,et al.  Ultrafast time-gated imaging in thick tissues: a step toward optical mammography. , 1993, Optics letters.

[66]  Martin Laurenzis,et al.  Nonline-of-sight laser gated viewing of scattered photons , 2014 .

[67]  Katsuyuki Kinoshita,et al.  New femtosecond streak camera with temporal resolution of 180 fs , 1994, Photonics West - Lasers and Applications in Science and Engineering.

[68]  R. Raskar,et al.  All Photons Imaging Through Volumetric Scattering , 2016, Scientific Reports.

[69]  Leonidas J. Guibas,et al.  Bidirectional Estimators for Light Transport , 1995 .