Acquiring and characterizing plane-to-ray indirect light transport

Separation of light transport into direct and indirect paths has enabled new visualizations of light in everyday scenes. However, indirect light itself contains a variety of components from subsurface scattering to diffuse and specular interreflections, all of which contribute to complex visual appearance. In this paper, we present a new imaging technique that captures and analyzes these components of indirect light via light transport between epipolar planes of illumination and rays of received light. This plane-to-ray light transport is captured using a rectified projector-camera system where we vary the offset between projector and camera rows (implemented as synchronization delay) as well as the exposure of each camera row. The resulting delay-exposure stack of images can capture live short and long-range indirect light transport, disambiguate subsurface scattering, diffuse and specular interreflections, and distinguish materials according to their subsurface scattering properties.

[1]  Tian-Tsong Ng,et al.  A Dual Theory of Inverse and Forward Light Transport , 2010, ECCV.

[2]  Ashok Veeraraghavan,et al.  Flexible Voxels for Motion-Aware Videography , 2010, ECCV.

[3]  Kiriakos N. Kutulakos,et al.  The Geometry of First-Returning Photons for Non-Line-of-Sight Imaging , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Stephen Lin,et al.  Diffuse-Specular Separation and Depth Recovery from Image Sequences , 2002, ECCV.

[5]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[6]  Pat Hanrahan,et al.  All-frequency shadows using non-linear wavelet lighting approximation , 2003, ACM Trans. Graph..

[7]  Matthew O'Toole,et al.  Decomposing global light transport using time of flight imaging , 2012, CVPR.

[8]  Marc Levoy,et al.  Symmetric photography: exploiting data-sparseness in reflectance fields , 2006, EGSR '06.

[9]  Yoav Y. Schechner,et al.  Illumination Multiplexing within Fundamental Limits , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Shree K. Nayar,et al.  Video from a single coded exposure photograph using a learned over-complete dictionary , 2011, 2011 International Conference on Computer Vision.

[11]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[12]  Stephen P. Boyd,et al.  CVXPY: A Python-Embedded Modeling Language for Convex Optimization , 2016, J. Mach. Learn. Res..

[13]  Ashok Veeraraghavan,et al.  Structured light 3D scanning in the presence of global illumination , 2011, CVPR 2011.

[14]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[15]  Shuntaro Yamazaki,et al.  Exploiting DLP Illumination Dithering for Reconstruction and Photography of High-Speed Scenes , 2011, International Journal of Computer Vision.

[16]  Soheil Darabi,et al.  Compressive Dual Photography , 2009, Comput. Graph. Forum.

[17]  Shree K. Nayar,et al.  Separation of Reflection Components Using Color and Polarization , 1997, International Journal of Computer Vision.

[18]  Pieter Peers,et al.  Compressive light transport sensing , 2009, ACM Trans. Graph..

[19]  Guy Godin,et al.  Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Donald P. Greenberg,et al.  Modeling the interaction of light between diffuse surfaces , 1984, SIGGRAPH.

[21]  Matthew O'Toole,et al.  Homogeneous codes for energy-efficient illumination and imaging , 2015, ACM Trans. Graph..

[22]  Tomoo Mitsunaga,et al.  Coded rolling shutter photography: Flexible space-time sampling , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[23]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, ACM Trans. Graph..

[24]  Shree K. Nayar,et al.  Multiplexing for Optimal Lighting , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Paul Debevec,et al.  The Light Stages and Their Applications to Photoreal Digital Actors , 2012, SIGGRAPH 2012.

[26]  Kiriakos N. Kutulakos,et al.  Optical computing for fast light transport analysis , 2010, SIGGRAPH 2010.

[27]  Kiriakos N. Kutulakos,et al.  A theory of inverse light transport , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[28]  James T. Kajiya,et al.  The rendering equation , 1998 .

[29]  Leonidas J. Guibas,et al.  Robust Monte Carlo methods for light transport simulation , 1997 .

[30]  William Whittaker,et al.  Epipolar time-of-flight imaging , 2017, ACM Trans. Graph..

[31]  Steve Marschner,et al.  A practical model for subsurface light transport , 2001, SIGGRAPH.

[32]  Wolfgang Heidrich,et al.  Material Classification Using Raw Time-of-Flight Measurements , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Matthew O'Toole,et al.  3D Shape and Indirect Appearance by Structured Light Transport , 2014, CVPR.

[34]  Matthew O'Toole,et al.  Primal-dual coding to probe light transport , 2012, ACM Trans. Graph..

[35]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH 2006.