Polarimetric spatio-temporal light transport probing

Fig. 1. We propose a computational light transport probingmethod that decomposes transport into full polarization, spatial and temporal dimensions.Wemodel this multi-dimensional light transport as a tensor and analyze low-rank structure in the polarization domain which is exploited by our polarimetric probing method. We instantiate our approach with two imaging systems for spatio-polarimetric and coaxial temporal-polarimetric capture. (a)&(d) Conventional intensity imagers integrate incident light intensity over space and time independently of the polarization states of light, losing geometric and material information encoded in the polarimetric transport. Capturing polarization-resolved spatial transport components of (b) epipolar and (c) non-epipolar dimensions enable fine-grained decomposition of light transport. Combining temporal and polarimetric dimensions, we separate (e) geometry-dependent reflections and (f) direct/indirect reflections that cannot be resolved in the temporal-only measurements.

[1]  R. Chipman,et al.  Interpretation of Mueller matrices based on polar decomposition , 1996 .

[2]  H. Arwin.  Application of ellipsometry techniques to biological materials , 2011 .

[3]  Gordon Wetzstein,et al.  Reconstructing Transient Images from Single-Photon Sensors , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Yasuhiro Mukaigawa,et al.  Enhancing Passive Non-Line-of-Sight Imaging Using Polarization Cues , 2019, ArXiv.

[5]  Kristin J. Dana,et al.  Shape from Sky: Polarimetric Normal Recovery Under The Sky , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Min H. Kim,et al.  Birefractive stereo imaging for single-shot depth acquisition , 2016, ACM Trans. Graph..

[7]  Abhijeet Ghosh,et al.  Polarization imaging reflectometry in the wild , 2017, ACM Trans. Graph..

[8]  Ramesh Raskar,et al.  Dynamic heterodyne interferometry , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[9]  R. Azzam,et al.  Stokes-vector and Mueller-matrix polarimetry [Invited]. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[10]  Ashok Veeraraghavan,et al.  Structured light 3D scanning in the presence of global illumination , 2011, CVPR 2011.

[11]  Yongdong Zhang,et al.  Scattering robust 3D reconstruction via polarized transient imaging. , 2016, Optics letters.

[12]  Matthew O'Toole,et al.  Optical computing for fast light transport analysis , 2010, ACM Trans. Graph..

[13]  Gordon Wetzstein,et al.  Computational Plenoptic Imaging , 2011, SIGGRAPH '12.

[14]  Min H. Kim,et al.  Simultaneous acquisition of polarimetric SVBRDF and normals , 2018, ACM Trans. Graph..

[15]  Adrián Jarabo,et al.  Bidirectional Rendering of Vector Light Transport , 2018, Comput. Graph. Forum.

[16]  Szymon Rusinkiewicz,et al.  A New Change of Variables for Efficient BRDF Representation , 1998, Rendering Techniques.

[17]  Gordon Wetzstein,et al.  Sub-picosecond photon-efficient 3D imaging using single-photon sensors , 2018, Scientific Reports.

[18]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[19]  Felix Heide,et al.  Single-Shot Monocular RGB-D Imaging Using Uneven Double Refraction , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Alankar Kotwal,et al.  Interferometric transmission probing with coded mutual intensity , 2020, ACM Trans. Graph..

[21]  Frédo Durand,et al.  Experimental analysis of BRDF models , 2005, EGSR '05.

[22]  Matthew O'Toole,et al.  Decomposing global light transport using time of flight imaging , 2012, CVPR.

[23]  R. Azzam,et al.  Photopolarimetric measurement of the Mueller matrix by Fourier analysis of a single detected signal. , 1978, Optics letters.

[24]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[25]  D. Leonard,et al.  Crystallographic orientation of uniaxial calcite and dolomite determined using reflection generalized ellipsometry , 2018, Journal of Applied Physics.

[26]  Diego Gutierrez,et al.  A framework for transient rendering , 2014, ACM Trans. Graph..

[27]  Reinhard Klein,et al.  Solving trigonometric moment problems for fast transient imaging , 2015, ACM Trans. Graph..

[28]  Ashok Veeraraghavan,et al.  Ellipsoidal path connections for time-gated rendering , 2019, ACM Trans. Graph..

[29]  Edoardo Charbon,et al.  Megapixel time-gated SPAD image sensor for 2D and 3D imaging applications , 2019, Optica.

[30]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, SIGGRAPH 2006.

[31]  Mohit Gupta,et al.  On controlling light transport in poor visibility environments , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Achuta Kadambi,et al.  Deep Shape from Polarization , 2020, ECCV.

[33]  Yoav Y. Schechner,et al.  Active Polarization Descattering , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Hans-Peter Seidel,et al.  Polarization and Phase-Shifting for 3D Scanning of Translucent Objects , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Gary A. Atkinson,et al.  Recovery of surface orientation from diffuse polarization , 2006, IEEE Transactions on Image Processing.

[36]  Hans-Peter Seidel,et al.  A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging , 2013, ACM Trans. Graph..

[37]  Wenzel Jakob,et al.  Image-based acquisition and modeling of polarimetric reflectance , 2020, ACM Trans. Graph..

[38]  Matthew O'Toole,et al.  3D Shape and Indirect Appearance by Structured Light Transport , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Gordon Wetzstein,et al.  Confocal non-line-of-sight imaging based on the light-cone transform , 2018, Nature.

[40]  Marcus A. Magnor,et al.  Modeling and Verifying the Polarizing Reflectance of Real-World Metallic Surfaces , 2012, IEEE Computer Graphics and Applications.

[41]  Jannik Boll Nielsen,et al.  Minimal BRDF sampling for two-shot near-field reflectance acquisition , 2016, ACM Trans. Graph..

[42]  Michael J. Cree,et al.  Resolving multiple propagation paths in time of flight range cameras using direct and global separation methods , 2015 .

[43]  Frédo Durand,et al.  Micron-scale light transport decomposition using interferometry , 2015, ACM Trans. Graph..

[44]  Jernej Barbic,et al.  Skin microstructure deformation with displacement map convolution , 2015, ACM Trans. Graph..

[45]  Jannik Boll Nielsen,et al.  On optimal, minimal BRDF sampling for reflectance acquisition , 2015, ACM Trans. Graph..

[46]  Hans-Peter Seidel,et al.  Eurographics Symposium on Rendering 2008 Combining Confocal Imaging and Descattering , 2022 .

[47]  Mukul Sarkar,et al.  Depth Resolution Enhancement in Time-of-Flight Cameras Using Polarization State of the Reflected Light , 2019, IEEE Transactions on Instrumentation and Measurement.

[48]  Aswin C. Sankaranarayanan,et al.  KRISM—Krylov Subspace-based Optical Computing of Hyperspectral Images , 2018, ACM Trans. Graph..

[49]  Matthew O'Toole,et al.  Homogeneous codes for energy-efficient illumination and imaging , 2015, ACM Trans. Graph..

[50]  P. Debevec,et al.  Practical modeling and acquisition of layered facial reflectance , 2008, SIGGRAPH Asia '08.

[51]  Matthew O'Toole,et al.  Temporal frequency probing for 5D transient analysis of global light transport , 2014, ACM Trans. Graph..

[52]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[53]  Kenneth C. Ludema,et al.  The influences of roughness on film thickness measurements by Mueller matrix ellipsometry , 1994 .

[54]  Abhijeet Ghosh,et al.  Deep Polarization Imaging for 3D Shape and SVBRDF Acquisition , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  E. Collett.  Field Guide to Polarization , 2005 .

[56]  Yongdong Zhang,et al.  Adaptive polarization-difference transient imaging for depth estimation in scattering media. , 2018, Optics letters.

[57]  Ramesh Raskar,et al.  Polarized 3D: High-Quality Depth Sensing with Polarization Cues , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[58]  Ramesh Raskar,et al.  A light transport model for mitigating multipath interference in Time-of-flight sensors , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Gianluca Agresti,et al.  Combination of Spatially-Modulated ToF and Structured Light for MPI-Free Depth Estimation , 2018, ECCV Workshops.

[60]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[61]  Pieter Peers,et al.  Circularly polarized spherical illumination reflectometry , 2010, ACM Trans. Graph..

[62]  Qionghai Dai,et al.  Decomposing Global Light Transport Using Time of Flight Imaging , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[64]  Matthew O'Toole,et al.  Primal-dual coding to probe light transport , 2012, ACM Trans. Graph..

[65]  Mikhail Itskov,et al.  Tensor Algebra and Tensor Analysis for Engineers , 2009, Mathematical Engineering.

[66]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.

[67]  Paul E. Debevec,et al.  Multiview face capture using polarized spherical gradient illumination , 2011, ACM Trans. Graph..

[68]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[69]  Nirmalya Ghosh,et al.  Tissue polarimetry: concepts, challenges, applications, and outlook. , 2011, Journal of biomedical optics.

[70]  Gordon Wetzstein,et al.  Non-line-of-sight Imaging with Partial Occluders and Surface Normals , 2017, ACM Trans. Graph..

[71]  Yasushi Yagi,et al.  Descattering of transmissive observation using Parallel High-Frequency Illumination , 2013, IEEE International Conference on Computational Photography (ICCP).

[72]  Yasuhiro Mukaigawa,et al.  Acquiring and characterizing plane-to-ray indirect light transport , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[73]  Diego Gutierrez,et al.  Recent advances in transient imaging: A computer graphics and vision perspective , 2016, Vis. Informatics.