Polarimetric Spatio-Temporal Light Transport Probing

Fig. 1. We propose a computational light transport probingmethod that decomposes transport into full polarization, spatial and temporal dimensions.Wemodel this multi-dimensional light transport as a tensor and analyze low-rank structure in the polarization domain which is exploited by our polarimetric probing method. We instantiate our approach with two imaging systems for spatio-polarimetric and coaxial temporal-polarimetric capture. (a)&(d) Conventional intensity imagers integrate incident light intensity over space and time independently of the polarization states of light, losing geometric and material information encoded in the polarimetric transport. Capturing polarization-resolved spatial transport components of (b) epipolar and (c) non-epipolar dimensions enable fine-grained decomposition of light transport. Combining temporal and polarimetric dimensions, we separate (e) geometry-dependent reflections and (f) direct/indirect reflections that cannot be resolved in the temporal-only measurements.

[1]  Kristin J. Dana,et al.  Shape from Sky: Polarimetric Normal Recovery Under The Sky , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Yiming Lin,et al.  Deep Polarization Imaging for 3D Shape and SVBRDF Acquisition , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Wenzel Jakob,et al.  Image-based acquisition and modeling of polarimetric reflectance , 2020, ACM Trans. Graph..

[4]  Alankar Kotwal,et al.  Interferometric transmission probing with coded mutual intensity , 2020, ACM Trans. Graph..

[5]  Felix Heide,et al.  Single-Shot Monocular RGB-D Imaging Using Uneven Double Refraction , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Achuta Kadambi,et al.  Deep Shape from Polarization , 2020, ECCV.

[7]  Edoardo Charbon,et al.  Megapixel time-gated SPAD image sensor for 2D and 3D imaging applications , 2019, Optica.

[8]  Yasuhiro Mukaigawa,et al.  Enhancing Passive Non-Line-of-Sight Imaging Using Polarization Cues , 2019, ArXiv.

[9]  Ashok Veeraraghavan,et al.  Ellipsoidal path connections for time-gated rendering , 2019, ACM Trans. Graph..

[10]  Mukul Sarkar,et al.  Depth Resolution Enhancement in Time-of-Flight Cameras Using Polarization State of the Reflected Light , 2019, IEEE Transactions on Instrumentation and Measurement.

[11]  D. Leonard,et al.  Crystallographic orientation of uniaxial calcite and dolomite determined using reflection generalized ellipsometry , 2018, Journal of Applied Physics.

[12]  Min H. Kim,et al.  Simultaneous acquisition of polarimetric SVBRDF and normals , 2018, ACM Trans. Graph..

[13]  Ravi Ramamoorthi,et al.  Connecting measured BRDFs to analytic BRDFs by data-driven diffuse-specular separation , 2018, ACM Trans. Graph..

[14]  Gianluca Agresti,et al.  Combination of Spatially-Modulated ToF and Structured Light for MPI-Free Depth Estimation , 2018, ECCV Workshops.

[15]  Adrián Jarabo,et al.  Bidirectional Rendering of Vector Light Transport , 2018, Comput. Graph. Forum.

[16]  Gordon Wetzstein,et al.  Sub-picosecond photon-efficient 3D imaging using single-photon sensors , 2018, Scientific Reports.

[17]  Yasuhiro Mukaigawa,et al.  Acquiring and characterizing plane-to-ray indirect light transport , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[18]  Ramesh Raskar,et al.  Dynamic heterodyne interferometry , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[19]  Yongdong Zhang,et al.  Adaptive polarization-difference transient imaging for depth estimation in scattering media. , 2018, Optics letters.

[20]  Gordon Wetzstein,et al.  Confocal non-line-of-sight imaging based on the light-cone transform , 2018, Nature.

[21]  Aswin C. Sankaranarayanan,et al.  KRISM—Krylov Subspace-based Optical Computing of Hyperspectral Images , 2018, ACM Trans. Graph..

[22]  Gordon Wetzstein,et al.  Non-line-of-sight Imaging with Partial Occluders and Surface Normals , 2017, ACM Trans. Graph..

[23]  Abhijeet Ghosh,et al.  Polarization imaging reflectometry in the wild , 2017, ACM Trans. Graph..

[24]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[25]  Edwin R. Hancock,et al.  Linear Differential Constraints for Photo-Polarimetric Height Estimation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Gordon Wetzstein,et al.  Reconstructing Transient Images from Single-Photon Sensors , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  A. Petrova-Mayor,et al.  Analysis and manipulation of the induced changes in the state of polarization by mirror scanners. , 2017, Applied optics.

[28]  Min H. Kim,et al.  Birefractive stereo imaging for single-shot depth acquisition , 2016, ACM Trans. Graph..

[29]  Jannik Boll Nielsen,et al.  Minimal BRDF sampling for two-shot near-field reflectance acquisition , 2016, ACM Trans. Graph..

[30]  Diego Gutierrez,et al.  Recent advances in transient imaging: A computer graphics and vision perspective , 2016, Vis. Informatics.

[31]  Yongdong Zhang,et al.  Scattering robust 3D reconstruction via polarized transient imaging. , 2016, Optics letters.

[32]  R. Azzam,et al.  Stokes-vector and Mueller-matrix polarimetry [Invited]. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[33]  Matthew O'Toole,et al.  3D Shape and Indirect Appearance by Structured Light Transport , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Yu-Lung Lo,et al.  Compensation of non-ideal beam splitter polarization distortion effect in Michelson interferometer , 2016 .

[35]  Ramesh Raskar,et al.  Polarized 3D: High-Quality Depth Sensing with Polarization Cues , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[36]  Michael J. Cree,et al.  Resolving multiple propagation paths in time of flight range cameras using direct and global separation methods , 2015 .

[37]  Jannik Boll Nielsen,et al.  On optimal, minimal BRDF sampling for reflectance acquisition , 2015, ACM Trans. Graph..

[38]  Reinhard Klein,et al.  Solving trigonometric moment problems for fast transient imaging , 2015, ACM Trans. Graph..

[39]  Frédo Durand,et al.  Micron-scale light transport decomposition using interferometry , 2015, ACM Trans. Graph..

[40]  Matthew O'Toole,et al.  Homogeneous codes for energy-efficient illumination and imaging , 2015, ACM Trans. Graph..

[41]  Jernej Barbic,et al.  Skin microstructure deformation with displacement map convolution , 2015, ACM Trans. Graph..

[42]  Ramesh Raskar,et al.  A light transport model for mitigating multipath interference in Time-of-flight sensors , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Diego Gutierrez,et al.  A framework for transient rendering , 2014, ACM Trans. Graph..

[44]  Matthew O'Toole,et al.  Temporal frequency probing for 5D transient analysis of global light transport , 2014, ACM Trans. Graph..

[45]  Wolfgang Heidrich,et al.  Low-budget transient imaging using photonic mixer devices , 2013, ACM Trans. Graph..

[46]  Hans-Peter Seidel,et al.  A reconfigurable camera add-on for high dynamic range, multispectral, polarization, and light-field imaging , 2013, ACM Trans. Graph..

[47]  Yasushi Yagi,et al.  Descattering of transmissive observation using Parallel High-Frequency Illumination , 2013, IEEE International Conference on Computational Photography (ICCP).

[48]  Matthew O'Toole,et al.  Primal-dual coding to probe light transport , 2012, ACM Trans. Graph..

[49]  Qionghai Dai,et al.  Decomposing Global Light Transport Using Time of Flight Imaging , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Marcus A. Magnor,et al.  Modeling and Verifying the Polarizing Reflectance of Real-World Metallic Surfaces , 2012, IEEE Computer Graphics and Applications.

[51]  R. Raskar,et al.  Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging , 2012, Nature Communications.

[52]  Gordon Wetzstein,et al.  Computational Plenoptic Imaging , 2011, SIGGRAPH '12.

[53]  Paul E. Debevec,et al.  Multiview face capture using polarized spherical gradient illumination , 2011, ACM Trans. Graph..

[54]  Nirmalya Ghosh,et al.  Tissue polarimetry: concepts, challenges, applications, and outlook. , 2011, Journal of biomedical optics.

[55]  Ashok Veeraraghavan,et al.  Structured light 3D scanning in the presence of global illumination , 2011, CVPR 2011.

[56]  H. Arwin.  Application of ellipsometry techniques to biological materials , 2011 .

[57]  Matthew O'Toole,et al.  Optical computing for fast light transport analysis , 2010, ACM Trans. Graph..

[58]  Pieter Peers,et al.  Circularly polarized spherical illumination reflectometry , 2010, ACM Trans. Graph..

[59]  Mikhail Itskov,et al.  Tensor Algebra and Tensor Analysis for Engineers , 2009, Mathematical Engineering.

[60]  Yoav Y. Schechner,et al.  Active Polarization Descattering , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  P. Debevec,et al.  Practical modeling and acquisition of layered facial reflectance , 2008, SIGGRAPH Asia '08.

[62]  Mohit Gupta,et al.  On controlling light transport in poor visibility environments , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, ACM Trans. Graph..

[64]  Hans-Peter Seidel,et al.  Polarization and Phase-Shifting for 3D Scanning of Translucent Objects , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, ACM Trans. Graph..

[66]  Gary A. Atkinson,et al.  Recovery of surface orientation from diffuse polarization , 2006, IEEE Transactions on Image Processing.

[67]  E. Collett.  Field Guide to Polarization , 2005 .

[68]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[69]  Frédo Durand,et al.  Experimental analysis of BRDF models , 2005, EGSR '05.

[70]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[71]  Szymon Rusinkiewicz,et al.  A New Change of Variables for Efficient BRDF Representation , 1998, Rendering Techniques.

[72]  R. Chipman,et al.  Interpretation of Mueller matrices based on polar decomposition , 1996 .

[73]  Kenneth C. Ludema,et al.  The influences of roughness on film thickness measurements by Mueller matrix ellipsometry , 1994 .

[74]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[75]  R. Azzam,et al.  Photopolarimetric measurement of the Mueller matrix by Fourier analysis of a single detected signal. , 1978, Optics letters.

[76]  S. Nayar,et al.  Phasor Imaging: A Generalization of Correlation-Based Time-of-Flight Imaging , 2014 .

[77]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[78]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .

[79]  Hans-Peter Seidel,et al.  Eurographics Symposium on Rendering 2008 Combining Confocal Imaging and Descattering , 2022 .