Learning Neural Transmittance for Efficient Rendering of Reflectance Fields

Recently neural volumetric representations such as neural reflectance fields have been widely applied to faithfully reproduce the appearance of real-world objects and scenes under novel viewpoints and lighting conditions. However, it remains challenging and time-consuming to render such representations under complex lighting such as environment maps, which requires individual ray marching towards each single light to calculate the transmittance at every sampled point. In this paper, we propose a novel method based on precomputed Neural Transmittance Functions to accelerate the rendering of neural reflectance fields. Our neural transmittance functions enable us to efficiently query the transmittance at an arbitrary point in space along an arbitrary ray without tedious ray marching, which effectively reduces the time-complexity of the rendering. We propose a novel formulation for the neural transmittance function, and train it jointly with the neural reflectance fields on images captured under collocated camera and light, while enforcing monotonicity. Results on real and synthetic scenes demonstrate almost two order of magnitude speedup for renderings under environment maps with minimal accuracy loss.

[1]  Yannick Hold-Geoffroy,et al.  Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images , 2020, ECCV.

[2]  Tom Lokovic,et al.  Deep shadow maps , 2000, SIGGRAPH.

[3]  Donald S. Fussell,et al.  Uniformly Sampled Light Fields , 1998, Rendering Techniques.

[4]  Pratul P. Srinivasan,et al.  NeRF , 2020, ECCV.

[5]  Yannick Hold-Geoffroy,et al.  Neural Reflectance Fields for Appearance Acquisition , 2020, ArXiv.

[6]  Yiyi Liao,et al.  Deep Marching Cubes: Learning Explicit Surface Representations , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Markus H. Gross,et al.  Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks , 2017, ArXiv.

[9]  Hans-Peter Seidel,et al.  Interactive Global Illumination Using Implicit Visibility , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[10]  Kalyan Sunkavalli,et al.  Deep 3D Capture: Geometry and Reflectance From Sparse Multi-View Images , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Jitendra Malik,et al.  Learning a Multi-View Stereo Machine , 2017, NIPS.