A Survey on Gradient‐Domain Rendering

Monte Carlo methods for physically‐based light transport simulation are broadly adopted in the feature film production, animation and visual effects industries. These methods, however, often result in noisy images and have slow convergence. As such, improving the convergence of Monte Carlo rendering remains an important open problem. Gradient‐domain light transport is a recent family of techniques that can accelerate Monte Carlo rendering by up to an order of magnitude, leveraging a gradient‐based estimation and a reformulation of the rendering problem as an image reconstruction. This state of the art report comprehensively frames the fundamentals of gradient‐domain rendering, as well as the pragmatic details behind practical gradient‐domain uniand bidirectional path tracing and photon density estimation algorithms. Moreover, we discuss the various image reconstruction schemes that are crucial to accurate and stable gradient‐domain rendering. Finally, we benchmark various gradient‐domain techniques against the state‐of‐the‐art in denoising methods before discussing open problems.

[1]  Matthias Zwicker,et al.  The Beam Radiance Estimate for Volumetric Photon Mapping , 2008, SIGGRAPH '08.

[2]  Csaba Kelemen,et al.  Simple and Robust Mutation Strategy for Metropolis Light Transport Algorithm , 2001 .

[3]  Yves D. Willems,et al.  Bi-directional path tracing , 1993 .

[4]  Johannes Hanika,et al.  Monte Carlo Methods for Volumetric Light Transport Simulation , 2018, Comput. Graph. Forum.

[5]  Anton Kaplanyan,et al.  The natural-constraint representation of the path space for efficient light transport simulation , 2014, ACM Trans. Graph..

[6]  Min Chen,et al.  Theory and application of specular path perturbation , 2000, TOGS.

[7]  Pravin Bhat,et al.  Gradient -domain techniques for image & video processing , 2009 .

[8]  László Szirmay-Kalos,et al.  Gradient-Domain PET Reconstruction , 2017, 2017 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC).

[9]  H. Jensen,et al.  Stochastic progressive photon mapping , 2009, ACM Trans. Graph..

[10]  Jaakko Lehtinen,et al.  Aether: an embedded domain specific sampling language for Monte Carlo rendering , 2017, ACM Trans. Graph..

[11]  H. Jensen,et al.  A progressive error estimation framework for photon density estimation , 2010, SIGGRAPH 2010.

[12]  Matthias Zwicker,et al.  Robust Denoising using Feature and Color Information , 2013, Comput. Graph. Forum.

[13]  Matthias Zwicker,et al.  Radiance caching for participating media , 2007, SIGGRAPH '07.

[14]  Jacopo Pantaleoni,et al.  A path space extension for robust light transport simulation , 2012, ACM Trans. Graph..

[15]  Bochang Moon,et al.  Feature Generation for Adaptive Gradient‐Domain Path Tracing , 2018, Comput. Graph. Forum.

[16]  Homan Igehy,et al.  Tracing ray differentials , 1999, SIGGRAPH.

[17]  Hans-Peter Seidel,et al.  Laplacian kernel splatting for efficient depth-of-field and motion blur synthesis or reconstruction , 2018, ACM Trans. Graph..

[18]  Jaakko Lehtinen,et al.  Gradient-Domain Bidirectional Path Tracing , 2015, EGSR.

[19]  Jaakko Lehtinen,et al.  Improved sampling for gradient-domain metropolis light transport , 2014, ACM Trans. Graph..

[20]  Leonidas J. Guibas,et al.  Optimally combining sampling techniques for Monte Carlo rendering , 1995, SIGGRAPH.

[21]  Henrik Wann Jensen,et al.  Importance Driven Path Tracing using the Photon Map , 1995, Rendering Techniques.

[22]  Derek Nowrouzezahrai,et al.  Unifying points, beams, and paths in volumetric light transport simulation , 2014, ACM Trans. Graph..

[23]  Rama Chellappa,et al.  An algebraic approach to surface reconstruction from gradient fields , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[24]  智也 山口 “Image-space control variates for rendering”の実装報告 , 2017 .

[25]  Csaba Kelemen,et al.  Dependent Tests Driven Filtering in Monte-Carlo Global Illumination , 2002, Eurographics.

[26]  James T. Kajiya,et al.  The rendering equation , 1986, SIGGRAPH.

[27]  Derek Nowrouzezahrai,et al.  A comprehensive theory of volumetric radiance estimation using photon points and beams , 2011, TOGS.

[28]  Carsten Dachsbacher,et al.  Selective guided sampling with complete light transport paths , 2018, ACM Trans. Graph..

[29]  Anton Kaplanyan,et al.  Improved Half Vector Space Light Transport , 2015, Comput. Graph. Forum.

[30]  Kadi Bouatouch,et al.  Radiance caching for efficient global illumination computation , 2005 .

[31]  Mark Meyer,et al.  Kernel-predicting convolutional networks for denoising Monte Carlo renderings , 2017, ACM Trans. Graph..

[32]  Steve Marschner,et al.  Manifold exploration , 2012, ACM Trans. Graph..

[33]  Derek Nowrouzezahrai,et al.  Gradient-Domain Vertex Connection and Merging , 2017, EGSR.

[34]  Jaakko Lehtinen,et al.  Gradient-domain path tracing , 2015, ACM Trans. Graph..

[35]  Michael F. Cohen,et al.  Fourier Analysis of the 2D Screened Poisson Equation for Gradient Domain Problems , 2008, ECCV.

[36]  Wojciech Jarosz,et al.  Beyond points and beams , 2017, ACM Trans. Graph..

[37]  Derek Nowrouzezahrai,et al.  Gradient-domain volumetric photon density estimation , 2018, ACM Trans. Graph..

[38]  Pablo Bauszat,et al.  Gradient-domain path reusing , 2017, ACM Trans. Graph..

[39]  Jaakko Lehtinen,et al.  Temporal gradient-domain path tracing , 2016, ACM Trans. Graph..

[40]  Mark Meyer,et al.  Denoising with kernel prediction and asymmetric loss functions , 2018, ACM Trans. Graph..

[41]  Kenny Mitchell,et al.  Nonlinearly Weighted First‐order Regression for Denoising Monte Carlo Renderings , 2016, Comput. Graph. Forum.

[42]  Timo Aila,et al.  Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder , 2017, ACM Trans. Graph..

[43]  Matthias Zwicker,et al.  Regularizing Image Reconstruction for Gradient‐Domain Rendering with Feature Patches , 2016, Comput. Graph. Forum.

[44]  Tamy Boubekeur,et al.  Bayesian Collaborative Denoising for Monte Carlo Rendering , 2017, Comput. Graph. Forum.

[45]  Jaakko Lehtinen,et al.  Differentiable Monte Carlo ray tracing through edge sampling , 2018, ACM Trans. Graph..

[46]  Toshiya Hachisuka,et al.  Robust light transport simulation via metropolised bidirectional estimators , 2016, ACM Trans. Graph..

[47]  Jacopo Pantaleoni,et al.  Matrix Bidirectional Path Tracing , 2018, EGSR.

[48]  Jaakko Lehtinen,et al.  Gradient-domain metropolis light transport , 2013, ACM Trans. Graph..

[49]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[50]  Mateu Sbert,et al.  Accelerating Path Tracing by Re-Using Paths , 2002, Rendering Techniques.

[51]  Tobias Ritschel,et al.  On-line learning of parametric mixture models for light transport simulation , 2014, ACM Trans. Graph..

[52]  J. Shewchuk An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[53]  Jaakko Lehtinen,et al.  Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering , 2015, Comput. Graph. Forum.

[54]  Frédo Durand,et al.  Eurographics Symposium on Rendering 2015 Probabilistic Connections for Bidirectional Path Tracing Bidirectional Path Tracing Probabilistic Connections for Bidirectional Path Tracing , 2022 .

[55]  Pablo Bauszat,et al.  Spectral Gradient Sampling for Path Tracing , 2018, Comput. Graph. Forum.

[56]  Michael Goesele,et al.  Image-based rendering in the gradient domain , 2013, ACM Trans. Graph..

[57]  David Thomas,et al.  The Art in Computer Programming , 2001 .

[58]  Nancy Argüelles,et al.  Author ' s , 2008 .

[59]  H. Jensen Realistic Image Synthesis Using Photon Mapping , 2001 .

[60]  Derek Nowrouzezahrai,et al.  Gradient‐Domain Photon Density Estimation , 2017, Comput. Graph. Forum.

[61]  Yves D. Willems,et al.  Path Differentials and Applications , 2001, Rendering Techniques.

[62]  Paul S. Heckbert,et al.  Irradiance gradients , 2008, SIGGRAPH '08.

[63]  Leonidas J. Guibas,et al.  Bidirectional Estimators for Light Transport , 1995 .

[64]  Markus H. Gross,et al.  Practical Path Guiding for Efficient Light‐Transport Simulation , 2017, Comput. Graph. Forum.

[65]  Philipp Slusallek,et al.  Light transport simulation with vertex connection and merging , 2012, ACM Trans. Graph..

[66]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[67]  Pascal Gautron Temporal radiance caching , 2008, SIGGRAPH '08.