Removing the Noise in Monte Carlo Rendering with General Image Denoising Algorithms

Monte Carlo rendering systems can produce important visual effects such as depth of field, motion blur, and area lighting, but the rendered images suffer from objectionable noise at low sampling rates. Although years of research in image processing has produced powerful denoising algorithms, most of them assume that the noise is spatially‐invariant over the entire image and cannot be directly applied to denoise Monte Carlo rendering. In this paper, we propose a new approach that enables the use of any spatially‐invariant image denoising technique to remove the noise in Monte Carlo renderings. Our key insight is to use a noise estimation metric to locally identify the amount of noise in different parts of the image, coupled with a multilevel algorithm that denoises the image in a spatially‐varying manner using a standard denoising technique. We also propose a new way to perform adaptive sampling that uses the noise estimation metric to identify the noisy regions in which to place more samples. We show that our framework runs in a few seconds with modern denoising algorithms and produces results that outperform state‐of‐the‐art techniques in Monte Carlo rendering.

[1]  Turner Whitted,et al.  An improved illumination model for shaded display , 1979, CACM.

[2]  James T. Kajiya,et al.  The rendering equation , 1986, SIGGRAPH.

[3]  Robert L. Cook,et al.  Stochastic sampling in computer graphics , 1988, TOGS.

[4]  Don P. Mitchell,et al.  Generating antialiased images at low sampling densities , 1987, SIGGRAPH.

[5]  R. Redner,et al.  A note on the use of nonlinear filtering in computer graphics , 1990, IEEE Computer Graphics and Applications.

[6]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[7]  Holly E. Rushmeier,et al.  Energy preserving non-linear filters , 1994, SIGGRAPH.

[8]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[9]  Gary W. Meyer,et al.  A perceptually based adaptive sampling algorithm , 1998, SIGGRAPH.

[10]  Robert L. Cook,et al.  Distributed ray tracing , 1998 .

[11]  Baining Guo,et al.  Progressive radiance evaluation using directional coherence maps , 1998, SIGGRAPH.

[12]  Michael D. McCool,et al.  Anisotropic diffusion for Monte Carlo noise reduction , 1999, TOGS.

[13]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[14]  Donald P. Greenberg,et al.  Combining edges and points for interactive high-quality rendering , 2003, ACM Trans. Graph..

[15]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[16]  Mateu Sbert,et al.  Refinement Criteria Based on f-Divergences , 2003, Rendering Techniques.

[17]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[18]  H. Jensen,et al.  Wavelet importance sampling: efficiently evaluating products of complex functions , 2005, SIGGRAPH 2005.

[19]  Ruifeng Xu,et al.  A novel Monte Carlo noise reduction operator , 2005, IEEE Computer Graphics and Applications.

[20]  Karen O. Egiazarian,et al.  Image denoising with block-matching and 3D filtering , 2006, Electronic Imaging.

[21]  Mark Meyer,et al.  Statistical acceleration for animated global illumination , 2006, SIGGRAPH 2006.

[22]  Karen O. Egiazarian,et al.  Video denoising by sparse 3D transform-domain collaborative filtering , 2007, 2007 15th European Signal Processing Conference.

[23]  W.R. Mark,et al.  Large ray packets for real-time Whitted ray tracing , 2008, 2008 IEEE Symposium on Interactive Ray Tracing.

[24]  Matthias Zwicker,et al.  Multidimensional adaptive sampling and reconstruction for ray tracing , 2008, ACM Trans. Graph..

[25]  Karen O. Egiazarian,et al.  Image restoration by sparse 3D transform-domain collaborative filtering , 2008, Electronic Imaging.

[26]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[27]  Frédo Durand,et al.  Fourier depth of field , 2009, TOGS.

[28]  Frédo Durand,et al.  Frequency analysis and sheared reconstruction for rendering motion blur , 2009, SIGGRAPH 2009.

[29]  R. Ramamoorthi,et al.  Adaptive wavelet rendering , 2009, SIGGRAPH 2009.

[30]  Hendrik P. A. Lensch,et al.  Edge-avoiding À-Trous wavelet transform for fast global illumination filtering , 2010, HPG '10.

[31]  Soheil Darabi,et al.  Compressive Rendering: A Rendering Application of Compressed Sensing , 2011, IEEE Transactions on Visualization and Computer Graphics.

[32]  Frédo Durand,et al.  Practical filtering for efficient ray-traced directional occlusion , 2011, ACM Trans. Graph..

[33]  Jaakko Lehtinen,et al.  Temporal light field reconstruction for rendering distribution effects Citation , 2011 .

[34]  Marcus A. Magnor,et al.  Eurographics Symposium on Rendering 2011 Guided Image Filtering for Interactive High-quality Global Illumination , 2022 .

[35]  ZwickerMatthias,et al.  Adaptive sampling and reconstruction using greedy error minimization , 2011 .

[36]  DurandFrédo,et al.  Practical filtering for efficient ray-traced directional occlusion , 2011 .

[37]  Jun-Hai Yong,et al.  Efficient Depth‐of‐Field Rendering with Adaptive Sampling and Multiscale Reconstruction , 2011, Comput. Graph. Forum.

[38]  Matthias Zwicker,et al.  Adaptive sampling and reconstruction using greedy error minimization , 2011, ACM Trans. Graph..

[39]  Frédo Durand,et al.  Frequency analysis and sheared filtering for shadow light fields of complex occluders , 2011, TOGS.

[40]  Jaakko Lehtinen,et al.  Reconstructing the indirect light field for global illumination , 2012, ACM Trans. Graph..

[41]  Soheil Darabi,et al.  On filtering the noise from the random parameters in Monte Carlo rendering , 2012, TOGS.