Sample‐Based Manifold Filtering for Interactive Global Illumination and Depth of Field

We present a fast reconstruction filtering method for images generated with Monte Carlo–based rendering techniques. Our approach specializes in reducing global illumination noise in the presence of depth‐of‐field effects at very low sampling rates and interactive frame rates. We employ edge‐aware filtering in the sample space to locally improve outgoing radiance of each sample. The improved samples are then distributed in the image plane using a fast, linear manifold‐based approach supporting very large circles of confusion. We evaluate our filter by applying it to several images containing noise caused by Monte Carlo–simulated global illumination, area light sources and depth of field. We show that our filter can efficiently denoise such images at interactive frame rates on current GPUs and with as few as 4–16 samples per pixel. Our method operates only on the colour and geometric sample information output of the initial rendering process. It does not make any assumptions on the underlying rendering technique and sampling strategy and can therefore be implemented completely as a post‐process filter.

[1]  Lei Xiao,et al.  Compressive Rendering of Multidimensional Scenes , 2010, Video Processing and Computational Video.

[2]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[3]  Pradeep Sen,et al.  Removing the Noise in Monte Carlo Rendering with General Image Denoising Algorithms , 2013, Comput. Graph. Forum.

[4]  F. Durand,et al.  Temporal light field reconstruction for rendering distribution effects , 2011, ACM Trans. Graph..

[5]  R. Ramamoorthi,et al.  Adaptive wavelet rendering , 2009, SIGGRAPH 2009.

[6]  Morgan McGuire,et al.  Filtering approaches for real-time anti-aliasing , 2011, SIGGRAPH '11.

[7]  Manuel Menezes de Oliveira Neto,et al.  Adaptive manifolds for real-time high-dimensional filtering , 2012, ACM Trans. Graph..

[8]  Martin Kraus,et al.  Depth‐of‐Field Rendering by Pyramidal Image Processing , 2007, Comput. Graph. Forum.

[9]  Anselmo Lastra,et al.  Fast Summed‐Area Table Generation and its Applications , 2005, Comput. Graph. Forum.

[10]  Jean-Michel Morel,et al.  Boosting monte carlo rendering by ray histogram fusion , 2014, ACM Trans. Graph..

[11]  Elmar Eisemann,et al.  Interactive Indirect Illumination Using Voxel Cone Tracing , 2011, Comput. Graph. Forum.

[12]  Jun-Hai Yong,et al.  Efficient Depth‐of‐Field Rendering with Adaptive Sampling and Multiscale Reconstruction , 2011, Comput. Graph. Forum.

[13]  Timo Aila,et al.  A local image reconstruction algorithm for stochastic rendering , 2011, SI3D.

[14]  F. Durand,et al.  Flash photography enhancement via intrinsic relighting , 2004, ACM Trans. Graph..

[15]  Fatih Porikli,et al.  Constant time O(1) bilateral filtering , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Frédo Durand,et al.  Axis-aligned filtering for interactive physically-based diffuse indirect lighting , 2013, ACM Trans. Graph..

[17]  Jie Guo,et al.  Point-wise Adaptive Filtering for Fast Monte Carlo Noise Reduction , 2012, PG.

[18]  Soheil Darabi,et al.  On filtering the noise from the random parameters in Monte Carlo rendering , 2012, TOGS.

[19]  Yung-Yu Chuang,et al.  SURE-based optimization for adaptive sampling and reconstruction , 2012, ACM Trans. Graph..

[20]  Narendra Ahuja,et al.  Real-time O(1) bilateral filtering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Robert L. Cook,et al.  Distributed ray tracing , 1984, SIGGRAPH.

[22]  Franklin C. Crow,et al.  Summed-area tables for texture mapping , 1984, SIGGRAPH.

[23]  Matthias Zwicker,et al.  Adaptive rendering with non-local means filtering , 2012, ACM Trans. Graph..

[24]  H. Seidel,et al.  Real-time lens blur effects and focus control , 2010, ACM Trans. Graph..

[25]  Jim X. Chen,et al.  Accurate Depth of Field Simulation in Real Time , 2007, Comput. Graph. Forum.

[26]  Marcus A. Magnor,et al.  Eurographics Symposium on Rendering 2011 Guided Image Filtering for Interactive High-quality Global Illumination , 2022 .

[27]  Don P. Mitchell,et al.  Generating antialiased images at low sampling densities , 1987, SIGGRAPH.

[28]  Ravi Ramamoorthi,et al.  Axis-aligned filtering for interactive sampled soft shadows , 2012, ACM Trans. Graph..

[29]  Karthikeyan Vaidyanathan,et al.  Layered Light Field Reconstruction for Defocus Blur , 2015, TOGS.

[30]  Jaakko Lehtinen,et al.  Reconstructing the indirect light field for global illumination , 2012, ACM Trans. Graph..

[31]  Brian A. Barsky,et al.  An Algorithm for Rendering Generalized Depth of Field Effects Based on Simulated Heat Diffusion , 2007, ICCSA.

[32]  Elmar Eisemann,et al.  Interactive Indirect Illumination Using Voxel Cone Tracing , 2011, Comput. Graph. Forum.

[33]  Manuel Menezes de Oliveira Neto,et al.  Domain transform for edge-aware image and video processing , 2011, ACM Trans. Graph..

[34]  Richard Szeliski,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[35]  Matthias Zwicker,et al.  Multidimensional adaptive sampling and reconstruction for ray tracing , 2008, ACM Trans. Graph..

[36]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[37]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Tomas Akenine-Möller,et al.  Layered Reconstruction for Defocus and Motion Blur , 2014, Comput. Graph. Forum.

[39]  Matthias Zwicker,et al.  Adaptive sampling and reconstruction using greedy error minimization , 2011, ACM Trans. Graph..

[40]  Frédo Durand,et al.  Fourier depth of field , 2009, TOGS.

[41]  Hendrik P. A. Lensch,et al.  Edge-avoiding À-Trous wavelet transform for fast global illumination filtering , 2010, HPG '10.

[42]  Hans-Peter Seidel,et al.  Imperfect shadow maps for efficient computation of indirect illumination , 2008, SIGGRAPH Asia '08.

[43]  Hans-Peter Seidel,et al.  Depth-of-field rendering with multiview synthesis , 2009, ACM Trans. Graph..

[44]  Matthias Zwicker,et al.  Robust Denoising using Feature and Color Information , 2013, Comput. Graph. Forum.

[45]  Wolfgang Engel ShaderX2: Shader Programming Tips and Tricks with DirectX 9.0 , 2003 .

[46]  James T. Kajiya,et al.  The rendering equation , 1986, SIGGRAPH.

[47]  Przemyslaw Rokita,et al.  Generating depth of-field effects in virtual reality applications , 1996, IEEE Computer Graphics and Applications.