Fast Polygonal Splatting using Directional Kernel Difference

Depth-of-field (DoF) filtering is an important image-processing task for producing blurred images similar to those obtained with a large aperture camera lens. DoF filtering applies an image convolution with a spatially varying kernel and is thus computationally expensive, even on modern computational hardware. In this paper, we introduce an approach for fast and accurate DoF filtering for polygonal kernels, where the value is constant inside the kernel. Our approach is an extension of the existing approach based on discrete differenced kernels. The performance gain here hinges upon the fact that kernels typically become sparse (i.e., mostly zero) when taking the difference. We extended the existing approach to conventional axis-aligned differences to non-axis-aligned differences. The key insight is that taking such differences along the directions of the edges makes polygonal kernels significantly sparser than just taking the difference along the axis-aligned directions, as in existing studies. Compared to a naive image convolution, we achieve an order of magnitude speedup, allowing a real-time application of polygonal kernels even on high-resolution images. CCS Concepts • Computing methodologies → Image processing; Rasterization; Image-based rendering; Massively parallel algorithms;

[1]  Frédo Durand,et al.  Fourier depth of field , 2009, TOGS.

[2]  Sungkil Lee,et al.  Real‐Time Depth‐of‐Field Rendering Using Point Splatting on Per‐Pixel Layers , 2008, Comput. Graph. Forum.

[3]  Paul S. Heckbert,et al.  Filtering by repeated integration , 1986, SIGGRAPH.

[4]  Maitreya Suin,et al.  Depth-Guided Dense Dynamic Filtering Network for Bokeh Effect Rendering , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[5]  BGGAN: Bokeh-Glass Generative Adversarial Network for Rendering Realistic Bokeh , 2020, ECCV Workshops.

[6]  Martin Kraus,et al.  Depth‐of‐Field Rendering by Pyramidal Image Processing , 2007, Comput. Graph. Forum.

[7]  Yair Movshovitz-Attias,et al.  Synthetic depth-of-field with a single-camera mobile phone , 2018, ACM Trans. Graph..

[8]  Michael Potmesil,et al.  A lens and aperture camera model for synthetic image generation , 1981, SIGGRAPH '81.

[9]  Jong-Seok Lee,et al.  Efficient Bokeh Effect Rendering using Generative Adversarial Network , 2020, 2020 IEEE International Conference on Consumer Electronics - Asia (ICCE-Asia).

[10]  Michael O. Polley,et al.  Wavelet Synthesis Net for Disparity Estimation to Synthesize DSLR Calibre Bokeh Effect on Smartphones , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[12]  Przemyslaw Rokita,et al.  Fast generation of depth of field effects in computer graphics , 1993, Comput. Graph..

[13]  Zeev Farbman,et al.  Convolution pyramids , 2011, ACM Trans. Graph..

[14]  Radu Timofte,et al.  Rendering Natural Camera Bokeh Effect with Deep Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[15]  Hans-Peter Seidel,et al.  Deep Shading: Convolutional Neural Networks for Screen Space Shading , 2016, Comput. Graph. Forum.

[16]  Brian A. Barsky,et al.  Depth of field postprocessing for layered scenes using constant-time rectangle spreading , 2009, Graphics Interface.

[17]  PotmesilMichael,et al.  A lens and aperture camera model for synthetic image generation , 1981 .

[18]  H. Seidel,et al.  Real-time lens blur effects and focus control , 2010, ACM Trans. Graph..

[19]  Huchuan Lu,et al.  DeepLens: Shallow Depth Of Field From A Single Image , 2018, ACM Trans. Graph..

[20]  Hans-Peter Seidel,et al.  Laplacian kernel splatting for efficient depth-of-field and motion blur synthesis or reconstruction , 2018, ACM Trans. Graph..

[21]  Anselmo Lastra,et al.  Fast Summed‐Area Table Generation and its Applications , 2005, Comput. Graph. Forum.

[22]  Mark J. Harris,et al.  Parallel Prefix Sum (Scan) with CUDA , 2011 .

[23]  Bokeh Rendering from Defocus Estimation , 2020, ECCV Workshops.

[24]  Douglas Lanman,et al.  Modeling and Synthesis of Aperture Effects in Cameras , 2008, CAe.

[25]  Kentaro Imajo Fast Gaussian filtering algorithm using splines , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[26]  Hans-Peter Seidel,et al.  Depth-of-field rendering with multiview synthesis , 2009, ACM Trans. Graph..

[27]  Guillaume Abadie,et al.  Advances in real-time rendering in games , 2018, ACM SIGGRAPH 2018 Courses.

[28]  Franklin C. Crow,et al.  Summed-area tables for texture mapping , 1984, SIGGRAPH.

[29]  Sungkil Lee,et al.  Real-Time Depth-of-Field Rendering Using Anisotropically Filtered Mipmap Interpolation , 2009, IEEE Transactions on Visualization and Computer Graphics.

[30]  Jonathan T. Barron,et al.  Fast bilateral-space stereo for synthetic defocus , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Sungkil Lee,et al.  Efficient Ray Tracing Through Aspheric Lenses and Imperfect Bokeh Synthesis , 2016, Comput. Graph. Forum.

[32]  Robert L. Cook,et al.  Distributed ray tracing , 1984, SIGGRAPH.