Laplacian kernel splatting for efficient depth-of-field and motion blur synthesis or reconstruction

Simulating combinations of depth-of-field and motion blur is an important factor to cinematic quality in synthetic images but can take long to compute. Splatting the point-spread function (PSF) of every pixel is general and provides high quality, but requires prohibitive compute time. We accelerate this in two steps: In a pre-process we optimize for sparse representations of the Laplacian of all possible PSFs that we call spreadlets. At runtime, spreadlets can be splat efficiently to the Laplacian of an image. Integrating this image produces the final result. Our approach scales faithfully to strong motion and large out-of-focus areas and compares favorably in speed and quality with off-line and interactive approaches. It is applicable to both synthesizing from pinhole as well as reconstructing from stochastic images, with or without layering.

[1]  Michael D. McCool,et al.  Anisotropic diffusion for Monte Carlo noise reduction , 1999, TOGS.

[2]  Shree K. Nayar,et al.  Practical Rendering of Multiple Scattering Effects in Participating Media , 2004, Rendering Techniques.

[3]  Hans-Peter Seidel,et al.  Depth-of-field rendering with multiview synthesis , 2009, ACM Trans. Graph..

[4]  Michael Goesele,et al.  Image-based rendering in the gradient domain , 2013, ACM Trans. Graph..

[5]  Robert L. Cook,et al.  The Reyes image rendering architecture , 1987, SIGGRAPH.

[6]  Karthikeyan Vaidyanathan,et al.  Layered Light Field Reconstruction for Defocus Blur , 2015, TOGS.

[7]  Martin Kraus,et al.  Depth‐of‐Field Rendering by Pyramidal Image Processing , 2007, Comput. Graph. Forum.

[8]  Hans-Peter Seidel,et al.  Real-time lens blur effects and focus control , 2010, ACM Trans. Graph..

[9]  Hans-Peter Seidel,et al.  Depth-of-field rendering with multiview synthesis , 2009, SIGGRAPH 2009.

[10]  Michael F. Cohen,et al.  GradientShop: A gradient-domain optimization framework for image and video filtering , 2010, TOGS.

[11]  Janne Kontkanen,et al.  Irradiance Filtering for Monte Carlo Ray Tracing , 2006 .

[12]  Hans-Peter Seidel,et al.  Deep Shading: Convolutional Neural Networks for Screen Space Shading , 2016, Comput. Graph. Forum.

[13]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[14]  Jaakko Lehtinen,et al.  Temporal light field reconstruction for rendering distribution effects Citation , 2011 .

[15]  Hans-Peter Seidel,et al.  Image Compression with Anisotropic Diffusion , 2008, Journal of Mathematical Imaging and Vision.

[16]  Yann LeCun,et al.  Boxlets: A Fast Convolution Algorithm for Signal Processing and Neural Networks , 1998, NIPS.

[17]  Tomas Akenine-Möller,et al.  Stochastic rasterization using time-continuous triangles , 2007, GH '07.

[18]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[19]  Soheil Darabi,et al.  On filtering the noise from the random parameters in Monte Carlo rendering , 2012, TOGS.

[20]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[21]  Rui Wang,et al.  Real‐time Depth of Field Rendering via Dynamic Light Field Generation and Filtering , 2010, Comput. Graph. Forum.

[22]  Michael Potmesil,et al.  A lens and aperture camera model for synthetic image generation , 1981, SIGGRAPH '81.

[23]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[24]  Robert L. Cook,et al.  Distributed ray tracing , 1984, SIGGRAPH.

[25]  Frédo Durand,et al.  Frequency analysis and sheared reconstruction for rendering motion blur , 2009, ACM Trans. Graph..

[26]  F. Durand,et al.  Temporal light field reconstruction for rendering distribution effects , 2011, ACM Trans. Graph..

[27]  Pat Hanrahan,et al.  Data-parallel rasterization of micropolygons with defocus and motion blur , 2009, High Performance Graphics.

[28]  Franklin C. Crow,et al.  Summed-area tables for texture mapping , 1984, SIGGRAPH.

[29]  Stephen Lin,et al.  Diffusion curve textures for resolution independent texture mapping , 2012, ACM Trans. Graph..

[30]  Hans-Peter Seidel,et al.  Deep screen space , 2014, I3D.

[31]  Pradeep Sen,et al.  Removing the Noise in Monte Carlo Rendering with General Image Denoising Algorithms , 2013, Comput. Graph. Forum.

[32]  Hans-Peter Seidel,et al.  Minimal Warping: Planning Incremental Novel‐view Synthesis , 2017, Comput. Graph. Forum.

[33]  Paul S. Heckbert,et al.  Filtering by repeated integration , 1986, SIGGRAPH.

[34]  Frédo Durand,et al.  5D Covariance tracing for efficient defocus and motion blur , 2013, TOGS.

[35]  Hans-Peter Seidel,et al.  Physically-based real-time lens flare rendering , 2011, ACM Trans. Graph..

[36]  Tomas Akenine-Möller,et al.  Layered Reconstruction for Defocus and Motion Blur , 2014, Comput. Graph. Forum.

[37]  Morgan McGuire,et al.  A reconstruction filter for plausible motion blur , 2012, I3D '12.

[38]  Pascal Barla,et al.  Diffusion curves: a vector representation for smooth-shaded images , 2008, ACM Trans. Graph..

[39]  Frédo Durand,et al.  Fourier depth of field , 2009, TOGS.

[40]  Rob Hummel,et al.  American Cinematographer Manual , 2002 .

[41]  Kurt Akeley,et al.  The accumulation buffer: hardware support for high-quality rendering , 1990, SIGGRAPH.

[42]  Zeev Farbman,et al.  Convolution pyramids , 2011, ACM Trans. Graph..

[43]  D. Salesin,et al.  Diffusion curves: a vector representation for smooth-shaded images , 2013, CACM.

[44]  Karen O. Egiazarian,et al.  Image denoising with block-matching and 3D filtering , 2006, Electronic Imaging.

[45]  Jaakko Lehtinen,et al.  Gradient-domain metropolis light transport , 2013, ACM Trans. Graph..

[46]  Eric P. Lafortune,et al.  Monte Carlo light tracing with direct computation of pixel intensities , 1993 .

[47]  Sungkil Lee,et al.  Real‐Time Depth‐of‐Field Rendering Using Point Splatting on Per‐Pixel Layers , 2008, Comput. Graph. Forum.

[48]  Michael F. Cohen,et al.  Fourier Analysis of the 2D Screened Poisson Equation for Gradient Domain Problems , 2008, ECCV.

[49]  Eugene Fiume,et al.  Constant-time filtering with space-variant kernels , 1988, SIGGRAPH.

[50]  Pat Hanrahan,et al.  A realistic camera model for computer graphics , 1995, SIGGRAPH.