Vectorization for Fast, Analytic, and Differentiable Visibility

In Computer Graphics, the two main approaches to rendering and visibility involve ray tracing and rasterization. However, a limitation of both approaches is that they essentially use point sampling. This is the source of noise and aliasing, and also leads to significant difficulties for differentiable rendering. In this work, we present a new rendering method, which we call vectorization, that computes 2D point-to-region integrals analytically, thus eliminating point sampling in the 2D integration domain such as for pixel footprints and area lights. Our vectorization revisits the concept of beam tracing, and handles the hidden surface removal problem robustly and accurately. That is, for each intersecting triangle inserted into the viewport of a beam in an arbitrary order, we are able to maintain all the visible regions formed by intersections and occlusions, thanks to our Visibility Bounding Volume Hierarchy structure. As a result, our vectorization produces perfectly anti-aliased visibility, accurate and analytic shading and shadows, and most important, fast and noise-free gradients with Automatic Differentiation or Finite Differences that directly enables differentiable rendering without any changes to our rendering pipeline. Our results are inherently high-quality and noise-free, and our gradients are one to two orders of magnitude faster than those computed with existing differentiable rendering methods.

[1]  Pascal Barla,et al.  Diffusion curves: a vector representation for smooth-shaded images , 2008, SIGGRAPH 2008.

[2]  Lei Yang,et al.  A Survey of Temporal Antialiasing Techniques , 2020, Comput. Graph. Forum.

[3]  Seth J. Teller,et al.  Real-time occlusion culling for models with large occluders , 1997, SI3D.

[4]  Jonathan Richard Shewchuk,et al.  Lecture Notes on Geometric Robustness , 2013 .

[5]  Jaakko Lehtinen,et al.  Modular primitives for high-performance differentiable rendering , 2020, ACM Trans. Graph..

[6]  Frédo Durand,et al.  Unbiased warped-area sampling for differentiable rendering , 2020, ACM Trans. Graph..

[7]  Derrick G. Kourie,et al.  Light Beam Tracing for Multi-Bounce Specular and Glossy Transport Paths , 2014, SAICSIT '14.

[8]  Ravi Ramamoorthi,et al.  A differential theory of radiative transfer , 2019, ACM Trans. Graph..

[9]  C. Peng SCALABLE VECTOR GRAPHICS (SVG) , 2000 .

[10]  Dinesh Manocha,et al.  Accelerated occlusion culling using shadow frusta , 1997, SCG '97.

[11]  Pat Hanrahan,et al.  Beam tracing polygonal objects , 1984, SIGGRAPH.

[12]  Michael McKenna Worst-case optimal hidden-surface removal , 1987, TOGS.

[13]  Mark H. Overmars,et al.  On a Class of O(n2) Problems in Computational Geometry , 1995, Comput. Geom..

[14]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[15]  Dinesh Manocha,et al.  FastV : From-point Visibility Culling on Complex Models , 2009 .

[16]  Daniel R. Baum,et al.  Improving radiosity solutions through the use of analytically determined form-factors , 1989, SIGGRAPH.

[17]  Frédo Durand,et al.  3d visibility: analytical study and applications , 1999 .

[18]  Edwin E. Catmull,et al.  A hidden-surface algorithm with anti-aliasing , 1978, SIGGRAPH.

[19]  Edwin H. Blake,et al.  Hardware Accelerated Visibility Preprocessing using Adaptive Sampling , 2004, Rendering Techniques.

[20]  Michael I. Jordan,et al.  A Linearly-Convergent Stochastic L-BFGS Algorithm , 2015, AISTATS.

[21]  Eric Heitz,et al.  Real-time polygonal-light shading with linearly transformed cosines , 2016, ACM Trans. Graph..

[22]  Tzu-Mao Li,et al.  Differentiable vector graphics rasterization for editing and learning , 2020, ACM Trans. Graph..

[23]  Frédo Durand,et al.  Inverse Diffusion Curves Using Shape Optimization , 2016, IEEE Transactions on Visualization and Computer Graphics.

[24]  Ravi Ramamoorthi,et al.  A Real-time Beam Tracer with Application to Exact Soft Shadows , 2007, Rendering Techniques.

[25]  AmanatidesJohn,et al.  Ray tracing with cones , 1984 .

[26]  Jaakko Lehtinen,et al.  Differentiable Monte Carlo ray tracing through edge sampling , 2018, ACM Trans. Graph..

[27]  Hao Li,et al.  Soft Rasterizer: A Differentiable Renderer for Image-Based 3D Reasoning , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[28]  Derrick G. Kourie,et al.  Exploring the use of glossy light volumes for interactive global illumination , 2010, AFRIGRAPH '10.

[29]  Carsten Dachsbacher,et al.  The SGGX microflake distribution , 2015, ACM Trans. Graph..

[30]  Ketan Mulmuley,et al.  An efficient algorithm for hidden surface removal , 1989, SIGGRAPH.

[31]  Michael T. Goodrich A polygonal approach to hidden-line and hidden-surface elimination , 1992, CVGIP Graph. Model. Image Process..

[32]  Micha Sharir,et al.  A simple output-sensitive algorithm for hidden surface removal , 1992, TOGS.

[33]  Stefan Jeschke,et al.  Analytic Anti‐Aliasing of Linear Functions on Polytopes , 2012, Comput. Graph. Forum.

[34]  Shaun Nirenstein,et al.  A low dimensional framework for exact polygon-to-polygon occlusion queries , 2005, EGSR '05.

[35]  Edwin H. Blake,et al.  Exact From-Region Visibility Culling , 2002, Rendering Techniques.

[36]  Baining Guo,et al.  Non‐Linear Beam Tracing on a GPU , 2011, Comput. Graph. Forum.

[37]  Jirí Bittner Efficient construction of visibility maps using approximate occlusion sweep , 2002, SCCG '02.

[38]  Merlin Nimier-David Radiative Backpropagation: An Adjoint Method for Lightning-Fast Di erentiable Rendering , 2020 .

[39]  WattMark Light-water interaction using backward beam tracing , 1990 .

[40]  Kenny Mitchell,et al.  Visibility Silhouettes for Semi‐Analytic Spherical Integration , 2014, Comput. Graph. Forum.

[41]  F. Frances Yao,et al.  Computational Geometry , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[42]  Ketan Mulmuley,et al.  An Efficient Algorithm for Hidden Surface Removal, II , 1994, J. Comput. Syst. Sci..

[43]  Otto Nurmi A fast line-sweep algorithm for hidden line elimination , 1985, BIT Comput. Sci. Sect..

[44]  Micha Sharir,et al.  Efficient hidden surface removal for objects with small union size , 1991, SCG '91.

[45]  Steven Fortune Topological beam tracing , 1999, SCG '99.

[46]  Cheng Zhang,et al.  Path-space differentiable rendering , 2020, ACM Trans. Graph..

[47]  Wenzel Jakob,et al.  Reparameterizing discontinuous integrands for differentiable rendering , 2019, ACM Trans. Graph..

[48]  Kurt Mehlhorn,et al.  LEDA: a platform for combinatorial and geometric computing , 1997, CACM.

[49]  John C. Stutz,et al.  Modeling Images of Natural 3D Surfaces: Overview and Potential Applications , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[50]  Josiah Manson,et al.  Analytic Rasterization of Curves with Polynomial Filters , 2013, Comput. Graph. Forum.

[51]  Frédo Durand,et al.  Downsampling scattering parameters for rendering anisotropic media , 2016, ACM Trans. Graph..

[52]  Jorge Nocedal,et al.  A Stochastic Quasi-Newton Method for Large-Scale Optimization , 2014, SIAM J. Optim..

[53]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[54]  Kurt Mehlhorn,et al.  Classroom Examples of Robustness Problems in Geometric Computations , 2004, ESA.

[55]  Peter R. Atherton,et al.  Hidden surface removal using polygon area sorting , 1977, SIGGRAPH.

[56]  Chee-Keng Yap,et al.  Recent progress in exact geometric computation , 2005, J. Log. Algebraic Methods Program..

[57]  Fabrice Neyret,et al.  Hybrid mesh‐volume LoDs for all‐scale pre‐filtering of complex 3D assets , 2017, Comput. Graph. Forum.

[58]  Joseph O'Rourke,et al.  Computational Geometry in C. , 1995 .

[59]  Gert Vegter,et al.  In handbook of discrete and computational geometry , 1997 .

[60]  森重伸也 “Real‐Time Polygonal‐Light Shading with Linearly Transformed Cosines”の実装報告 , 2016 .

[61]  David A. Duce,et al.  Scalable Vector Graphics SVG 1.0 Specification , 2000 .

[62]  Ji ˇ ´ Bittner,et al.  Hierarchical Techniques for Visibility Computations , 2002 .

[63]  O. Apostu Analytic visibility in Plücker space : from theory to practical applications , 2012 .

[64]  A. James Stewart,et al.  Computing the Approximate Visibility map, with Applications to Form Factors and Discontinuity Meshing , 1998, Rendering Techniques.

[65]  Bob Carpenter,et al.  The Stan Math Library: Reverse-Mode Automatic Differentiation in C++ , 2015, ArXiv.

[66]  Edwin E. Catmull,et al.  An analytic visible surface algorithm for independent pixel processing , 1984, SIGGRAPH.

[67]  Wan-Yen Lo,et al.  Accelerating 3D deep learning with PyTorch3D , 2019, SIGGRAPH Asia 2020 Courses.

[68]  Frederick P. Brooks,et al.  Towards image realism with interactive update rates in complex virtual building environments , 1990, I3D '90.

[69]  Elmar Eisemann,et al.  Interactive Indirect Illumination Using Voxel Cone Tracing , 2011, Comput. Graph. Forum.

[70]  Vlastimil Havran,et al.  Hierarchical visibility culling with occlusion trees , 1998, Proceedings. Computer Graphics International (Cat. No.98EX149).

[71]  Shuang Zhao,et al.  Accurate appearance preserving prefiltering for rendering displacement-mapped surfaces , 2019, ACM Trans. Graph..

[72]  Craig Gotsman,et al.  Optimized occlusion culling using five-dimensional subdivision , 1999, Comput. Graph..

[73]  Pascal Barla,et al.  Diffusion curves: a vector representation for smooth-shaded images , 2008, ACM Trans. Graph..

[74]  Homan Igehy,et al.  Tracing ray differentials , 1999, SIGGRAPH.

[75]  Michael Wimmer,et al.  Analytic Visibility on the GPU , 2013, Comput. Graph. Forum.

[76]  Morgan McGuire,et al.  Hashed alpha testing , 2017, I3D.

[77]  Michael Wimmer,et al.  Non-Sampled Anti-Aliasing , 2013, VMV.

[78]  Aaron E. Lefohn,et al.  Correlation-aware semi-analytic visibility for antialiased rendering , 2018, High Performance Graphics.

[79]  Yong He,et al.  Decoupled coverage anti-aliasing , 2015, HPG '15.

[80]  Chee-Keng Yap,et al.  Towards Exact Geometric Computation , 1997, Comput. Geom..

[81]  Kurt Mehlhorn,et al.  Exact geometric computation in LEDA , 1995, SCG '95.

[82]  Jeff Erickson,et al.  Finite-resolution hidden surface removal , 1999, SODA '00.