Compressive Rendering of Multidimensional Scenes

Recently, we proposed the idea of using compressed sensing to reconstruct the 2D images produced by a rendering system, a process we called compressive rendering. In this work, we present the natural extension of this idea to multidimensional scene signals as evaluated by a Monte Carlo rendering system. Basically, we think of a distributed ray tracing system as taking point samples of a multidimensional scene function that is sparse in a transform domain. We measure a relatively small set of point samples and then use compressed sensing algorithms to reconstruct the original multidimensional signal by looking for sparsity in a transform domain. Once we reconstruct an approximation to the original scene signal, we can integrate it down to a final 2D image which is output by the rendering system. This general form of compressive rendering allows us to produce effects such as depth-of-field, motion blur, and area light sources, and also renders animated sequences efficiently.

[1]  Donald P. Greenberg,et al.  Interactive global illumination in dynamic scenes , 2002, SIGGRAPH.

[2]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[3]  Matthias Zwicker,et al.  Multidimensional adaptive sampling and reconstruction for ray tracing , 2008, ACM Trans. Graph..

[4]  Markus H. Gross,et al.  Two methods for wavelet-based volume rendering , 1997, Comput. Graph..

[5]  Emmanuel J. Candès,et al.  Error correction via linear programming , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[6]  Farrokh Marvasti,et al.  Analysis and recovery of multidimensional signals from irregular samples using nonlinear and iterative techniques , 1994, Signal Process..

[7]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[8]  Soheil Darabi,et al.  Compressive Dual Photography , 2009, Comput. Graph. Forum.

[9]  Pat Hanrahan,et al.  All-frequency shadows using non-linear wavelet lighting approximation , 2003, ACM Trans. Graph..

[10]  F. Marvasti Nonuniform sampling : theory and practice , 2001 .

[11]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[12]  Soheil Darabi,et al.  Compressive Rendering: A Rendering Application of Compressed Sensing , 2011, IEEE Transactions on Visualization and Computer Graphics.

[13]  George Drettakis,et al.  Interactive Rendering using the Render Cache , 1999, Rendering Techniques.

[14]  Pat Hanrahan,et al.  Shadow silhouette maps , 2003, ACM Trans. Graph..

[15]  S. Mallat A wavelet tour of signal processing , 1998 .

[16]  Dani Lischinski,et al.  Combining hierarchical radiosity and discontinuity meshing , 1993, SIGGRAPH.

[17]  Jan Kautz,et al.  Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments , 2002 .

[18]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[19]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[20]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2002, The Kluwer International Series in Engineering and Computer Science.

[21]  Marc Levoy,et al.  Frequency domain volume rendering , 1993, SIGGRAPH.

[22]  H. Jensen Realistic Image Synthesis Using Photon Mapping , 2001 .

[23]  Pradeep Sen,et al.  Silhouette maps for improved texture magnification , 2004, Graphics Hardware.

[24]  Pat Hanrahan,et al.  Triple product wavelet integrals for all-frequency relighting , 2004, ACM Trans. Graph..

[25]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[26]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[27]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[28]  J. Painter,et al.  Antialiased ray tracing by adaptive progressive refinement , 1989, SIGGRAPH.

[29]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[30]  Turner Whitted,et al.  An improved illumination model for shaded display , 1979, CACM.

[31]  Baining Guo,et al.  Progressive radiance evaluation using directional coherence maps , 1998, SIGGRAPH.

[32]  David Salesin,et al.  Progressive Previewing of Ray-Traced Images Using Image Plane Disconinuity Meshing , 1997, Rendering Techniques.

[33]  Soheil Darabi,et al.  Eurographics Symposium on Rendering 2010 Compressive Estimation for Signal Integration in Rendering , 2022 .

[34]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[35]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[36]  Donald P. Greenberg,et al.  Combining edges and points for interactive high-quality rendering , 2003, ACM Trans. Graph..

[37]  Greg Humphreys,et al.  A spatial data structure for fast Poisson-disk sample generation , 2006, ACM Trans. Graph..

[38]  Thomas Malzbender,et al.  Fourier volume rendering , 1993, TOGS.

[39]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[40]  Leonidas J. Guibas,et al.  Robust Monte Carlo methods for light transport simulation , 1997 .

[41]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[42]  Philippe Bekaert,et al.  Advanced Global Illumination, Second Edition , 2006 .

[43]  Don P. Mitchell,et al.  Generating antialiased images at low sampling densities , 1987, SIGGRAPH.

[44]  Gary W. Meyer,et al.  A frequency based ray tracer , 1995, SIGGRAPH.

[45]  Pat Hanrahan,et al.  Wavelet Projections for Radiosity , 1994, Comput. Graph. Forum.

[46]  P. Hanrahan,et al.  Triple product wavelet integrals for all-frequency relighting , 2004, SIGGRAPH 2004.

[47]  Pat Hanrahan,et al.  An efficient representation for irradiance environment maps , 2001, SIGGRAPH.

[48]  Pat Hanrahan,et al.  Wavelet radiosity , 1993, SIGGRAPH.

[49]  Pat Hanrahan,et al.  A rapid hierarchical radiosity algorithm , 1991, SIGGRAPH.

[50]  T. Strohmer,et al.  Efficient numerical methods in non-uniform sampling theory , 1995 .

[51]  R. Ramamoorthi,et al.  Adaptive wavelet rendering , 2009, SIGGRAPH 2009.

[52]  David Salesin,et al.  Wavelets for computer graphics: theory and applications , 1996 .

[53]  Pieter Peers,et al.  Compressive light transport sensing , 2009, ACM Trans. Graph..

[54]  Yaakov Tsaig,et al.  Extensions of compressed sensing , 2006, Signal Process..

[55]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[56]  Peter Schröder,et al.  Wavelets in computer graphics , 1996, Proc. IEEE.

[57]  Robert L. Cook,et al.  Distributed ray tracing , 1984, SIGGRAPH.

[58]  Philippe Bekaert,et al.  Advanced global illumination , 2006 .