Compressive light transport sensing

In this article we propose a new framework for capturing light transport data of a real scene, based on the recently developed theory of compressive sensing. Compressive sensing offers a solid mathematical framework to infer a sparse signal from a limited number of nonadaptive measurements. Besides introducing compressive sensing for fast acquisition of light transport to computer graphics, we develop several innovations that address specific challenges for image-based relighting, and which may have broader implications. We develop a novel hierarchical decoding algorithm that improves reconstruction quality by exploiting interpixel coherency relations. Additionally, we design new nonadaptive illumination patterns that minimize measurement noise and further improve reconstruction quality. We illustrate our framework by capturing detailed high-resolution reflectance fields for image-based relighting.

[1]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[2]  Wojciech Matusik,et al.  Progressively-Refined Reflectance Functions from Natural Illumination , 2004 .

[3]  Pieter Peers,et al.  Relighting with 4D incident light fields , 2003, ACM Trans. Graph..

[4]  J. Tropp,et al.  SIGNAL RECOVERY FROM PARTIAL INFORMATION VIA ORTHOGONAL MATCHING PURSUIT , 2005 .

[5]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[6]  Pat Hanrahan,et al.  All-frequency shadows using non-linear wavelet lighting approximation , 2003, ACM Trans. Graph..

[7]  Hans-Peter Seidel,et al.  Adaptive sampling of reflectance fields , 2007, TOGS.

[8]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[9]  E.J. Candes Compressive Sampling , 2022 .

[10]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[11]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[12]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[13]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[14]  David B. Dunson,et al.  Multi-Task Compressive Sensing , 2007 .

[15]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[16]  Marc Levoy,et al.  Symmetric photography: exploiting data-sparseness in reflectance fields , 2006, EGSR '06.

[17]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[18]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[19]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[20]  Michael Elad,et al.  Optimized Projections for Compressed Sensing , 2007, IEEE Transactions on Signal Processing.

[21]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[22]  Pieter Peers,et al.  Inferring reflectance functions from wavelet noise , 2005, EGSR '05.

[23]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[24]  David Salesin,et al.  Environment matting extensions: towards higher accuracy and real-time capture , 2000, SIGGRAPH.

[25]  P. Debevec,et al.  A Reflective Light Stage , 2006 .

[26]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[27]  Pieter Peers,et al.  Wavelet Environment matting , 2003, Rendering Techniques.

[28]  R. Vershynin,et al.  Signal Recovery from Inaccurate and Incomplete Measurements via Regularized Orthogonal Matching Pursuit , 2010 .

[29]  Yaakov Tsaig,et al.  Extensions of compressed sensing , 2006, Signal Process..

[30]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[31]  Harry Shum,et al.  Eurographics Symposium on Rendering (2004) All-frequency Precomputed Radiance Transfer for Glossy Objects , 2022 .

[32]  Stephen J. Wright Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[33]  Deanna Needell,et al.  Signal Recovery From Incomplete and Inaccurate Measurements Via Regularized Orthogonal Matching Pursuit , 2007, IEEE Journal of Selected Topics in Signal Processing.

[34]  David B. Dunson,et al.  Multitask Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[35]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[36]  Pieter Peers,et al.  Smooth Reconstruction and Compact Representation of Reflectance Functions for Image-based Relighting , 2004, Rendering Techniques.

[37]  Hyun Sung Chang,et al.  Learning Compressed Sensing , 2007 .

[38]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.