Compressive Dual Photography

The accurate measurement of the light transport characteristics of a complex scene is an important goal in computer graphics and has applications in relighting and dual photography. However, since the light transport data sets are typically very large, much of the previous research has focused on adaptive algorithms that capture them efficiently. In this work, we propose a novel, non‐adaptive algorithm that takes advantage of the compressibility of the light transport signal in a transform domain to capture it with less acquisitions than with standard approaches. To do this, we leverage recent work in the area of compressed sensing, where a signal is reconstructed from a few samples assuming that it is sparse in a transform domain. We demonstrate our approach by performing dual photography and relighting by using a much smaller number of acquisitions than would normally be needed. Because our algorithm is not adaptive, it is also simpler to implement than many of the current approaches.

[1]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[2]  Richard G. Baraniuk,et al.  An Architecture for Compressive Imaging , 2006, 2006 International Conference on Image Processing.

[3]  Rebecca Willett,et al.  Compressive coded aperture video reconstruction , 2008, 2008 16th European Signal Processing Conference.

[4]  O. Milenkovic,et al.  Designing Compressive Sensing DNA Microarrays , 2007, 2007 2nd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing.

[5]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[6]  Hans-Peter Seidel,et al.  Adaptive sampling of reflectance fields , 2007, TOGS.

[7]  Pieter Peers,et al.  Relighting with 4D incident light fields , 2003, ACM Trans. Graph..

[8]  Hans-Peter Seidel,et al.  DISCO: acquisition of translucent objects , 2004, ACM Trans. Graph..

[9]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[10]  Mislav Grgic,et al.  Image compression using wavelets , 1999, ISIE '99. Proceedings of the IEEE International Symposium on Industrial Electronics (Cat. No.99TH8465).

[11]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[12]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[13]  Andrew Jones,et al.  Relighting human locomotion with flowed reflectance fields , 2006, EGSR '06.

[14]  Wojciech Matusik,et al.  Progressively-Refined Reflectance Functions from Natural Illumination , 2004 .

[15]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[16]  E.J. Candes Compressive Sampling , 2022 .

[17]  Marc Levoy,et al.  Dual photography , 2005, SIGGRAPH 2005.

[18]  Rebecca Willett,et al.  Compressive coded aperture superresolution image reconstruction , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Emmanuel J. Cand The Restricted Isometry Property and Its Implications for Compressed Sensing , 2008 .

[20]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[21]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[22]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  H. Seidel,et al.  DISCO: acquisition of translucent objects , 2004, ACM Trans. Graph..

[24]  Pradeep Sen,et al.  Compressive cooperative sensing and mapping in mobile networks , 2009, ACC.

[25]  Shree K. Nayar,et al.  Compressive Structured Light for Recovering Inhomogeneous Participating Media , 2008, ECCV.

[26]  E. Candès,et al.  Error correction via linear programming , 2005, FOCS 2005.

[27]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, SIGGRAPH 2005.

[28]  Pieter Peers,et al.  Compressive light transport sensing , 2009, ACM Trans. Graph..

[29]  Pieter Peers,et al.  Wavelet Environment matting , 2003, Rendering Techniques.

[30]  Paul E. Debevec,et al.  A dual light stage , 2005, EGSR '05.

[31]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[32]  Pieter Peers,et al.  Inferring reflectance functions from wavelet noise , 2005, EGSR '05.

[33]  Shree K. Nayar,et al.  A theory of multiplexed illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[34]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[35]  Richard G. Baraniuk,et al.  A new compressive imaging camera architecture using optical-domain compression , 2006, Electronic Imaging.

[36]  Lu Gan Block Compressed Sensing of Natural Images , 2007, 2007 15th International Conference on Digital Signal Processing.

[37]  Andrew Jones,et al.  Relighting human locomotion with flowed reflectance fields , 2006, EGSR '06.

[38]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2002, The Kluwer International Series in Engineering and Computer Science.

[39]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[40]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[41]  Marc Levoy,et al.  Symmetric photography: exploiting data-sparseness in reflectance fields , 2006, EGSR '06.

[42]  Emmanuel J. Candès,et al.  Error correction via linear programming , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[43]  P. Sen,et al.  Compressed mapping of communication signal strength , 2008, MILCOM 2008 - 2008 IEEE Military Communications Conference.

[44]  Armando Manduca,et al.  Highly Undersampled Magnetic Resonance Image Reconstruction via Homotopic $\ell_{0}$ -Minimization , 2009, IEEE Transactions on Medical Imaging.

[45]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1991, CACM.

[46]  Pieter Peers,et al.  Smooth Reconstruction and Compact Representation of Reflectance Functions for Image-based Relighting , 2004, Rendering Techniques.

[47]  Karen O. Egiazarian,et al.  Compressed Sensing Image Reconstruction Via Recursive Spatially Adaptive Filtering , 2007, ICIP.