Compressive environment matting

The existing high-quality environment matting methods usually require the capturing of a few thousand sample images and spend a few hours in data acquisition. In this paper, a novel environment matting algorithm is proposed to capture and extract the environment matte data effectively and efficiently. First, the recently developed compressive sensing theory is incorporated to reformulate the environment matting problem and simplify the data acquisition process. Next, taking into account the special properties of light refraction and reflection effects of transparent objects, two advanced priors, group clustering and Gaussian priors, as well as other basic constraints are introduced during the matte data recovery process to combat with the limited image samples, suppress the effects of the measurement noise resulted from data acquisition, and faithfully recover the sparse environment matte data. Compared with most of the existing environment matting methods, our algorithm significantly simplifies and accelerates the environment matting extraction process while still achieving high-accurate composition results.

[1]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[2]  Justin P. Haldar,et al.  Compressed-Sensing MRI With Random Encoding , 2011, IEEE Transactions on Medical Imaging.

[3]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[4]  V.K. Goyal,et al.  Compressive Sampling and Lossy Compression , 2008, IEEE Signal Processing Magazine.

[5]  David Salesin,et al.  Environment matting extensions: towards higher accuracy and real-time capture , 2000, SIGGRAPH.

[6]  Gabriel Peyré Best basis compressed sensing , 2010, IEEE Trans. Signal Process..

[7]  Yee-Hong Yang,et al.  Frequency-based environment matting , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[8]  Pieter Peers,et al.  Relighting with 4D incident light fields , 2003, ACM Trans. Graph..

[9]  D. Donoho,et al.  Sparse MRI: The application of compressed sensing for rapid MR imaging , 2007, Magnetic resonance in medicine.

[10]  Pieter Peers,et al.  Inferring reflectance functions from wavelet noise , 2005, EGSR '05.

[11]  Richard G. Baraniuk,et al.  An Architecture for Compressive Imaging , 2006, 2006 International Conference on Image Processing.

[12]  Junzhou Huang,et al.  Towards robust and effective shape modeling: Sparse shape composition , 2012, Medical Image Anal..

[13]  Soheil Darabi,et al.  Compressive Rendering: A Rendering Application of Compressed Sensing , 2011, IEEE Transactions on Visualization and Computer Graphics.

[14]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[15]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[16]  H. Seidel,et al.  Eikonal rendering: efficient light transport in refractive objects , 2007, SIGGRAPH 2007.

[17]  Junzhou Huang,et al.  Learning with dynamic group sparsity , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[18]  Soheil Darabi,et al.  Compressive Dual Photography , 2009, Comput. Graph. Forum.

[19]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[20]  Marc Levoy,et al.  Symmetric photography: exploiting data-sparseness in reflectance fields , 2006, EGSR '06.

[21]  Soheil Darabi,et al.  Eurographics Symposium on Rendering 2010 Compressive Estimation for Signal Integration in Rendering , 2022 .

[22]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[23]  Hans-Peter Seidel,et al.  Adaptive sampling of reflectance fields , 2007, TOGS.

[24]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[25]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[26]  Shree K. Nayar,et al.  Compressive Structured Light for Recovering Inhomogeneous Participating Media , 2008, ECCV.

[27]  Andrew W. Fitzgibbon,et al.  Image-based environment matting , 2002, SIGGRAPH '02.

[28]  Jianfei Cai,et al.  Fast environment matting extraction using compressive sensing , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[29]  Zhouchen Lin,et al.  Kernel Nyström method for light transport , 2009, ACM Trans. Graph..

[30]  Wojciech Matusik,et al.  Progressively-Refined Reflectance Functions from Natural Illumination , 2004 .

[31]  Pieter Peers,et al.  Compressive light transport sensing , 2009, ACM Trans. Graph..

[32]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[33]  Wojciech Matusik,et al.  Acquisition and Rendering of Transparent and Refractive Objects , 2002, Rendering Techniques.

[34]  Pieter Peers,et al.  Wavelet Environment matting , 2003, Rendering Techniques.