Error-Tolerant Image Compositing

Gradient-domain compositing is an essential tool in computer vision and its applications, e.g., seamless cloning, panorama stitching, shadow removal, scene completion and reshuffling. While easy to implement, these gradient-domain techniques often generate bleeding artifacts where the composited image regions do not match. One option is to modify the region boundary to minimize such mismatches. However, this option may not always be sufficient or applicable, e.g., the user or algorithm may not allow the selection to be altered. We propose a new approach to gradient-domain compositing that is robust to inaccuracies and prevents color bleeding without changing the boundary location. Our approach improves standard gradient-domain compositing in two ways. First, we define the boundary gradients such that the produced gradient field is nearly integrable. Second, we control the integration process to concentrate residuals where they are less conspicuous. We show that our approach can be formulated as a standard least-squares problem that can be solved with a sparse linear system akin to the classical Poisson equation. We demonstrate results on a variety of scenes. The visual quality and run-time complexity compares favorably to other approaches.

[1]  Cheng Lu,et al.  On the removal of shadows from images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Andrew Zisserman,et al.  Get Out of my Picture! Internet-based Inpainting , 2009, BMVC.

[3]  Edward H. Adelson,et al.  Recovering intrinsic images from a single image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Frédo Durand,et al.  Two-scale tone management for photographic look , 2006, SIGGRAPH 2006.

[6]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[7]  William T. Freeman,et al.  The Patch Transform , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[9]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, SIGGRAPH 2008.

[10]  Jian Sun,et al.  Drag-and-drop pasting , 2006, SIGGRAPH 2006.

[11]  Pierre Kornprobst,et al.  Mathematical problems in image processing - partial differential equations and the calculus of variations , 2010, Applied mathematical sciences.

[12]  Michael F. Cohen,et al.  GradientShop: A gradient-domain optimization framework for image and video filtering , 2010, TOGS.

[13]  Rama Chellappa,et al.  Enforcing integrability by error correction using l1-minimization , 2009, CVPR.

[14]  Rama Chellappa,et al.  What Is the Range of Surface Reconstructions from a Gradient Field? , 2006, ECCV.

[15]  Alexei A. Efros,et al.  Photo clip art , 2007, SIGGRAPH 2007.

[16]  Sylvain Lefebvre,et al.  An Interactive Perceptual Rendering Pipeline using Contrast and Spatial Masking , 2007, Rendering Techniques.

[17]  Philip Dutré,et al.  The influence of shape on the perception of material reflectance , 2007, SIGGRAPH 2007.

[18]  Aseem Agarwala Efficient gradient-domain compositing using quadtrees , 2007, SIGGRAPH 2007.

[19]  Todor Georgiev,et al.  Covariant Derivatives and Vision , 2006, ECCV.

[20]  Cheng Lu,et al.  Entropy Minimization for Shadow Removal , 2009, International Journal of Computer Vision.

[21]  Kavita Bala,et al.  Perception of complex aggregates , 2008, SIGGRAPH 2008.

[22]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[23]  Frédo Durand,et al.  De-emphasis of distracting image regions using texture power maps , 2005, APGV '05.

[24]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2007, SIGGRAPH 2007.

[25]  Dani Lischinski,et al.  Coordinates for instant image cloning , 2009, SIGGRAPH 2009.

[26]  Bruce Walter,et al.  Visual equivalence: towards a new standard for image fidelity , 2007, SIGGRAPH 2007.

[27]  Shmuel Peleg,et al.  Seamless Image Stitching in the Gradient Domain , 2004, ECCV.

[28]  Antonio Torralba,et al.  Infinite Images: Creating and Exploring a Large Photorealistic Virtual Space , 2008, Proceedings of the IEEE.