Coordinates for instant image cloning

Seamless cloning of a source image patch into a target image is an important and useful image editing operation, which has received considerable research attention in recent years. This operation is typically carried out by solving a Poisson equation with Dirichlet boundary conditions, which smoothly interpolates the discrepancies between the boundary of the source patch and the target across the entire cloned area. In this paper we introduce an alternative, coordinate-based approach, where rather than solving a large linear system to perform the aforementioned interpolation, the value of the interpolant at each interior pixel is given by a weighted combination of values along the boundary. More specifically, our approach is based on Mean-Value Coordinates (MVC). The use of coordinates is advantageous in terms of speed, ease of implementation, small memory footprint, and parallelizability, enabling real-time cloning of large regions, and interactive cloning of video streams. We demonstrate a number of applications and extensions of the coordinate-based framework.

[1]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[2]  L. Greengard,et al.  A Fast Adaptive Multipole Algorithm for Particle Simulations , 1988 .

[3]  Pat Hanrahan,et al.  A rapid hierarchical radiosity algorithm , 1991, SIGGRAPH.

[4]  Joe D. Warren,et al.  Barycentric coordinates for convex polytopes , 1996, Adv. Comput. Math..

[5]  S. Palmer.  Vision Science : Photons to Phenomenology , 1999 .

[6]  Yair Weiss,et al.  Deriving intrinsic images from image sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[7]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[8]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .

[9]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[10]  M. Floater.  Mean value coordinates , 2003, Computer Aided Geometric Design.

[11]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[12]  Shmuel Peleg,et al.  Seamless Image Stitching in the Gradient Domain , 2004, ECCV.

[13]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[14]  N. Ahuja,et al.  Seamless video editing , 2004, ICPR 2004.

[15]  Martin Reimers,et al.  Mean value coordinates in 3D , 2005, Comput. Aided Geom. Des..

[16]  J. Warren,et al.  Mean value coordinates for closed triangular meshes , 2005, SIGGRAPH 2005.

[17]  P. Schröder,et al.  Sparse matrix solvers on the GPU: conjugate gradients and multigrid , 2003, SIGGRAPH Courses.

[18]  Jian Sun,et al.  Drag-and-drop pasting , 2006, SIGGRAPH 2006.

[19]  R. Szeliski.  Locally adapted hierarchical basis preconditioning , 2006, SIGGRAPH 2006.

[20]  Kai Hormann,et al.  Mean value coordinates for arbitrary planar polygons , 2006, TOGS.

[21]  Mark Meyer,et al.  Harmonic coordinates for character articulation , 2007, SIGGRAPH 2007.

[22]  Aseem Agarwala.  Efficient gradient-domain compositing using quadtrees , 2007, SIGGRAPH 2007.

[23]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Nancy S. Pollard,et al.  Real-time gradient-domain painting , 2008, SIGGRAPH 2008.

[25]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[26]  Hans-Peter Seidel,et al.  Higher Order Barycentric Coordinates , 2008, Comput. Graph. Forum.

[27]  Michael Kazhdan,et al.  Streaming multigrid for gradient-domain operations on large images , 2008, SIGGRAPH 2008.