Intent-aware image cloning

Currently, gradient domain methods are popular for producing seamless cloning of a source image patch into a target image. However, structure conflicts between the source image patch and the target image may generate artifacts, preventing the general practices. In this paper, we tackle the challenge by incorporating the users’ intent in outlining the source patch, where the boundary drawn generally has different appearances from the objects of interest. We first reveal that artifacts exist in the over-included region, the region outside the objects of interest in the source patch. Then we use the diversity from the boundary to approximately distinguish the objects from the over-included region, and design a new algorithm to make the target image adaptively take effects in blending. So the structure conflicts can be efficiently suppressed to remove the artifacts around the objects of interest in the composite result. Moreover, we develop an interpolation measure to composite the final image rather than solving a Poisson equation, and speed up the interpolation by treating pixels in clusters and using hierarchical sampling techniques. Our method is simple to use for instant and high-quality image cloning, in which users only need to outline a region of interested objects to process. Our experimental results have demonstrated the effectiveness of our cloning method.

[1]  Stefan Jeschke,et al.  A GPU Laplacian solver for diffusion curves and Poisson image editing , 2009, ACM Trans. Graph..

[2]  Lizhuang Ma,et al.  Seamless video composition using optimized mean-value cloning , 2010, The Visual Computer.

[3]  Frédo Durand,et al.  Defocus video matting , 2005, ACM Trans. Graph..

[4]  Manuel Menezes de Oliveira Neto,et al.  Shared Sampling for Real‐Time Alpha Matting , 2010, Comput. Graph. Forum.

[5]  Eli Shechtman,et al.  Image melding , 2012, ACM Trans. Graph..

[6]  Aseem Agarwala,et al.  Efficient gradient-domain compositing using quadtrees , 2007, ACM Trans. Graph..

[7]  Wojciech Matusik,et al.  Multi-scale image harmonization , 2010, SIGGRAPH 2010.

[8]  Zeev Farbman,et al.  Coordinates for instant image cloning , 2009, ACM Trans. Graph..

[9]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[10]  Michael F. Cohen,et al.  Simultaneous Matting and Compositing , 2006, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Jian Sun,et al.  Drag-and-drop pasting , 2006, SIGGRAPH 2006.

[12]  Michael M. Kazhdan,et al.  Streaming multigrid for gradient-domain operations on large images , 2008, ACM Trans. Graph..

[13]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[14]  Hui Du,et al.  Object cloning using constrained mean value interpolation , 2012, The Visual Computer.

[15]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Jian Sun,et al.  Poisson matting , 2004, ACM Trans. Graph..

[17]  Dani Lischinski,et al.  Spectral Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Fabio Pellacini,et al.  AppProp: all-pairs appearance-space edit propagation , 2008, ACM Trans. Graph..

[19]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[20]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[21]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[22]  Richard Szeliski,et al.  Locally adapted hierarchical basis preconditioning , 2006, SIGGRAPH '06.

[23]  Michael Kazhdan,et al.  Streaming multigrid for gradient-domain operations on large images , 2008, SIGGRAPH 2008.

[24]  Ruofeng Tong,et al.  Content-aware copying and pasting in images , 2010, The Visual Computer.

[25]  Yun Zhang,et al.  Environment-Sensitive cloning in images , 2011, The Visual Computer.