Inverse image editing

We study the problem of inverse image editing, which recovers a semantically-meaningful editing history from a source image and an edited copy. Our approach supports a wide range of commonly-used editing operations such as cropping, object insertion and removal, linear and non-linear color transformations, and spatially-varying adjustment brushes. Given an input image pair, we first apply a dense correspondence method between them to match edited image regions with their sources. For each edited region, we determine geometric and semantic appearance operations that have been applied. Finally, we compute an optimal editing path from the region-level editing operations, based on predefined semantic constraints. The recovered history can be used in various applications such as image re-editing, edit transfer, and image revision control. A user study suggests that the editing histories generated from our system are semantically comparable to the ones generated by artists.

[1]  Li-Yi Wei,et al.  Nonlinear revision control for images , 2011, SIGGRAPH 2011.

[2]  Joachim Weickert,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Optic Flow in Harmony Optic Flow in Harmony Optic Flow in Harmony , 2022 .

[3]  Mira Dontcheva,et al.  A Framework for , 2011 .

[4]  Ralph R. Martin,et al.  Internet visual media processing: a survey with graphics and vision applications , 2013, The Visual Computer.

[5]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[7]  Frédo Durand,et al.  Interactive Visual Histories for Vector Graphics , 2009 .

[8]  Olga Sorkine-Hornung,et al.  Transfusive image manipulation , 2012, ACM Trans. Graph..

[9]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[10]  Carsten Rother,et al.  PatchMatch Stereo - Stereo Matching with Slanted Support Windows , 2011, BMVC.

[11]  Björn Hartmann,et al.  Delta: a tool for representing and comparing workflows , 2012, CHI.

[12]  David A. Forsyth,et al.  A Subdivision-Based Representation for Vector Image Editing , 2012, IEEE Transactions on Visualization and Computer Graphics.

[13]  Chia-Ling Tsai,et al.  Registration of Challenging Image Pairs: Initialization, Estimation, and Decision , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Ligang Liu,et al.  Animated construction of line drawings , 2011, ACM Trans. Graph..

[15]  Jitendra Malik,et al.  Large displacement optical flow , 2009, CVPR.

[16]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[17]  Ralph R. Martin,et al.  Automatic and topology-preserving gradient mesh generation for image vectorization , 2009, ACM Trans. Graph..

[18]  Shi-Min Hu,et al.  RepFinder: finding approximately repeated scene elements for image editing , 2010, ACM Trans. Graph..

[19]  Yongwei Nie,et al.  Fast Exact Nearest Patch Matching for Patch-Based Image Editing and Processing , 2011, IEEE Transactions on Visualization and Computer Graphics.

[20]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[21]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[22]  Shi-Min Hu,et al.  Change Blindness Images , 2013, IEEE Transactions on Visualization and Computer Graphics.

[23]  Dani Lischinski,et al.  Non-rigid dense correspondence with applications for image enhancement , 2011, ACM Trans. Graph..

[24]  Shi-Min Hu,et al.  ImageAdmixture: Putting Together Dissimilar Objects from Groups , 2012, IEEE Transactions on Visualization and Computer Graphics.

[25]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[26]  James F. O'Brien,et al.  Exposing photo manipulation with inconsistent reflections , 2012, TOGS.

[27]  Sylvain Paris,et al.  Learning photographic global tonal adjustment with a database of input / output image pairs , 2011, CVPR 2011.

[28]  Steven K. Feiner,et al.  Editable graphical histories , 1988, [Proceedings] 1988 IEEE Workshop on Visual Languages.

[29]  Takeo Igarashi,et al.  Generating photo manipulation tutorials by demonstration , 2009, ACM Trans. Graph..

[30]  James F. O'Brien,et al.  Exposing photo manipulation with inconsistent shadows , 2013, TOGS.

[31]  Jeffrey Heer,et al.  Graphical Histories for Visualization: Supporting Analysis, Communication, and Evaluation , 2008, IEEE Transactions on Visualization and Computer Graphics.

[32]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.