2D Image Deformation Based on Guaranteed Feature Correspondence and Mesh Mapping

Image deformation has ubiquitous usage in multimedia applications. It morphs one image into another through a seamless transition. Existing techniques either mainly focus on the correspondence mapping of interior features of the objects in two images, without considering object contours, or sketch contours manually, resulting in tedious work for users. Thus, we propose a 2D image deformation method, which extracts object contours automatically, considers both inner features and contours as constraints and preserves image features in terms of visual importance. Our method first automatically extracts the object contours in the source and target images and then allows users to sketch some interior features in both the images. Then, our method tessellates two images to generate two triangular meshes and builds a guaranteed bijective mesh mapping between them. We also prove the bijectivity of our mesh mapping and discuss its other desirable properties. Then, our method generates the intermediate images between the source and target images by calculating the intermediate meshes and pixels of each intermediate image. Our method realizes automatic contour extraction, provides an intuitive user interface and utilizes harmonic maps to establish a bijective mesh mapping. Therefore, it preserves more significant features with less distortion and works well for many image deformation cases in real time.

[1]  P. Kanongchaiyos,et al.  Face morphing using critical point filters , 2012, 2012 Ninth International Conference on Computer Science and Software Engineering (JCSSE).

[2]  Craig Gotsman,et al.  Variational harmonic maps for space deformation , 2009, ACM Trans. Graph..

[3]  Chalapathy Neti,et al.  Animating expressive faces across languages , 2004, IEEE Transactions on Multimedia.

[4]  Thaddeus Beier,et al.  Feature-based image metamorphosis , 1992, SIGGRAPH.

[5]  Yaron Lipman,et al.  Bounded distortion mapping spaces for triangular meshes , 2012, ACM Trans. Graph..

[6]  Ying He,et al.  Sketch based image deformation and editing with guaranteed feature correspondence , 2011, VRCAI '11.

[7]  Mark Meyer,et al.  Harmonic coordinates for character articulation , 2007, ACM Trans. Graph..

[8]  Tony DeRose,et al.  Multiresolution analysis of arbitrary meshes , 1995, SIGGRAPH.

[9]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[10]  Scott Schaefer,et al.  Bijective parameterization with free boundaries , 2015, ACM Trans. Graph..

[11]  George Wolberg,et al.  Image morphing: a survey , 1998, The Visual Computer.

[12]  Wei Zeng,et al.  Hyperbolic Harmonic Mapping for Surface Registration , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Nur Arad,et al.  Image Warping Using Few Anchor Points and Radial Functions , 1995, Comput. Graph. Forum.

[14]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[15]  Yaser Sheikh,et al.  Recycle-GAN: Unsupervised Video Retargeting , 2018, ECCV.

[16]  Rynson W. H. Lau,et al.  Contour-Based Warping , 1998, Graph. Model. Image Process..

[17]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[18]  Daniel Cohen-Or,et al.  Green Coordinates , 2008, ACM Trans. Graph..

[19]  Ari Rappoport,et al.  Shape blending using the star-skeleton representation , 1995, IEEE Computer Graphics and Applications.

[20]  Rephael Wenger,et al.  Constructing Piecewise Linear Homeomorphisms of Simple Polygons , 1997, J. Algorithms.

[21]  Marc Alexa,et al.  As-rigid-as-possible shape interpolation , 2000, SIGGRAPH.

[22]  Jürgen Jost,et al.  Harmonic mappings between Riemannian manifolds , 1983 .

[23]  Ligang Liu,et al.  A Local/Global Approach to Mesh Parameterization , 2008, Comput. Graph. Forum.

[24]  Ofir Weber,et al.  Controllable conformal maps for shape deformation and interpolation , 2010, ACM Trans. Graph..

[25]  Marc Alexa,et al.  Sketch Based Image Deformation , 2007, VMV.

[26]  Chao-Hung Lin,et al.  Patch-Based Image Warping for Content-Aware Retargeting , 2013, IEEE Transactions on Multimedia.

[27]  Scott Schaefer,et al.  Image deformation using moving least squares , 2006, ACM Trans. Graph..

[28]  Enhua Wu,et al.  Sketch-based design for green geometry and image deformation , 2011, Multimedia Tools and Applications.

[29]  David Salesin,et al.  Synthesizing realistic facial expressions from photographs , 1998, SIGGRAPH.

[30]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Gershon Elber,et al.  Image Morphing with Feature Preserving Texture , 1999, Comput. Graph. Forum.

[32]  Wen-Huang Cheng,et al.  Animating Still Landscape Photographs Through Cloud Motion Creation , 2016, IEEE Transactions on Multimedia.

[33]  Craig Gotsman,et al.  High quality compatible triangulations , 2004, Engineering with Computers.

[34]  Yao Zhao,et al.  Region-Aware 3-D Warping for DIBR , 2016, IEEE Transactions on Multimedia.

[35]  Daniel Cohen-Or,et al.  Contour blending using warp-guided distance field interpolation , 1996, Proceedings of Seventh Annual IEEE Visualization '96.

[36]  William V. Baxter,et al.  Compatible Embedding for 2D Shape Animation , 2009, IEEE Transactions on Visualization and Computer Graphics.