Automating Image Morphing Using Structural Similarity on a Halfway Domain

The main challenge in achieving good image morphs is to create a map that aligns corresponding image elements. Our aim is to help automate this often tedious task. We compute the map by optimizing the compatibility of corresponding warped image neighborhoods using an adaptation of structural similarity. The optimization is regularized by a thin-plate spline and may be guided by a few user-drawn points. We parameterize the map over a halfway domain and show that this representation offers many benefits. The map is able to treat the image pair symmetrically, model simple occlusions continuously, span partially overlapping images, and define extrapolated correspondences. Moreover, it enables direct evaluation of the morph in a pixel shader without mesh rasterization. We improve the morphs by optimizing quadratic motion paths and by seamlessly extending content beyond the image boundaries. We parallelize the algorithm on a GPU to achieve a responsive interface and demonstrate challenging morphs obtained with little effort.

[1]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.

[2]  Eli Shechtman,et al.  Regenerative morphing , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  John Snyder,et al.  Freeform vector graphics with controlled thin-plate splines , 2011, ACM Trans. Graph..

[4]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[6]  Olga Sorkine-Hornung,et al.  Transfusive image manipulation , 2012, ACM Trans. Graph..

[7]  D. W. Choi,et al.  Image Morphing Using Mass-Spring System , 2011 .

[8]  Yuefeng Zhang.  A fuzzy approach to digital image warping , 1996, IEEE Computer Graphics and Applications.

[9]  Richard Szeliski,et al.  High dynamic range video , 2003, ACM Trans. Graph..

[10]  Harry Shum,et al.  Pattern-based texture metamorphosis , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[11]  Martin Bichsel.  Automatic interpolation and recognition of face images by morphing , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[12]  Sung Yong Shin,et al.  Image morphing using deformable surfaces , 1994, Proceedings of Computer Animation '94.

[13]  Michele Covell,et al.  Eigen-points: control-point location using principal component analyses , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[14]  Enhua Wu,et al.  Robust image metamorphosis immune from ghost and blur , 2012, The Visual Computer.

[15]  Sung Yong Shin,et al.  Image Morphing Using Deformation Techniques , 1996, Comput. Animat. Virtual Worlds.

[16]  Lei Yang,et al.  Image-based bidirectional scene reprojection , 2011, ACM Trans. Graph..

[17]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[18]  Thomas W. Sederberg,et al.  A work minimization approach to image morphing , 1998, The Visual Computer.

[19]  George Wolberg,et al.  Digital image warping , 1990 .

[20]  Ira Kemelmacher-Shlizerman,et al.  Exploring photobios , 2011, ACM Trans. Graph..

[21]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[22]  J. Kiefer,et al.  Sequential minimax search for a maximum , 1953 .

[23]  Eli Shechtman,et al.  Image melding , 2012, ACM Trans. Graph..

[24]  Frédo Durand,et al.  Texture design using a simplicial complex of morphable textures , 2005, SIGGRAPH '05.

[25]  David Mumford,et al.  A Bayesian treatment of the stereo correspondence problem using half-occluded regions , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Luiz Velho,et al.  Warping and morphing of graphical objects , 1998 .

[27]  Jarek Rossignac,et al.  Steady affine motions and morphs , 2011, TOGS.

[28]  Carlo Tomasi,et al.  Edge, Junction, and Corner Detection Using Color Distributions , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  David Beymer,et al.  Feature correspondence by interleaving shape and texture computations , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[30]  Do-Won Choi,et al.  A fast overlap control in image morphing using mass-spring system , 2011, 2011 4th International Congress on Image and Signal Processing.

[31]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[32]  Wojciech Matusik,et al.  Moving gradients: a path-based method for plausible image interpolation , 2009, ACM Trans. Graph..

[33]  S. B. Kang,et al.  Survey of image-based representations and compression techniques , 2003, IEEE Trans. Circuits Syst. Video Technol..

[34]  George Wolberg,et al.  Image morphing: a survey , 1998, The Visual Computer.

[35]  Fei Yang,et al.  Face morphing using 3D-aware appearance optimization , 2012, Graphics Interface.

[36]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[37]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..