As-rigid-as-possible image registration for hand-drawn cartoon animations

We present a new approach to deformable image registration suitable for articulated images such as hand-drawn cartoon characters and human postures. For such type of data state-of-the-art techniques typically yield undesirable results. We propose a novel geometrically motivated iterative scheme where point movements are decoupled from shape consistency. By combining locally optimal block matching with as-rigid-as-possible shape regularization, our algorithm allows us to register images undergoing large free-form deformations and appearance variations. We demonstrate its practical usability in various challenging tasks performed in the cartoon animation production pipeline including unsupervised inbetweening, example-based shape deformation, auto-painting, editing, and motion retargeting.

[1]  Jirí Zára,et al.  Colorization of black-and-white cartoons , 2005, Image Vis. Comput..

[2]  Quan Chen,et al.  Enhanced auto coloring with hierarchical region matching , 2005, Comput. Animat. Virtual Worlds.

[3]  Václav Hlavác,et al.  Efficient MRF Deformation Model for Non-Rigid Image Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Ming Xie Feature matching and affine transformation for 2D cell animation , 2006, The Visual Computer.

[5]  Kai Xu,et al.  2D shape deformation based on rigid square matching , 2008 .

[6]  Jan Modersitzki,et al.  Numerical Methods for Image Registration , 2004 .

[7]  M. Teschner,et al.  Meshless deformations based on shape matching , 2005, SIGGRAPH 2005.

[8]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[9]  M. Pauly,et al.  Embedded deformation for shape manipulation , 2007, SIGGRAPH 2007.

[10]  Ezzatollah Salari,et al.  Successive elimination algorithm for motion estimation , 1995, IEEE Trans. Image Process..

[11]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[12]  Quan Chen,et al.  Auto Coloring with Enhanced Character Registration , 2008, Int. J. Comput. Games Technol..

[13]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[14]  Marc Alexa,et al.  As-rigid-as-possible surface modeling , 2007, Symposium on Geometry Processing.

[15]  Markus H. Gross,et al.  Adaptive Space Deformations Based on Rigid Cells , 2007, Comput. Graph. Forum.

[16]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[17]  John Dingliana,et al.  LazyBrush: Flexible Painting Tool for Hand‐drawn Cartoons , 2009, Comput. Graph. Forum.

[18]  Kun Zhou,et al.  2D shape deformation using nonlinear least squares optimization , 2006, The Visual Computer.

[19]  Nasser Kehtarnavaz,et al.  Brain Functional Localization: A Survey of Image Registration Techniques , 2007, IEEE Transactions on Medical Imaging.

[20]  Christoph Bregler,et al.  Turning to the masters: motion capturing cartoons , 2002, ACM Trans. Graph..

[21]  Nassir Navab,et al.  Dense image registration through MRFs and efficient linear programming , 2008, Medical Image Anal..

[22]  Suh-Yin Lee,et al.  Automatic cel painting in computer-assisted cartoon production using similarity recognition , 1997 .

[23]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Stefan Heldmann,et al.  A Fast and Flexible Image Registration Toolbox , 2007, Bildverarbeitung für die Medizin.

[25]  Josien P. W. Pluim,et al.  Evaluation of Optimization Methods for Nonrigid Medical Image Registration Using Mutual Information and B-Splines , 2007, IEEE Transactions on Image Processing.

[26]  Takeo Igarashi,et al.  As-rigid-as-possible shape manipulation , 2005, SIGGRAPH '05.

[27]  Oliver Schmitt,et al.  Superfast elastic registration of histologic images of a whole rat brain for 3D reconstruction , 2004, SPIE Medical Imaging.

[28]  Doug L. James,et al.  FastLSM: fast lattice shape matching for robust real-time deformation , 2007, ACM Trans. Graph..

[29]  Marc Alexa,et al.  As-rigid-as-possible shape interpolation , 2000, SIGGRAPH.

[30]  Bobby Bodenheimer,et al.  Eurographics/ Acm Siggraph Symposium on Computer Animation (2006) Re-using Traditional Animation: Methods for Semi-automatic Segmentation and Inbetweening , 2022 .

[31]  Markus H. Gross,et al.  Meshless deformations based on shape matching , 2005, ACM Trans. Graph..

[32]  Scott Schaefer,et al.  Image deformation using moving least squares , 2006, ACM Trans. Graph..

[33]  Markus H. Gross,et al.  An approach to computer-supported cartooning , 2005, The Visual Computer.

[34]  Yanzhen Wang,et al.  2D shape deformation based on rigid square matching , 2008, Comput. Animat. Virtual Worlds.

[35]  Andrew Chi-Sing Leung,et al.  Animating animal motion from still , 2008, SIGGRAPH Asia '08.

[36]  Seah Hock Soon,et al.  Computer-assisted coloring by matching line drawings , 2000, The Visual Computer.

[37]  Tian Feng,et al.  Computer-assisted coloring by matching line drawings , 2000 .

[38]  Max A. Viergever,et al.  A survey of medical image registration , 1998, Medical Image Anal..

[39]  Alexander Kort,et al.  Computer aided inbetweening , 2002, NPAR '02.

[40]  Suh-Yin Lee,et al.  Automatic Cel Painting in Computer-assisted Cartoon Production using Similarity Recognition , 1997, Comput. Animat. Virtual Worlds.