Line‐Drawing Video Stylization

We present a method to automatically convert videos and CG animations to stylized animated line drawings. Using a data‐driven approach, the animated drawings can follow the sketching style of a specific artist. Given an input video, we first extract edges from the video frames and vectorize them to curves. The curves are matched to strokes from an artist's library, while following the artist's stroke distribution and characteristics. The key challenge in this process is to match the large number of curves in the frames over time, despite topological and geometric changes, allowing to maintain temporal coherence in the output animation. We solve this problem using constrained optimization to build correspondences between tracked points and create smooth sheets over time. These sheets are then replaced with strokes from the artist's database to render the final animation. We evaluate our tracking algorithm on various examples and show stylized animation results based on various artists.

[1]  Paul L. Rosin,et al.  Image and Video-Based Artistic Stylisation , 2012, Computational Imaging and Vision.

[2]  Ariel Shamir,et al.  Style and abstraction in portrait sketching , 2013, ACM Trans. Graph..

[3]  Adam Finkelstein,et al.  Coherent stylized silhouettes , 2003, ACM Trans. Graph..

[4]  Markus H. Gross,et al.  BetweenIT: An Interactive Tool for Tight Inbetweening , 2010, Comput. Graph. Forum.

[5]  John P. Collomosse,et al.  Video Paintbox: The fine art of video painting , 2005, Comput. Graph..

[6]  Robert L. Stevenson,et al.  Robust contour tracking based on a coupling between geodesic active contours and conditional random fields , 2011, J. Vis. Commun. Image Represent..

[7]  Aseem Agarwala,et al.  SnakeToonz: a semi-automatic approach to creating cel animation from video , 2002, NPAR '02.

[8]  Markus H. Gross,et al.  Temporal noise control for sketchy animation , 2011, NPAR '11.

[9]  Rui Hu,et al.  Stylized ambient displays of digital media collections , 2011, Comput. Graph..

[10]  Stephen DiVerdi,et al.  HelpingHand: example-based stroke stylization , 2012, ACM Trans. Graph..

[11]  Seungyong Lee,et al.  Coherent line drawing , 2007, NPAR '07.

[12]  Arvind Kumar,et al.  Exact and Heuristic Algorithms for the Weapon-Target Assignment Problem , 2007, Oper. Res..

[13]  John P. Collomosse,et al.  Stroke surfaces: temporally coherent artistic animations from video , 2005, IEEE Transactions on Visualization and Computer Graphics.

[14]  Joëlle Thollot,et al.  Active strokes: coherent line stylization for animated 3D models , 2012, NPAR '12.

[15]  Derek Nowrouzezahrai,et al.  Learning hatching for pen-and-ink illustration of surfaces , 2012, TOGS.

[16]  Harry Shum,et al.  Video tooning , 2004, ACM Trans. Graph..

[17]  Michael Cohen,et al.  Video tooning , 2004, SIGGRAPH 2004.

[18]  Markus H. Gross,et al.  Topology-driven vectorization of clean line drawings , 2013, ACM Trans. Graph..

[19]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, ACM Trans. Graph..

[20]  Joshua B. Tenenbaum,et al.  Learning style translation for the lines of a drawing , 2003, TOGS.

[21]  Seah Hock Soon,et al.  Animated 3D Line Drawings with Temporal Coherence , 2013, Comput. Graph. Forum.

[22]  Mei Han,et al.  Efficient hierarchical graph-based video segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Sariel Har-Peled,et al.  The fréchet distance revisited and extended , 2012, TALG.

[24]  Tamy Boubekeur,et al.  Spatio-temporal analysis for parameterizing animated lines , 2011, NPAR '11.

[25]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[26]  Aaron Hertzmann,et al.  AniPaint: Interactive Painterly Animation from Video , 2012, IEEE Transactions on Visualization and Computer Graphics.

[27]  Nicolas Papadakis,et al.  A Variational Technique for Time Consistent Tracking of Curves and Motion , 2008, Journal of Mathematical Imaging and Vision.

[28]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..

[29]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[30]  Joëlle Thollot,et al.  Temporally Coherent Video Stylization , 2013, Image and Video-Based Artistic Stylisation.

[31]  Tobias Isenberg,et al.  State of the "Art”: A Taxonomy of Artistic Stylization Techniques for Images and Video , 2013, IEEE Transactions on Visualization and Computer Graphics.