Video tooning

We describe a system for transforming an input video into a highly abstracted, spatio-temporally coherent cartoon animation with a range of styles. To achieve this, we treat video as a space-time volume of image data. We have developed an anisotropic kernel mean shift technique to segment the video data into contiguous volumes. These provide a simple cartoon style in themselves, but more importantly provide the capability to semi-automatically rotoscope semantically meaningful regions.In our system, the user simply outlines objects on keyframes. A mean shift guided interpolation algorithm is then employed to create three dimensional semantic regions by interpolation between the keyframes, while maintaining smooth trajectories along the time dimension. These regions provide the basis for creating smooth two dimensional edge sheets and stroke sheets embedded within the spatio-temporal video volume. The regions, edge sheets, and stroke sheets are rendered by slicing them at particular times. A variety of styles of rendering are shown. The temporal coherence provided by the smoothed semantic regions and sheets results in a temporally consistent non-photorealistic appearance.

[1]  Adam Finkelstein,et al.  Stylized video cubes , 2002, SCA '02.

[2]  Adam Finkelstein,et al.  Video mosaics , 2002, NPAR '02.

[3]  Aaron Hertzmann,et al.  Paint by relaxation , 2001, Proceedings. Computer Graphics International 2001.

[4]  William E. Lorensen,et al.  Marching cubes: a high resolution 3D surface construction algorithm , 1996 .

[5]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[6]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[7]  Ramesh C. Jain,et al.  Digital video segmentation , 1994, MULTIMEDIA '94.

[8]  Peter Meer,et al.  Synergism in low level vision , 2002, Object recognition supported by user interaction for service robots.

[9]  Peter Litwinowicz,et al.  Processing images and video for an impressionist effect , 1997, SIGGRAPH.

[10]  Atreyi Kankanhalli,et al.  Automatic partitioning of full-motion video , 1993, Multimedia Systems.

[11]  Dong Zhang,et al.  A New Shot Boundary Detection Algorithm , 2001, IEEE Pacific Rim Conference on Multimedia.

[12]  Alexander Kort,et al.  Computer aided inbetweening , 2002, NPAR '02.

[13]  A. Volgenant,et al.  A shortest augmenting path algorithm for dense and sparse linear assignment problems , 1987, Computing.

[14]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[16]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[17]  Ken Perlin,et al.  Painterly rendering for video and interaction , 2000, NPAR '00.

[18]  M. Floater.  Mean value coordinates , 2003, Computer Aided Geometric Design.

[19]  Michael Hoch,et al.  A semi-automatic system for edge tracking with snakes , 1996, The Visual Computer.

[20]  Aseem Agarwala,et al.  SnakeToonz: a semi-automatic approach to creating cel animation from video , 2002, NPAR '02.

[21]  Daniel DeMenthon,et al.  SPATIO-TEMPORAL SEGMENTATION OF VIDEO BY HIERARCHICAL MEAN SHIFT ANALYSIS , 2002 .

[22]  Siu Chi Hsu,et al.  Drawing and animation using skeletal strokes , 1994, SIGGRAPH.

[23]  Bo Thiesson,et al.  Image and Video Segmentation by Anisotropic Kernel Mean Shift , 2004, ECCV.

[24]  John Collomosse,et al.  Stroke Surfaces: A Spatio-temporal Framework for Temporally Coherent Non- photorealistic Animations , 2003 .

[25]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.