Video Painting with Space-Time-Varying Style Parameters

Artists use different means of stylization to control the focus on different objects in the scene. This allows them to portray complex meaning and achieve certain artistic effects. Most prior work on painterly rendering of videos, however, uses only a single painting style, with fixed global parameters, irrespective of objects and their layout in the images. This often leads to inadequate artistic control. Moreover, brush stroke orientation is typically assumed to follow an everywhere continuous directional field. In this paper, we propose a video painting system that accounts for the spatial support of objects in the images or videos, and uses this information to specify style parameters and stroke orientation for painterly rendering. Since objects occupy distinct image locations and move relatively smoothly from one video frame to another, our object-based painterly rendering approach is characterized by style parameters that coherently vary in space and time. Space-time-varying style parameters enable more artistic freedom, such as emphasis/de-emphasis, increase or decrease of contrast, exaggeration or abstraction of different objects in the scene in a temporally coherent fashion.

[1]  M. H. Heycock,et al.  Papers , 1971, BMJ : British Medical Journal.

[2]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[3]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[4]  Arthur C. Sanderson,et al.  Determining Object Orientation Using Ellipse Fitting , 1985, Other Conferences.

[5]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[6]  Paul Haeberli,et al.  Paint by numbers: abstract image representations , 1990, SIGGRAPH.

[7]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[8]  Charles R. Dyer,et al.  Computing spatiotemporal relations for dynamic perceptual organization , 1993 .

[9]  Brian Cabral,et al.  Imaging vector fields using line integral convolution , 1993, SIGGRAPH.

[10]  Lambertus Hesselink,et al.  Visualizing second-order tensor fields with hyperstreamlines , 1993, IEEE Computer Graphics and Applications.

[11]  Hans-Christian Hege,et al.  Fast and resolution independent line integral convolution , 1995, SIGGRAPH.

[12]  Barbara J. Meier.  Painterly rendering for animation , 1996, SIGGRAPH.

[13]  Victoria Interrante,et al.  Illustrating surface shape in volume data via principal direction-driven 3D line integral convolution , 1997, SIGGRAPH.

[14]  Peter Litwinowicz,et al.  Processing images and video for an impressionist effect , 1997, SIGGRAPH.

[15]  Murat Kunt,et al.  Spatiotemporal Segmentation Based on Region Merging , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Aaron Hertzmann,et al.  Painterly rendering with curved brush strokes of multiple sizes , 1998, SIGGRAPH.

[17]  David Salesin,et al.  Proceedings of the 1st international symposium on Non-photorealistic animation and rendering , 2000 .

[18]  Ken Perlin,et al.  Painterly rendering for video and interaction , 2000, NPAR '00.

[19]  Peter-Pike J. Sloan,et al.  Video Cubism , 2001 .

[20]  Aaron Hertzmann,et al.  Paint by relaxation , 2001, Proceedings. Computer Graphics International 2001.

[21]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[22]  Kwan-Liu Ma,et al.  Non-photorealistic rendering using watercolor inspired textures and illumination , 2001, Proceedings Ninth Pacific Conference on Computer Graphics and Applications. Pacific Graphics 2001.

[23]  Bruce Gooch,et al.  Non-photorealistic rendering , 2001 .

[24]  Aseem Agarwala,et al.  SnakeToonz: a semi-automatic approach to creating cel animation from video , 2002, NPAR '02.

[25]  Adam Finkelstein,et al.  Stylized video cubes , 2002, SCA '02.

[26]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Jarke J. van Wijk,et al.  Image based flow visualization , 2002, ACM Trans. Graph..

[28]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[29]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[30]  Levente Kovács,et al.  Creating animations combining stochastic paintbrush transformation and motion detection , 2002, Object recognition supported by user interaction for service robots.

[31]  Daniel DeMenthon,et al.  A Survey of Spatio-Temporal Grouping Techniques , 2002 .

[32]  Aaron Hertzmann,et al.  Fast paint texture , 2002, NPAR '02.

[33]  J. Stam.  Real-Time Fluid Dynamics for Games , 2003 .

[34]  Joëlle Thollot,et al.  Dynamic Canvas for Non-Photorealistic Walkthroughs , 2003, Graphics Interface.

[35]  Wen-Hsiang Tsai,et al.  Adaptive data hiding in palette images by color ordering and mapping with security protection , 2004, IEEE Transactions on Communications.

[36]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, SIGGRAPH 2004.

[37]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[38]  Irfan A. Essa,et al.  Image and video based painterly animation , 2004, NPAR '04.

[39]  Harry Shum,et al.  Video tooning , 2004, ACM Trans. Graph..

[40]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..

[41]  John P. Collomosse,et al.  Stroke surfaces: temporally coherent artistic animations from video , 2005, IEEE Transactions on Visualization and Computer Graphics.

[42]  Yang Wang,et al.  Spatiotemporal video segmentation based on graphical models , 2005, IEEE Transactions on Image Processing.

[43]  Konstantin Mischaikow,et al.  Vector field design on surfaces , 2006, TOGS.

[44]  Adrien Bousseau,et al.  Interactive watercolor rendering with temporal coherence and abstraction , 2006, NPAR.

[45]  Benoit Huet,et al.  Graph-Based Spatio-temporal Region Extraction , 2006, ICIAR.

[46]  Eugene Zhang,et al.  Interactive Tensor Field Design and Visualization on Surfaces , 2007, IEEE Transactions on Visualization and Computer Graphics.

[47]  David Salesin,et al.  Video watercolorization using bidirectional texture advection , 2007, SIGGRAPH 2007.

[48]  Youngsup Park,et al.  Painterly animation using motion maps , 2008, Graph. Model..

[49]  William Brendel,et al.  Video object segmentation by tracking regions , 2009, 2009 IEEE 12th International Conference on Computer Vision.