Real-time directional stylization of images and videos

This paper describes a real-time non-photorealistic rendering system that automatically produces stylistic effects from photos and videos. The algorithm includes the capture of salient shapes in the image using a smooth direction field and the abstraction of both colors and shapes along the direction field. In addition to the directional stylization, the proposed algorithm has many good properties. First, the temporal coherence is preserved without any extra processing. Second, the level of abstraction can be controlled within constant time complexity. Last but not least, the parallel GPU implementation allows the real-time stylization of online videos. A variety of experimental results demonstrate the effectiveness of the system in producing high-quality abstract illustrations.

[1]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[2]  John P. Collomosse,et al.  Stroke surfaces: temporally coherent artistic animations from video , 2005, IEEE Transactions on Visualization and Computer Graphics.

[3]  Harry Shum,et al.  Video tooning , 2004, ACM Trans. Graph..

[4]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Jan Kautz,et al.  Acquisition and analysis of bispectral bidirectional reflectance and reradiation distribution functions , 2009 .

[6]  Brian Cabral,et al.  Imaging vector fields using line integral convolution , 1993, SIGGRAPH.

[7]  Joachim Weickert,et al.  Anisotropic diffusion in image processing , 1996 .

[8]  Seungyong Lee,et al.  Flow-Based Image Abstraction , 2009, IEEE Transactions on Visualization and Computer Graphics.

[9]  Li Shen,et al.  Fast Shape-Simplifying Image Abstraction Using Graphics Hardware , 2009, Edutainment.

[10]  Jiawen Chen,et al.  Real-time edge-aware image processing with the bilateral grid , 2007, ACM Trans. Graph..

[11]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[12]  Henry Kang,et al.  Image and Video Abstraction by Coherence‐Enhancing Filtering , 2011, Comput. Graph. Forum.

[13]  Shin‐Tson Wu,et al.  Wiley Series in Pure and Applied Optics , 2012 .

[14]  Lucas J. van Vliet,et al.  Separable bilateral filtering for fast video preprocessing , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[15]  Song-Chun Zhu,et al.  From image parsing to painterly rendering , 2009, TOGS.

[16]  R. Deriche.  Recursively Implementing the Gaussian and its Derivatives , 1993 .

[17]  Xiaogang Jin,et al.  Mathematical Marbling , 2012, IEEE Computer Graphics and Applications.

[18]  Jürgen Döllner,et al.  Image and Video Abstraction by Anisotropic Kuwahara Filtering , 2009, Comput. Graph. Forum.

[19]  Seungyong Lee,et al.  Shape‐simplifying Image Abstraction , 2008, Comput. Graph. Forum.

[20]  Nicolai Petkov,et al.  Artistic Edge and Corner Enhancing Smoothing , 2007, IEEE Transactions on Image Processing.

[21]  Xiaogang Jin,et al.  Real-time saliency-aware video abstraction , 2009, The Visual Computer.

[22]  S. Eiho,et al.  Processing of RI-Angiocardiographic Images , 1976 .

[23]  Xiaogang Jin,et al.  Real-time feature-aware video abstraction , 2008, The Visual Computer.

[24]  Seungyong Lee,et al.  Coherent line drawing , 2007, NPAR '07.

[25]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, SIGGRAPH 2006.

[26]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[27]  Aaron Hertzmann,et al.  Non-Photorealistic Rendering and the science of art , 2010, NPAR.