Image and Video Abstraction by Anisotropic Kuwahara Filtering

We present a non‐photorealistic rendering technique to transform color images and videos into painterly abstractions. It is based on a generalization of the Kuwahara filter that is adapted to the local shape of features, derived from the smoothed structure tensor. Contrary to conventional edge‐preserving filters, our filter generates a painting‐like flattening effect along the local feature directions while preserving shape boundaries. As opposed to conventional painting algorithms, it produces temporally coherent video abstraction without extra processing. The GPU implementation of our method processes video in real‐time. The results have the clearness of cartoon illustrations but also exhibit directional information as found in oil paintings.

[1]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[3]  Adam Finkelstein.  Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering , 2002 .

[4]  Peter Litwinowicz,et al.  Processing images and video for an impressionist effect , 1997, SIGGRAPH.

[5]  Bruno Lévy,et al.  Ardeco: automatic region detection and conversion , 2006, EGSR '06.

[6]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[7]  Bo Thiesson,et al.  Image and Video Segmentation by Anisotropic Kernel Mean Shift , 2004, ECCV.

[8]  F. Tomita,et al.  Extraction of Multiple Regions by Smoothing in Selected Neighborhoods , 1977 .

[9]  Aaron Hertzmann,et al.  A survey of stroke-based rendering , 2003, IEEE Computer Graphics and Applications.

[10]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  R. Boomgaard,et al.  Decomposition of the Kuwahara-Nagao Operator in terms of Linear Smoothing and Morphological Sharpening , 2002 .

[12]  M. Nagao,et al.  Edge preserving smoothing , 1979 .

[13]  Lucas J. van Vliet,et al.  Separable bilateral filtering for fast video preprocessing , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[14]  S. Eiho,et al.  Processing of RI-Angiocardiographic Images , 1976 .

[15]  Irfan A. Essa,et al.  Image and video based painterly animation , 2004, NPAR '04.

[16]  Shlomo Greenberg,et al.  Improved structure-adaptive anisotropic filter , 2006, Pattern Recognit. Lett..

[17]  Harry Shum,et al.  Video tooning , 2004, ACM Trans. Graph..

[18]  Kendall Preston,et al.  Digital processing of biomedical images , 1976 .

[19]  Aaron Hertzmann,et al.  Fast paint texture , 2002, NPAR '02.

[20]  Tuan Q. Pham.  Spatiotonal adaptivity in super-resolution of under-sampled image sequences , 2006 .

[21]  Pascal Barla,et al.  Structure-preserving manipulation of photographs , 2007, NPAR '07.

[22]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, ACM Trans. Graph..

[23]  Douglas DeCarlo,et al.  Visual interest and NPR: an evaluation and manifesto , 2004, NPAR '04.

[24]  Seungyong Lee,et al.  Flow-Based Image Abstraction , 2009, IEEE Transactions on Visualization and Computer Graphics.

[25]  Seungyong Lee,et al.  Shape‐simplifying Image Abstraction , 2008, Comput. Graph. Forum.

[26]  Harry Shum,et al.  Color sketch generation , 2006, NPAR.

[27]  John P. Collomosse,et al.  Stroke surfaces: temporally coherent artistic animations from video , 2005, IEEE Transactions on Visualization and Computer Graphics.

[28]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, ACM Trans. Graph..

[29]  Guang-Zhong Yang,et al.  Structure adaptive anisotropic image filtering , 1996, Image Vis. Comput..

[30]  Nicolai Petkov,et al.  Artistic Edge and Corner Enhancing Smoothing , 2007, IEEE Transactions on Image Processing.

[31]  Thomas Brox,et al.  Nonlinear structure tensors , 2006, Image Vis. Comput..

[32]  Joost van de Weijer,et al.  Least Squares and Robust Estimation of Local Image Structure , 2003, International Journal of Computer Vision.

[33]  Jürgen Döllner,et al.  Image Abstraction by Structure Adaptive Filtering , 2008, TPCG.