Image Abstraction by Structure Adaptive Filtering

In this work we present a framework of automatic non-photorealistic image processing techniques that create simplified stylistic illustrations from color images, videos and 3D renderings. To smooth low-contrast regions while preserving edges, we present a new fast separated implementation of the bilateral filter. Our approach works by filtering in direction of the gradient and then filtering the intermediate result in perpendicular direction. When applied iteratively, our approach does not suffer from horizontal or vertical artifacts and creates smooth output at curved boundaries. To extract salient important edges we first apply a one-dimensional difference-of-Gaussians filter in direction of the gradient and then apply smoothing along a flow field which we derive from the smoothed structure tensor. Our method creates smooth coherent output for line and curve segments.

[1]  Dirk Bartz,et al.  Stylized augmented reality for improved immersion , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[2]  Silvano Di Zenzo,et al.  A note on the gradient of a multi-image , 1986, Comput. Vis. Graph. Image Process..

[3]  Douglas DeCarlo,et al.  Abstracted painterly renderings using eye-tracking data , 2002, NPAR '02.

[4]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Brian Cabral,et al.  Imaging vector fields using line integral convolution , 1993, SIGGRAPH.

[6]  Tuan Q. Pham Spatiotonal adaptivity in super-resolution of under-sampled image sequences , 2006 .

[7]  Lucas J. van Vliet,et al.  Edge preserving orientation adaptive filtering , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[8]  Ben Weiss,et al.  Fast median and bilateral filtering , 2006, ACM Trans. Graph..

[9]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[10]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[11]  Gunther Wyszecki,et al.  Color Science: Concepts and Methods, Quantitative Data and Formulae, 2nd Edition , 2000 .

[12]  Andrew P. Witkin,et al.  Analyzing Oriented Patterns , 1985, IJCAI.

[13]  Irfan A. Essa,et al.  Image and video based painterly animation , 2004, NPAR '04.

[14]  Jiawen Chen,et al.  Real-time edge-aware image processing with the bilateral grid , 2007, SIGGRAPH 2007.

[15]  Seungyong Lee,et al.  Coherent line drawing , 2007, NPAR '07.

[16]  A. M. Taratorin,et al.  Digital Processing of Biomedical Dynamical Images , 1988 .

[17]  Lucas J. van Vliet,et al.  Separable bilateral filtering for fast video preprocessing , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[18]  Niall Redmond,et al.  Adaptive Abstraction of 3D scenes in Real-Time , 2007, Eurographics.

[19]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, ECCV.

[20]  K. Swamy,et al.  On Sylvester's criterion for positive-semidefinite matrices , 1973 .

[21]  Douglas DeCarlo,et al.  Visual interest and NPR: an evaluation and manifesto , 2004, NPAR '04.

[22]  Pascal Barla,et al.  Structure-preserving manipulation of photographs , 2007, NPAR '07.

[23]  H. Knutsson Representing Local Structure Using Tensors , 1989 .

[24]  Johan Wiklund,et al.  Multidimensional Orientation Estimation with Applications to Texture Analysis and Optical Flow , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Holger Winnemöller,et al.  Real-time video abstraction , 2006, SIGGRAPH 2006.

[26]  Jörg Weule,et al.  Non-Linear Gaussian Filters Performing Edge Preserving Diffusion , 1995, DAGM-Symposium.

[27]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[28]  Michael Cohen,et al.  Video tooning , 2004, SIGGRAPH 2004.

[29]  Aldo Cumani,et al.  Edge detection in multispectral images , 1991, CVGIP Graph. Model. Image Process..

[30]  Erik Reinhard,et al.  Human facial illustrations: Creation and psychophysical evaluation , 2004, TOGS.

[31]  Piet W. Verbeek,et al.  Estimators for Orientation and Anisotropy in Digitized Images , 1995 .

[32]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Harry Shum,et al.  Video tooning , 2004, ACM Trans. Graph..

[34]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[35]  Dorin Comaniciu,et al.  A common framework for nonlinear diffusion, adaptive smoothing, bilateral filtering and mean shift , 2004, Image Vis. Comput..