Abstracted painterly renderings using eye-tracking data

When used by artists, manual interfaces for painterly rendering can yield very satisfying abstract transformations of images. Automatic techniques produce interesting paintings as well, but can only recast pictures in a different style without performing any abstraction. At best, information is removed uniformly across the image, without emphasizing the important content. We describe a new approach for the creation of painterly renderings that draws on a model of human perception and is driven by eye-tracking data. This approach can perform meaningful abstraction using this data, with the minimum interaction possible: the user need only look at the image for several seconds. We demonstrate the effectiveness of this interactive technique and compare it with a fully automatic approach.

[1]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Dave Regan,et al.  Human perception of objects: early visual processing of spatial form defined by luminance , 2000 .

[3]  Andrew Hollingworth,et al.  Eye Movements During Scene Viewing: An Overview , 1998 .

[4]  David Salesin,et al.  Computer-generated watercolor , 1997, SIGGRAPH.

[5]  David Salesin,et al.  Interactive pen-and-ink illustration , 1994, SIGGRAPH.

[6]  M. A. Bouman,et al.  Perimetry of contrast detection thresholds of moving spatial sine wave patterns. III. The target extent as a sensitivity controlling parameter. , 1978, Journal of the Optical Society of America.

[7]  Donald P. Greenberg,et al.  A multiscale model of adaptation and spatial vision for realistic image display , 1998, SIGGRAPH.

[8]  Roel Vertegaal,et al.  Course 05: eye-based interaction in graphical systems: theory & practice , 2000, SIGGRAPH 2000.

[9]  Martin Reddy,et al.  Perceptually modulated level of detail for virtual environments , 1997 .

[10]  Aaron Hertzmann,et al.  Painterly rendering with curved brush strokes of multiple sizes , 1998, SIGGRAPH.

[11]  M. Just,et al.  Eye fixations and cognitive processes , 1976, Cognitive Psychology.

[12]  Denis G. Pelli,et al.  Human Perception of Objects: Early Visual Processing of Spatial Form Defined by Luminance, Color, Texture, Motion, and Binocular Disparity , 2001 .

[13]  S. Zeki,et al.  Inner Vision: An Exploration of Art and the Brain , 2000 .

[14]  Robert J. K. Jacob,et al.  Evaluation of eye gaze interaction , 2000, CHI.

[15]  N. Mackworth,et al.  The gaze selects informative details within pictures , 1967 .

[16]  M. A. Bouman,et al.  Perimetry of contrast detection thresholds of moving spatial sine wave patterns. I. The near peripheral visual field (eccentricity 0 degrees-8 degrees). , 1978, Journal of the Optical Society of America.

[17]  D. H. Kelly,et al.  Retinal inhomogeneity. I. Spatiotemporal contrast sensitivity. , 1984, Journal of the Optical Society of America. A, Optics and image science.

[18]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[19]  Roel Vertegaal,et al.  The GAZE groupware system: mediating joint attention in multiparty communication and collaboration , 1999, CHI '99.

[20]  A. L. I︠A︡rbus.  Eye Movements and Vision , 1967 .

[21]  Paul Haeberli,et al.  Paint by numbers: abstract image representations , 1990, SIGGRAPH.

[22]  Jan J. Koenderink,et al.  Perimetry of contrast detection thresholds of moving spatial sine wave patterns. II. The far peripheral visual field (eccentricity 0°–50°) , 1978 .

[23]  Andrea J. van Doorn,et al.  The Structure of Locally Orderless Images , 1999, International Journal of Computer Vision.

[24]  Michael C. Barris,et al.  Inner Vision: An Exploration of Art and the Brain. , 2000 .

[25]  Martin Reddy,et al.  Perceptually Optimized 3D Graphics , 2001, IEEE Computer Graphics and Applications.

[26]  Michio Shiraishi,et al.  An algorithm for automatic painterly rendering based on local source image approximation , 2000, NPAR '00.

[27]  Peter Litwinowicz,et al.  Processing images and video for an impressionist effect , 1997, SIGGRAPH.

[28]  Robert J. K. Jacob,et al.  Eye Movement-Based Human-Computer Interaction Techniques: Toward Non-Command Interfaces , 2003 .

[29]  D. Kahneman,et al.  Attention and Effort , 1973 .

[30]  Andrew T. Duchowski.  Acuity-matching resolution degradation through wavelet coefficient scaling , 2000, IEEE Trans. Image Process..