Painterly rendering techniques: a state‐of‐the‐art review of current approaches

In this publication we will look at the different methods presented over the past few decades which attempt to recreate digital paintings. While previous surveys concentrate on the broader subject of non‐photorealistic rendering, the focus of this paper is firmly placed on painterly rendering techniques. We compare different methods used to produce different output painting styles such as abstract, colour pencil, watercolour, oriental, oil and pastel. Whereas some methods demand a high level of interaction using a skilled artist, others require simple parameters provided by a user with little or no artistic experience. Many methods attempt to provide more automation with the use of varying forms of reference data. This reference data can range from still photographs, video, 3D polygonal meshes or even 3D point clouds. The techniques presented here endeavour to provide tools and styles that are not traditionally available to an artist. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  S. Eiho,et al.  Processing of RI-Angiocardiographic Images , 1976 .

[2]  Douglas DeCarlo,et al.  Abstracted painterly renderings using eye-tracking data , 2002, NPAR '02.

[3]  Steve Strassmann,et al.  Hairy brushes , 1986, SIGGRAPH.

[4]  Masayuki Nakajima,et al.  IR2s: interactive real photo to Sumi-e , 2010, NPAR.

[5]  Paul Haeberli,et al.  Paint by numbers: abstract image representations , 1990, SIGGRAPH.

[6]  Liang Lin,et al.  Painterly animation using video semantics and feature correspondence , 2010, NPAR.

[7]  Song-Chun Zhu,et al.  Sisley the abstract painter , 2010, NPAR.

[8]  Bruce Gooch,et al.  Interactive vector fields for painterly rendering , 2005, Graphics Interface.

[9]  Peter Meer,et al.  Synergism in low level vision , 2002, Object recognition supported by user interaction for service robots.

[10]  William V. Baxter,et al.  Detail-preserving paint modeling for 3D brushes , 2010, NPAR.

[11]  Irfan A. Essa,et al.  Image and video based painterly animation , 2004, NPAR '04.

[12]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[13]  Youngsup Park,et al.  Painterly animation using motion maps , 2008, Graph. Model..

[14]  Henry Kang,et al.  Image and Video Abstraction by Coherence‐Enhancing Filtering , 2011, Comput. Graph. Forum.

[15]  Luiz Velho,et al.  Digital halftoning with space filling curves , 1991, SIGGRAPH.

[16]  David Salesin,et al.  Computer-generated watercolor , 1997, SIGGRAPH.

[17]  Tong-Yee Lee,et al.  Stylized and abstract painterly rendering system using a multiscale segmented sphere hierarchy , 2006, IEEE Transactions on Visualization and Computer Graphics.

[18]  Aaron Hertzmann,et al.  A survey of stroke-based rendering , 2003, IEEE Computer Graphics and Applications.

[19]  H. Lee,et al.  Motion based Painterly Rendering , 2009, Comput. Graph. Forum.

[20]  Adam Finkelstein,et al.  Interactive painterly stylization of images, videos and 3D animations , 2010, I3D '10.

[21]  Barbara J. Meier.  Painterly rendering for animation , 1996, SIGGRAPH.

[22]  Peter Shirley,et al.  Artistic Vision: painterly rendering using computer vision techniques , 2002, NPAR '02.

[23]  Christopher D. Kulla,et al.  Using texture synthesis for non-photorealistic shading from paint samples , 2003, 11th Pacific Conference onComputer Graphics and Applications, 2003. Proceedings..

[24]  Markus H. Gross,et al.  Interactive 3D Painting on Point-Sampled Objects , 2004, PBG.

[25]  Sangwon Lee,et al.  Interactive 3D fluid jet painting , 2006, SIGGRAPH Sketches.

[26]  Thomas Strothotte,et al.  RenderBots—Multi‐Agent Systems for Direct Image Generation , 2005, Comput. Graph. Forum.

[27]  Peter Shirley,et al.  Artistic rendering of mountainous terrain , 2009, TOGS.

[28]  Jintae Lee.  Simulating Oriental Black-Ink Painting , 1999, IEEE Computer Graphics and Applications.

[29]  Blair MacIntyre,et al.  Watercolor inspired non-photorealistic rendering for augmented reality , 2008, VRST '08.

[30]  Masayuki Nakajima,et al.  Volumetric modeling of colored pencil drawing , 1999, Proceedings. Seventh Pacific Conference on Computer Graphics and Applications (Cat. No.PR00293).

[31]  Brian Cabral,et al.  Imaging vector fields using line integral convolution , 1993, SIGGRAPH.

[32]  Adam Finkelstein,et al.  WYSIWYG NPR: drawing strokes directly on 3D models , 2002, SIGGRAPH.

[33]  Frédo Durand,et al.  An invitation to discuss computer depiction , 2002, NPAR '02.

[34]  Adrien Bousseau,et al.  Interactive watercolor rendering with temporal coherence and abstraction , 2006, NPAR.

[35]  金子 光枝,et al.  The Old Man and the Seaにおける文体 , 1972 .

[36]  William V. Baxter,et al.  DAB: Interactive Haptic Painting with 3D Virtual Brushes , 2001, SIGGRAPH Courses.

[37]  Der-Lor Way,et al.  The Synthesis of Trees in Chinese Landscape Painting Using Silhouette and Texture Strokes , 2002, WSCG.

[38]  Kyunghyun Yoon,et al.  Directional texture transfer , 2010, NPAR.

[39]  Benjamin Z. Yao,et al.  Introduction to a Large-Scale General Purpose Ground Truth Database: Methodology, Annotation Tool and Benchmarks , 2007, EMMCVPR.

[40]  Ralph R. Martin,et al.  Video-based running water animation in Chinese painting style , 2009, Science in China Series F: Information Sciences.

[41]  Bruce Gooch,et al.  Viewing progress in non-photorealistic rendering through Heinlein's lens , 2010, NPAR.

[42]  David Salesin,et al.  Video watercolorization using bidirectional texture advection , 2007, SIGGRAPH 2007.

[43]  Lee Markosian,et al.  Art-based rendering of fur, grass, and trees , 1999, SIGGRAPH.

[44]  John P. Collomosse,et al.  Stroke surfaces: temporally coherent artistic animations from video , 2005, IEEE Transactions on Visualization and Computer Graphics.

[45]  Song-Chun Zhu,et al.  Primal sketch: Integrating structure and texture , 2007, Comput. Vis. Image Underst..

[46]  Xiaoyang Mao,et al.  Colored pencil filter with custom colors , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[47]  Hui Xu,et al.  Stylized rendering of 3D scanned real world environments , 2004, NPAR '04.

[48]  Aaron Hertzmann,et al.  Paint by relaxation , 2001, Proceedings. Computer Graphics International 2001.

[49]  Song-Chun Zhu,et al.  From image parsing to painterly rendering , 2009, TOGS.

[50]  Jintae Lee,et al.  Diffusion rendering of black ink paintings using new paper and ink models , 2001, Comput. Graph..

[51]  John Lansdown,et al.  Expressive rendering: a review of nonphotorealistic techniques , 1995, IEEE Computer Graphics and Applications.

[52]  Douglas DeCarlo,et al.  Stylization and abstraction of photographs , 2002, ACM Trans. Graph..

[53]  Michio Shiraishi,et al.  An algorithm for automatic painterly rendering based on local source image approximation , 2000, NPAR '00.

[54]  Elaine Cohen,et al.  Interactive artistic rendering , 2000, NPAR '00.

[55]  Peter Litwinowicz,et al.  Processing images and video for an impressionist effect , 1997, SIGGRAPH.

[56]  Siu Chi Hsu,et al.  Skeletal strokes , 1993, UIST '93.

[57]  Elaine Cohen,et al.  A non-photorealistic lighting model for automatic technical illustration , 1998, SIGGRAPH.

[58]  Robert W. Sumner,et al.  OverCoat: an implicit canvas for 3D painting , 2011, ACM Trans. Graph..

[59]  Matthias Zwicker,et al.  Pointshop 3D: an interactive system for point-based surface editing , 2002, SIGGRAPH.

[60]  Oliver Deussen,et al.  Real-time watercolor illustrations of plants using a blurred depth test , 2006, NPAR.

[61]  Chung-Ren Yan,et al.  Stylized Rendering Using Samples of a Painted Image , 2008, IEEE Transactions on Visualization and Computer Graphics.

[62]  Hui Xu,et al.  PointWorks: Abstraction and Rendering of Sparsely Scanned Outdoor Environments , 2004, Rendering Techniques.

[63]  Aaron Hertzmann,et al.  Painterly rendering with curved brush strokes of multiple sizes , 1998, SIGGRAPH.

[64]  John Collomosse,et al.  Painterly rendering using image salience , 2002, Proceedings 20th Eurographics UK Conference.

[65]  Mario Costa Sousa,et al.  Observational Model of Blenders and Erasers in Computer-Generated Pencil Rendering , 1999, Graphics Interface.

[66]  Chiew-Lan Tai,et al.  MoXi: real-time ink dispersion in absorbent paper , 2005, SIGGRAPH '05.

[67]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[68]  Mario Costa Sousa,et al.  Observational Models of Graphite Pencil Materials , 2000, Comput. Graph. Forum.

[69]  Tobias Isenberg,et al.  Modeling with rendering primitives: an interactive non-photorealistic canvas , 2007, NPAR '07.

[70]  Ken Perlin,et al.  Painterly rendering for video and interaction , 2000, NPAR '00.

[71]  Nicolai Petkov,et al.  Artistic Edge and Corner Enhancing Smoothing , 2007, IEEE Transactions on Image Processing.