Stroke-based stylization by learning sequential drawing examples

Abstract Among various traditional art forms, brush stroke drawing is one of the widely used styles in modern computer graphic tools such as GIMP, Photoshop and Painter. In this paper, we develop an AI-aided art authoring ( A 4 ) system of non-photorealistic rendering that allows users to automatically generate brush stroke paintings in a specific artist’s style. Within the reinforcement learning framework of brush stroke generation proposed by Xie et al. (2012), the first contribution in this paper is the application of regularized policy gradient method, which is more suitable for the stroke generation task; the other contribution is to learn artists’ drawing styles from video-captured stroke data by inverse reinforcement learning. Through experiments, we demonstrate that our system can successfully learn artists’ styles and render pictures with consistent and smooth brush strokes.

[1]  Seungyong Lee,et al.  Shape‐simplifying Image Abstraction , 2008, Comput. Graph. Forum.

[2]  Levent Burak Kara,et al.  Beautification of Design Sketches Using Trainable Stroke Clustering and Curve Fitting , 2011, IEEE Transactions on Visualization and Computer Graphics.

[3]  Derek Nowrouzezahrai,et al.  Learning hatching for pen-and-ink illustration of surfaces , 2012, TOGS.

[4]  Ligang Liu,et al.  Animated construction of line drawings , 2011, ACM Trans. Graph..

[5]  I. Jolliffe.  Principal Component Analysis , 2002 .

[6]  Frank Sehnke,et al.  Parameter-exploring policy gradients , 2010, Neural Networks.

[7]  Aaron Hertzmann,et al.  A survey of stroke-based rendering , 2003, IEEE Computer Graphics and Applications.

[8]  E. R. Davies,et al.  Machine vision - theory, algorithms, practicalities , 2004 .

[9]  Masayuki Nakajima,et al.  Contour-driven Sumi-e rendering of real photos , 2011, Comput. Graph..

[10]  Stefan Schaal,et al.  Natural Actor-Critic , 2003, Neurocomputing.

[11]  Jaakko Lehtinen,et al.  Sketching Clothoid Splines Using Shortest Paths , 2010, Comput. Graph. Forum.

[12]  Gang Niu,et al.  Analysis and Improvement of Policy Gradient Estimation , 2011, NIPS.

[13]  Chiew-Lan Tai,et al.  MoXi: real-time ink dispersion in absorbent paper , 2005, SIGGRAPH '05.

[14]  Jürgen Döllner,et al.  Image and Video Abstraction by Anisotropic Kuwahara Filtering , 2009, Comput. Graph. Forum.

[15]  William V. Baxter,et al.  Detail-preserving paint modeling for 3D brushes , 2010, NPAR.

[16]  Stephen DiVerdi,et al.  HelpingHand: example-based stroke stylization , 2012, ACM Trans. Graph..

[17]  Jirí Zára,et al.  Colorization of black-and-white cartoons , 2005, Image Vis. Comput..

[18]  Song-Chun Zhu,et al.  From image parsing to painterly rendering , 2009, TOGS.

[19]  Jun Morimoto,et al.  Efficient Sample Reuse in Policy Gradients with Parameter-Based Exploration , 2012, Neural Computation.

[20]  Jan Peters,et al.  A Survey on Policy Search for Robotics , 2013, Found. Trends Robotics.

[21]  David Salesin,et al.  Animating Chinese paintings through stroke-based decomposition , 2006, TOGS.

[22]  David Salesin,et al.  Video watercolorization using bidirectional texture advection , 2007, SIGGRAPH 2007.

[23]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[24]  Theodosios Pavlidis,et al.  An automatic beautifier for drawings and illustrations , 1985, SIGGRAPH.

[25]  Peter L. Bartlett,et al.  Variance Reduction Techniques for Gradient Estimates in Reinforcement Learning , 2001, J. Mach. Learn. Res..

[26]  Nelson Siu-Hang Chu,et al.  Real-time painting with an expressive virtual Chinese brush , 2004, IEEE Computer Graphics and Applications.

[27]  Hao Jiang,et al.  Automatic Facsimile of Chinese Calligraphic Writings , 2008, Comput. Graph. Forum.

[28]  C. Lawrence Zitnick,et al.  Handwriting beautification using token means , 2013, ACM Trans. Graph..