Texture Transfer Based on Energy Minimization for Painterly Rendering

Non-photorealistic rendering (NPR) creates images with artistic styles of paintings. In this field, a number of methods of converting photographed images into non-photorealistic ones have been developed, and can be categorized into filter-based and exemplar-based approaches. In this paper, we focus on the exemplar-based approach and propose a novel method which transfers a style of a reference pictorial image to a photographed image. Specifically, we first input a pair of target and reference images. The target image is converted by minimizing an energy function which is defined based on the difference in intensities between an output image and a target image, and the pattern dissimilarity between an output image and a reference image. The proposed method transfers structures and colors of textures in the reference image and generates continuous textures by minimizing the energy function. In experiments, we demonstrate the effectiveness of the proposed method using a variety of images and examine the influence of parameter changes and intensity adjustment for pre-processing on resultant images.

[1]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[2]  Kyunghyun Yoon,et al.  Directional texture transfer with edge enhancement , 2011, Comput. Graph..

[3]  Levente Kovács,et al.  Painterly rendering controlled by multiscale image features , 2004, SCCG '04.

[4]  P. Kay,et al.  Basic Color Terms: Their Universality and Evolution , 1973 .

[5]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[6]  Michael Ashikhmin,et al.  Fast Texture Transfer , 2003, IEEE Computer Graphics and Applications.

[7]  Zhuowen Tu,et al.  Parsing Images into Regions, Curves, and Curve Groups , 2006, International Journal of Computer Vision.

[8]  Seungyong Lee,et al.  Coherent line drawing , 2007, NPAR '07.

[9]  Jing Fan,et al.  An improved image analogy method based on adaptive CUDA-accelerated neighborhood matching framework , 2012, The Visual Computer.

[10]  Tobias Isenberg,et al.  State of the "Art”: A Taxonomy of Artistic Stylization Techniques for Images and Video , 2013, IEEE Transactions on Visualization and Computer Graphics.

[11]  James Davis,et al.  Mosaics of scenes with moving objects , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[12]  Zhuowen Tu,et al.  Image Parsing: Unifying Segmentation, Detection, and Recognition , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[13]  Eli Shechtman,et al.  Image melding , 2012, ACM Trans. Graph..

[14]  Masayuki Nakajima,et al.  Example-Based Color Stylization of Images , 2005, TAP.

[15]  Paul Haeberli,et al.  Paint by numbers: abstract image representations , 1990, SIGGRAPH.

[16]  Song-Chun Zhu,et al.  From image parsing to painterly rendering , 2009, TOGS.

[17]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[18]  P. Kay Basic Color Terms: Their Universality and Evolution , 1969 .

[19]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[20]  Jia-Guang Sun,et al.  Efficient example-based painting and synthesis of 2D directional texture , 2004, IEEE Transactions on Visualization and Computer Graphics.

[21]  Aaron Hertzmann,et al.  Paint by relaxation , 2001, Proceedings. Computer Graphics International 2001.

[22]  Aaron Hertzmann,et al.  Painterly rendering with curved brush strokes of multiple sizes , 1998, SIGGRAPH.

[23]  Song-Chun Zhu,et al.  Primal sketch: Integrating structure and texture , 2007, Comput. Vis. Image Underst..