Split and Match: Example-Based Adaptive Patch Sampling for Unsupervised Style Transfer

This paper presents a novel unsupervised method to transfer the style of an example image to a source image. The complex notion of image style is here considered as a local texture transfer, eventually coupled with a global color transfer. For the local texture transfer, we propose a new method based on an adaptive patch partition that captures the style of the example image and preserves the structure of the source image. More precisely, this example-based partition predicts how well a source patch matches an example patch. Results on various images show that our method outperforms the most recent techniques.

[1]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[2]  Frédo Durand,et al.  Data-driven hallucination of different times of day from a single outdoor photo , 2013, ACM Trans. Graph..

[3]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[4]  陈世峰.  Style Transfer via Image Component Analysis , 2013 .

[5]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[6]  Leon A. Gatys,et al.  A Neural Algorithm of Artistic Style , 2015, ArXiv.

[7]  Brendan J. Frey,et al.  Unsupervised image translation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[8]  Neus Sabater,et al.  Optimal Transportation for Example-Guided Color Transfer , 2014, ACCV.

[9]  Amit R.Sharma,et al.  Face Photo-Sketch Synthesis and Recognition , 2012 .

[10]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[12]  Frédo Durand,et al.  Style transfer for headshot portraits , 2014, ACM Trans. Graph..

[13]  A. Ms.PatilV..  Region Filling and Object Removal by Exemplar-Based Image Inpainting , 2012 .

[14]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[15]  R. Weale.  Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[16]  Shi-Min Hu,et al.  PatchTable: efficient patch queries for large datasets and applications , 2015, ACM Trans. Graph..

[17]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[18]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[19]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[20]  Frédo Durand,et al.  An invitation to discuss computer depiction , 2002, NPAR '02.

[21]  Xinhua Zhang,et al.  Consistent image analogies using semi-supervised learning , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Edward H. Adelson,et al.  Belief Propagation and Revision in Networks with Loops , 1997 .

[23]  Song-Chun Zhu,et al.  Towards a mathematical theory of primal sketch and sketchability , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  Song-Chun Zhu.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998 .

[25]  Pierre Bénard,et al.  Stylizing animation by example , 2013, ACM Trans. Graph..

[26]  Tobias Isenberg,et al.  State of the "Art”: A Taxonomy of Artistic Stylization Techniques for Images and Video , 2013, IEEE Transactions on Visualization and Computer Graphics.

[27]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[28]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .