Texture optimization for example-based synthesis

We present a novel technique for texture synthesis using optimization. We define a Markov Random Field (MRF)-based similarity metric for measuring the quality of synthesized texture with respect to a given input sample. This allows us to formulate the synthesis problem as minimization of an energy function, which is optimized using an Expectation Maximization (EM)-like algorithm. In contrast to most example-based techniques that do region-growing, ours is a joint optimization approach that progressively refines the entire texture. Additionally, our approach is ideally suited to allow for controllable synthesis of textures. Specifically, we demonstrate controllability by animating image textures using flow fields. We allow for general two-dimensional flow fields that may dynamically change over time. Applications of this technique include dynamic texturing of fluid animations and texture-based flow visualization.

[1]  S. C. Johnson.  Hierarchical clustering schemes , 1967, Psychometrika.

[2]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[3]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Rupert Paget,et al.  Texture synthesis via a noncausal nonparametric multiscale Markov random field , 1998, IEEE Trans. Image Process..

[5]  Jeremy S. De Bonet,et al.  Multiresolution sampling procedure for analysis and synthesis of texture images , 1997, SIGGRAPH.

[6]  Konstantin Mischaikow,et al.  Vector field design on surfaces , 2006, TOGS.

[7]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[8]  Charles Elkan,et al.  Using the Triangle Inequality to Accelerate k-Means , 2003, ICML.

[9]  Tomaso Poggio,et al.  Trainable Videorealistic Speech Animation , 2004, FGR.

[10]  Yizhou Yu,et al.  Feature matching and deformation for texture synthesis , 2004, SIGGRAPH 2004.

[11]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[12]  Baining Guo,et al.  Real-time texture synthesis by patch-based sampling , 2001, TOGS.

[13]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[14]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[15]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[16]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[17]  Kun Zhou,et al.  Synthesis of progressively-variant textures on arbitrary surfaces , 2003, ACM Trans. Graph..

[18]  Stefano Soatto,et al.  Editable dynamic textures , 2002, SIGGRAPH '02.

[19]  Christoph Bregler,et al.  Video Rewrite: Driving Visual Speech with Audio , 1997, SIGGRAPH.

[20]  Jessica K. Hodgins,et al.  Flow-based video synthesis and editing , 2004, SIGGRAPH 2004.

[21]  Andrew W. Fitzgibbon,et al.  Image-Based Rendering Using Image-Based Priors , 2005, International Journal of Computer Vision.

[22]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[23]  Frank Dellaert,et al.  Mixture trees for modeling and fast conditional sampling with applications in vision and graphics , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Eli Shechtman,et al.  Space-time video completion , 2004, CVPR 2004.

[25]  Fabrice Neyret,et al.  Advected textures , 2003, SCA '03.

[26]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[27]  Oliver Deussen,et al.  Wang Tiles for image and texture generation , 2003, ACM Trans. Graph..

[28]  Brendan J. Frey,et al.  Epitomic analysis of appearance and shape , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[29]  Eero P. Simoncelli,et al.  A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients , 2000, International Journal of Computer Vision.