Image quilting for texture synthesis and transfer

We present a simple image-based method of generating novel visual appearance in which a new image is synthesized by stitching together small patches of existing images. We call this process image quilting. First, we use quilting as a fast and very simple texture synthesis algorithm which produces surprisingly good results for a wide range of textures. Second, we extend the algorithm to perform texture transfer — rendering an object with a texture taken from a different object. More generally, we demonstrate how an image can be re-rendered in the style of a different image. The method works directly on the images and does not require 3D information.

[1]  R. Hetherington.  The Perception of the Visual World , 1952 .

[2]  Béla Julesz,et al.  Visual Pattern Discrimination , 1962, IRE Trans. Inf. Theory.

[3]  David Donovan Garber,et al.  Computational models for texture analysis and texture synthesis , 1981 .

[4]  E. Adelson,et al.  Early vision and texture perception , 1988, Nature.

[5]  P Perona,et al.  Preattentive texture discrimination with early vision mechanisms. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[6]  Kris Popat,et al.  Novel cluster-based probability model for texture synthesis, classification, and compression , 1993, Other Conferences.

[7]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[8]  Jeremy S. De Bonet,et al.  Multiresolution sampling procedure for analysis and synthesis of texture images , 1997, SIGGRAPH.

[9]  David Salesin,et al.  Computer-generated watercolor , 1997, SIGGRAPH.

[10]  David Salesin,et al.  Orientable textures for image-based pen-and-ink illustration , 1997, SIGGRAPH.

[11]  James Davis,et al.  Mosaics of scenes with moving objects , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[12]  Roger D. Hersch,et al.  Multi-color and artistic dithering , 1999, SIGGRAPH.

[13]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Adam Finkelstein,et al.  Lapped textures , 2000, SIGGRAPH.

[15]  Baining Guo,et al.  Chaos Mosaic: Fast and Memory Efficient Texture Synthesis , 2000 .

[16]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[17]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[18]  William T. Freeman,et al.  Quilting for Texture Synthesis and Transfer , 2001, SIGGRAPH 2001.

[19]  Baining Guo,et al.  Real-time texture synthesis by patch-based sampling , 2001, TOGS.

[20]  Paul Harrison,et al.  A Non-Hierarchical Procedure for Re-Synthesis of Complex Textures , 2001, WSCG.

[21]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.