Fragment-based image completion

We present a new method for completing missing parts caused by the removal of foreground or background elements from an image. Our goal is to synthesize a complete, visually plausible and coherent image. The visible parts of the image serve as a training set to infer the unknown parts. Our method iteratively approximates the unknown regions and composites adaptive image fragments into the image. Values of an inverse matte are used to compute a confidence map and a level set that direct an incremental traversal within the unknown area from high to low confidence. In each step, guided by a fast smooth approximation, an image fragment is selected from the most similar and frequent examples. As the selected fragments are composited, their likelihood increases along with the mean confidence of the image, until reaching a complete image. We demonstrate our method by completion of photographs and paintings.

[1]  O. Reiser,et al.  Principles Of Gestalt Psychology , 1936 .

[2]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.

[3]  Edward H. Adelson,et al.  Merging Images Through Pattern Decomposition , 1985, Optics & Photonics.

[4]  Paul Haeberli,et al.  Paint by numbers: abstract image representations , 1990, SIGGRAPH.

[5]  Gérard G. Medioni,et al.  Inferring global perceptual contours from local features , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Andrew S. Glassner,et al.  Proceedings of the 27th annual conference on Computer graphics and interactive techniques , 1994, SIGGRAPH 1994.

[7]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[8]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[9]  Takashi Totsuka,et al.  Combining frequency and spatial domain information for fast interactive image noise removal , 1996, SIGGRAPH.

[10]  Lance R. Williams,et al.  Stochastic Completion Fields: A Neural Model of Illusory Contour Shape and Salience , 1997, Neural Computation.

[11]  Lucas Pereira,et al.  Image replacement through texture synthesis , 1997, Proceedings of International Conference on Image Processing.

[12]  L. Pessoa,et al.  Finding out about filling-in: a guide to perceptual completion for visual science and the philosophy of perception. , 1998, The Behavioral and brain sciences.

[13]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[15]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[16]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[17]  Ronen Basri,et al.  Completion Energies and Scale , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Frédo Durand,et al.  A gentle introduction to bilateral filtering and its applications , 2007, SIGGRAPH Courses.

[19]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[20]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[21]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[22]  Tom Appolloni,et al.  Proceedings of the 29th annual conference on Computer graphics and interactive techniques , 2002, SIGGRAPH.

[23]  Shimon Ullman,et al.  Class-Specific, Top-Down Segmentation , 2002, ECCV.

[24]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[25]  Neil A. Dodgson,et al.  Self-similarity based texture editing , 2002, SIGGRAPH.

[26]  Klaus Mueller,et al.  Transferring color to greyscale images , 2002, ACM Trans. Graph..

[27]  Marie-Paule Cani,et al.  Hierarchical pattern mapping , 2002, ACM Trans. Graph..

[28]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[29]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[30]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[31]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.