Automatic Image Inpainting by Heuristic Texture and Structure Completion

This paper studies an image inpainting solution based on a primal sketch representation model [1], which divides an image into structure (sketchable) and texture (non-sketchable) components. This solution first predicts the missing structures, such as curves and corners, using a tensor voting algorithm [2]. Then the texture parts along structural sketches are synthesized with the patches sampled from the known regions, and the remaining texture parts are defused using a graph cuts algorithm [3]. Compared to the state-of-art image inpainting approaches, the characteristics of this solution include: 1) using the primal sketch representation model to guide completion for visual consistency; 2) achieving fully automatic completion. Finally, the experiments on the public datasets show above characteristics.

[1]  Song-Chun Zhu,et al.  Primal Sketch: Integrating Texture and Structure , 2011 .

[2]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[3]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[4]  W. Freeman,et al.  Generalized Belief Propagation , 2000, NIPS.

[5]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[6]  Gui-Song Xia,et al.  Compositional Boosting for Computing Hierarchical Image Structures , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[9]  Alexei A. Efros,et al.  Scene completion using millions of photographs , 2007, SIGGRAPH 2007.

[10]  Mi-Suen Lee,et al.  A Computational Framework for Segmentation and Grouping , 2000 .

[11]  H. Barlow Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi + 397 , 1983 .

[12]  Song-Chun Zhu,et al.  Equivalence of Julesz Ensembles and FRAME Models , 2000, International Journal of Computer Vision.

[13]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[14]  Song-Chun Zhu,et al.  Primal sketch: Integrating structure and texture , 2007, Comput. Vis. Image Underst..