Image inpainting by global structure and texture propagation

Image inpainting is a technique to repair damaged images or modify images in a non-detectable form. In this paper, a novel global algorithm for region filling is proposed for image inpainting. After removing objects from an image, our approach fills the regions using patches taken from the image. The filling process is formulated as an energy minimization problem by Markov random fields (MRFs) and the belief propagation (BP) is utilized to solve the problem. Our energy function includes structure and texture information obtained from the image. One challenge in using BP is that its computational complexity is the square of the number of label candidates. To reduce the large number of label candidates, we present a coarse-to-fine scheme where two BPs run with much smaller numbers of label candidates instead of one BP running with a large number of label candidates. Experimental results demonstrate the excellent performance of our algorithm over other related algorithms.

[1]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[2]  William A. Barrett,et al.  Object-based image editing , 2002, ACM Trans. Graph..

[3]  Paul Harrison,et al.  A Non-Hierarchical Procedure for Re-Synthesis of Complex Textures , 2001, WSCG.

[4]  Daniel Cohen-Or,et al.  Fragment-based image completion , 2003, ACM Trans. Graph..

[5]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[6]  Nikos Komodakis,et al.  Image Completion Using Global Optimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[7]  Chi-Keung Tang,et al.  Image repairing: robust image synthesis by adaptive ND tensor voting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[8]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[9]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[10]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[11]  Michael Ashikhmin,et al.  Synthesizing natural textures , 2001, I3D '01.

[12]  Tony F. Chan,et al.  Non-texture inpainting by curvature-driven diffusions (CDD) , 2001 .

[13]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  Louis Laborelli,et al.  Missing data correction in still images and image sequences , 2002, MULTIMEDIA '02.

[15]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[16]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[17]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.