Simultaneous structure and texture image inpainting

An algorithm for the simultaneous filling-in of texture and structure in regions of missing image information is presented. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these functions separately with structure and texture filling-in algorithms. The first function used in the decomposition is of bounded variation, representing the underlying image structure, while the second function captures the texture and possible noise. The region of missing information in the bounded variation image is reconstructed using image inpainting algorithms, while the same region in the texture image is filled-in with texture synthesis techniques. The original image is then reconstructed adding back these two sub-images. The novel contribution of the paper is then in the combination of these three previously developed components: image decomposition with inpainting and texture synthesis, which permits the simultaneous use of filling-in algorithms that are suited for different image characteristics. Examples on real images show the advantages of this proposed approach.

[1]  Gilberte Émile-Mâle The restorer's handbook of easel painting , 1976 .

[2]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  L. Evans Measure theory and fine properties of functions , 1992 .

[4]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[5]  P. Lions,et al.  Image selective smoothing and edge detection by nonlinear diffusion. II , 1992 .

[6]  David Mumford,et al.  Filtering, Segmentation and Depth , 1993, Lecture Notes in Computer Science.

[7]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[8]  J. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[9]  Anil C. Kokaram,et al.  Detection of missing data in image sequences , 1995, IEEE Trans. Image Process..

[10]  Anil C. Kokaram,et al.  Interpolation of missing data in image sequences , 1995, IEEE Trans. Image Process..

[11]  Anil K. Jain,et al.  Is there any texture in the image? , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[12]  Anil K. Jain,et al.  Is there any texture in the image? , 1996, Pattern Recognit..

[13]  Takashi Totsuka,et al.  Combining frequency and spatial domain information for fast interactive image noise removal , 1996, SIGGRAPH.

[14]  Jeremy S. De Bonet,et al.  Multiresolution sampling procedure for analysis and synthesis of texture images , 1997, SIGGRAPH.

[15]  David King The Commissar vanishes : the falsification of photographs and art in Stalin's Russia : photographs and graphics from the David King collection , 1997 .

[16]  Eero P. Simoncelli,et al.  Texture characterization via joint statistics of wavelet coefficient magnitudes , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[17]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[18]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[19]  Olivier Buisson,et al.  Detection and removal of line scratches in motion picture films , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[20]  Stanley Osher,et al.  Explicit Algorithms for a New Time Dependent Model Based on Level Set Motion for Nonlinear Deblurring and Noise Removal , 2000, SIAM J. Sci. Comput..

[21]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[22]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[23]  Yves Meyer,et al.  Oscillating Patterns in Image Processing and Nonlinear Evolution Equations: The Fifteenth Dean Jacqueline B. Lewis Memorial Lectures , 2001 .

[24]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[25]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[26]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[27]  G. Sapiro,et al.  Growing Fitted Textures , 2001 .

[28]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[29]  Stanley Osher,et al.  Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing , 2003, J. Sci. Comput..

[30]  Guillermo Sapiro,et al.  Structure and texture filling-in of missing image blocks in wireless transmission and compression applications , 2003, IEEE Trans. Image Process..

[31]  Guillermo Sapiro,et al.  Inpainting surface holes , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).