A variational model for filling-in gray level and color images

A variational approach for filling-in regions of missing data in gray-level and color images is introduced in this paper. The approach is based on joint interpolation of the image gray-levels and gradient/isophores directions, smoothly extending in an automatic fashion the isophote lines into the holes of missing data. This interpolation is computed solving the variational problem via its gradient descent flow, which lends to a set of coupled second order partial differential equations, one for the gray-levels and one for the gradient orientations. The process underlying this approach can be considered as an interpretation of the Gestaltist's principle of good continuation. No limitations are imposed on the topology of the holes, and all regions of missing data can be simultaneously processed, even if they are surrounded by completely different structures. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity. Examples of these applications are given.

[1]  Gilberte Émile-Mâle.  The restorer's handbook of easel painting , 1976 .

[2]  G. Anzellotti,et al.  Pairings between measures and bounded functions and compensated compactness , 1983 .

[3]  L. Evans.  Measure theory and fine properties of functions , 1992 .

[4]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[5]  Maurizio Paolini,et al.  Semicontinuity and relaxation properties of a curvature depending functional in 2D , 1993 .

[6]  David Mumford,et al.  Filtering, Segmentation and Depth , 1993, Lecture Notes in Computer Science.

[7]  Hans Knutsson,et al.  Signal processing for computer vision , 1994 .

[8]  J. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, SIGGRAPH.

[9]  Anil C. Kokaram,et al.  Interpolation of missing data in image sequences , 1995, IEEE Trans. Image Process..

[10]  Sanjit K. Mitra,et al.  A simple algorithm for restoration of images corrupted by streaks , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[11]  Takashi Totsuka,et al.  Combining frequency and spatial domain information for fast interactive image noise removal , 1996, SIGGRAPH.

[12]  David King.  The Commissar vanishes : the falsification of photographs and art in Stalin's Russia : photographs and graphics from the David King collection , 1997 .

[13]  Eero P. Simoncelli,et al.  Texture characterization via joint statistics of wavelet coefficient magnitudes , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[14]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[15]  Song-Chun Zhu.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998 .

[16]  Simon Masnou.  Filtrage et désocclusion d'images par méthodes d'ensembles de niveau , 1998 .

[17]  Song-Chun Zhu,et al.  GRADE: Gibbs reaction and diffusion equations , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[18]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[19]  Olivier Buisson,et al.  Detection and removal of line scratches in motion picture films , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[20]  Guillermo Sapiro,et al.  On the level lines and geometry of vector-valued images , 2000, IEEE Signal Processing Letters.

[21]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[22]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[23]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[24]  Tony F. Chan,et al.  Non-texture inpainting by curvature-driven diffusions (CDD) , 2001 .