Disocclusion by Joint Interpolation of Vector Fields and Gray Levels

In this paper we study a variational approach for filling in regions of missing data in two-dimensional and three-dimensional digital images. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity, or the zooming of images. The approach presented here, initially introduced in [IEEE Trans. Image Process., 10 (2001), pp. 1200--1211] is based on a joint interpolation of the image gray levels and gradient/isophotes directions, smoothly extending the isophote lines into the holes of missing data. The process underlying this approach can be considered as an interpretation of the Gestaltist's principle of good continuation. We study the existence of minimizers of our functional and its approximation by minima of smoother functionals. Then we present the numerical algorithm used to minimize it and display some numerical experiments.

[1]  Jianhong Shen,et al.  EULER'S ELASTICA AND CURVATURE BASED INPAINTINGS , 2002 .

[2]  Jitendra Malik,et al.  Contour Continuity in Region Based Image Segmentation , 1998, ECCV.

[3]  Lance R. Williams,et al.  Analytic solution of stochastic completion fields , 1995, Biological Cybernetics.

[4]  Tony F. Chan,et al.  Non-texture inpainting by curvature-driven diffusions (CDD) , 2001 .

[5]  Simon Masnou,et al.  Disocclusion: a variational approach using level lines , 2002, IEEE Trans. Image Process..

[6]  V. Caselles,et al.  Minimizing total variation flow , 2000, Differential and Integral Equations.

[7]  Tony F. Chan,et al.  Euler's Elastica and Curvature-Based Inpainting , 2003, SIAM J. Appl. Math..

[8]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  K. Kunisch,et al.  An active set strategy based on the augmented Lagrangian formulation for image restoration , 1999 .

[10]  Gilberte Émile-Mâle.  The restorer's handbook of easel painting , 1976 .

[11]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[12]  Emanuele Paolini,et al.  Partial regularity for quasi minimizers of perimeter , 1999 .

[13]  S. Ullman,et al.  Filling-in the gaps: The shape of subjective contours and a model for their generation , 1976, Biological Cybernetics.

[14]  Lance R. Williams,et al.  Characterizing the distribution of completion shapes with corners using a mixture of random processes , 1997, Pattern Recognit..

[15]  Olivier Buisson,et al.  Detection and removal of line scratches in motion picture films , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[16]  Berthold K. P. Horn.  The Curve of Least Energy , 1983, TOMS.

[17]  J. Morel,et al.  Connected components of sets of finite perimeter and applications to image processing , 2001 .

[18]  Stanley Osher,et al.  Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing , 2003, J. Sci. Comput..

[19]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[20]  L. Evans.  Measure theory and fine properties of functions , 1992 .

[21]  Lance R. Williams,et al.  Local Parallel Computation of Stochastic Completion Fields , 1997, Neural Computation.

[22]  James R. Bergen,et al.  Pyramid-based texture analysis/synthesis , 1995, Proceedings., International Conference on Image Processing.

[23]  Umberto Massari,et al.  Frontiere orientate di curvatura media assegnata in $L^P$ , 1975 .

[24]  L. Ambrosio,et al.  A direct variational approach to a problem arising in image reconstruction , 2003 .

[25]  Lance R. Williams,et al.  Stochastic Completion Fields: A Neural Model of Illusory Contour Shape and Salience , 1997, Neural Computation.

[26]  R. Shapley,et al.  Spatial and Temporal Properties of Illusory Contours and Amodal Boundary Completion , 1996, Vision Research.

[27]  L. Ambrosio,et al.  Functions of Bounded Variation and Free Discontinuity Problems , 2000 .

[28]  Anil C. Kokaram,et al.  Detection of missing data in image sequences , 1995, IEEE Trans. Image Process..

[29]  T. Chan,et al.  Exact Solutions to Total Variation Regularization Problems , 1996 .

[30]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[31]  C. Ballester,et al.  The Dirichlet Problem for the Total Variation Flow , 2001 .

[32]  Jean-Michel Morel,et al.  Topographic Maps and Local Contrast Changes in Natural Images , 1999, International Journal of Computer Vision.

[33]  F. Guichard.  Image iterative smoothing and P.D.E.'s , 2000 .

[34]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[35]  Ronen Basri,et al.  Completion Energies and Scale , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Allen R. Hanson,et al.  Perceptual completion of occluded surfaces , 1996, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[37]  G. Sapiro,et al.  A variational model for filling-in gray level and color images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[38]  Robert V. Kohn,et al.  Dual spaces of stresses and strains, with applications to Hencky plasticity , 1983 .

[39]  Enrico Giusti.  Boundary value problems for non-parametric surfaces of prescribed mean curvature , 1976 .

[40]  J. Vázquez,et al.  An $L^1$-theory of existence and uniqueness of solutions of nonlinear elliptic equations , 1995 .

[41]  Fuensanta Andreu Vaíllo,et al.  Existence and uniqueness of a solution for a parabolic quasilinear problem for linear growth functionals with $L^1$ data , 2002 .

[42]  Maurizio Paolini,et al.  Semicontinuity and relaxation properties of a curvature depending functional in 2D , 1993 .

[43]  E. Giorgi,et al.  Un nuovo tipo di funzionale del calcolo delle variazioni , 1988 .

[44]  Luigi Ambrosio.  Corso introduttivo alla teoria geometrica della misura ed alle superfici minime , 1997 .

[45]  R. Dautray,et al.  Analyse mathématique et calcul numérique pour les sciences et les techniques , 1984 .

[46]  Anil C. Kokaram,et al.  Interpolation of missing data in image sequences , 1995, IEEE Trans. Image Process..

[47]  Roger Temam.  On the continuity of the trace of vector functions with bounded deformation , 1981 .

[48]  Eero P. Simoncelli,et al.  Texture characterization via joint statistics of wavelet coefficient magnitudes , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[49]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[50]  D. Mumford.  Elastica and Computer Vision , 1994 .

[51]  Simon Masnou.  Filtrage et désocclusion d'images par méthodes d'ensembles de niveau , 1998 .

[52]  François Malgouyres,et al.  Edge Direction Preserving Image Zooming: A Mathematical and Numerical Analysis , 2001, SIAM J. Numer. Anal..

[53]  T. Gallouët,et al.  Nonlinear Elliptic Equations with Right Hand Side Measures , 1992 .

[54]  Tony F. Chan,et al.  Mathematical Models for Local Nontexture Inpaintings , 2002, SIAM J. Appl. Math..

[55]  G. Anzellotti,et al.  Pairings between measures and bounded functions and compensated compactness , 1983 .

[56]  Alexei A. Efros,et al.  Texture synthesis by non-parametric sampling , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[57]  V. Caselles,et al.  A Parabolic Quasilinear Problem for Linear Growth Functionals , 2002 .

[58]  Jean Serra,et al.  Image Analysis and Mathematical Morphology , 1983 .

[59]  L. Modica,et al.  Gradient theory of phase transitions with boundary contact energy , 1987 .

[60]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[61]  Takashi Totsuka,et al.  Combining frequency and spatial domain information for fast interactive image noise removal , 1996, SIGGRAPH.

[62]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[63]  Philippe Salembier,et al.  Connected operators and pyramids , 1993, Optics & Photonics.