Alternate Structural-Textural Video Inpainting for Spot Defects Correction in Movies

We propose a new video inpainting model for movies restoration application. Our model combines structural reconstruction with a diffusion-based method and textural reconstruction with a patch-based method. Both proposed energies (one for each method) are alternatively minimized in order to preserve the overall structure while adding textural refinement. While the structural reconstruction is obtained jointly with optical flow computation with several proximal approaches, the textural reconstruction is processed by a variational non-local approach. Preliminary results on different Middlebury frames show quality improvement in the reconstruction.

[1]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[2]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[3]  Patrick Pérez,et al.  Geometrically Guided Exemplar-Based Inpainting , 2011, SIAM J. Imaging Sci..

[4]  V. Do G. Lebrun,et al.  Inpainting d’Images Couleurs par Lissage Anisotrope et Synthèse de Textures Color Image Inpainting by Anisotropic Smoothing and Texture Synthesis , 2005 .

[5]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[6]  Patrick Pérez,et al.  Video Inpainting of Complex Scenes , 2014, SIAM J. Imaging Sci..

[7]  David Tschumperlé,et al.  Exemplar-Based Inpainting: Technical Review and New Heuristics for Better Geometric Reconstructions , 2015, IEEE Transactions on Image Processing.

[8]  Jean-Michel Morel,et al.  Level lines based disocclusion , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[9]  Jose Luis Lisani,et al.  Video Denoising with Optical Flow Estimation , 2018, Image Process. Line.

[10]  Yann Gousseau,et al.  Motion-consistent video inpainting , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[11]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[12]  Carola-Bibiane Schönlieb,et al.  A Variational Model for Joint Motion Estimation and Image Reconstruction , 2016, SIAM J. Imaging Sci..

[13]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[14]  Mads Nielsen,et al.  A Variational Algorithm For Motion Compensated Inpainting , 2004, BMVC.

[15]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[16]  Rachid Deriche,et al.  Computing Optical Flow via Variational Techniques , 1999, SIAM J. Appl. Math..

[17]  Aurélie Bugeau,et al.  Combining Texture Synthesis and Diffusion for Image Inpainting , 2009, VISAPP.

[18]  Yann Gousseau,et al.  Demonstration abstract: Motion-consistent video inpainting , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[19]  Jacques Blanc-Talon,et al.  Simultaneous Inpainting and Motion Estimation of Highly Degraded Video-Sequences , 2003, SCIA.

[20]  Guillermo Sapiro,et al.  A Variational Framework for Exemplar-Based Image Inpainting , 2011, International Journal of Computer Vision.

[21]  R. March,et al.  Analysis of a variational model for motion compensated inpainting , 2017 .

[22]  Mads Nielsen,et al.  On Variational Methods for Motion Compensated Inpainting , 2018, ArXiv.

[23]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..