Text-Video Completion Using Structure Repair and Texture Propagation

Today, more superimposed text is embedded within videos. Usually some text is unnecessary. Thus, one requires an approach to remove the text and complete the video. However, few conventional approaches complete the video well due to the large-sized text, structure regions, and various types of videos. In response, this study designed a text-video completion algorithm that poses text-video completion as structure repair and texture propagation. To repair the structure regions, the structure interpolation uses the new model's rotated block matching to estimate the initial location of completed regions and later refine the coordinates of completed regions. The information in the neighboring frames then fills the structure regions. To complete the structure regions without tedious manual interaction, the structure extension utilizes the spline curve estimation. Afterwards, derivative propagation realizes the texture region completion. The experiment results are based on several real TV programs, where all of the text regions were completed with spatio-temporal consistency. Additionally, comparisons present that the performance of the proposed algorithm is superior to those of conventional approaches. Its advantages include the reduction of design complexity by only integrating the structure information in multi-frame and the demonstration of structure consistency for realistic videos.

[1]  Tsung-Han Tsai,et al.  Advertisement video completion using hierarchical model , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[2]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Manuel Menezes de Oliveira Neto,et al.  Fast Digital Image Inpainting , 2001, VIIP.

[4]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[5]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[6]  Guillermo Sapiro,et al.  Video Inpainting Under Constrained Camera Motion , 2007, IEEE Transactions on Image Processing.

[7]  Tai-Pang Wu,et al.  Video repairing under variable illumination using cyclic motions , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Harry Shum,et al.  Image completion with structure propagation , 2005, ACM Trans. Graph..

[9]  Oscar C. Au,et al.  Video Error Concealment Using Spatio-Temporal Boundary Matching and Partial Differential Equation , 2008, IEEE Transactions on Multimedia.

[10]  Xiaochun Cao,et al.  Video Completion for Perspective Camera Under Constrained Motion , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  Jian Zhao,et al.  Efficient Object-Based Video Inpainting , 2006, 2006 International Conference on Image Processing.

[12]  Gang Pan,et al.  Structure-Aware Image Completion with Texture Propagation , 2011, 2011 Sixth International Conference on Image and Graphics.

[13]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part II: adaptive algorithms , 2006, IEEE Transactions on Image Processing.

[14]  Guillermo Sapiro,et al.  Filling-in by joint interpolation of vector fields and gray levels , 2001, IEEE Trans. Image Process..

[15]  H. J. Kim,et al.  Restoration of regions occluded by a caption in TV scene , 2003, TENCON 2003. Conference on Convergent Technologies for Asia-Pacific Region.

[16]  Ching-Tang Hsieh,et al.  Progressive Image Inpainting Based on Wavelet Transform , 2005, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[17]  Hang Joon Kim,et al.  Automatic text detection and removal in video sequences , 2003, Pattern Recognit. Lett..

[18]  Timothy K. Shih,et al.  Keynote 3: Digital Inpainting - Survey and Multilayer Image Inpainting Algorithms , 2005, ICITA.

[19]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  Onur G. Guleryuz,et al.  Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part I: theory , 2006, IEEE Transactions on Image Processing.

[21]  Tsung-Han Tsai,et al.  2DVTE: A two-directional videotext extractor for rapid and elaborate design , 2009, Pattern Recognit..

[22]  Nikos Komodakis,et al.  Image Completion Using Global Optimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[23]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[24]  Guillermo Sapiro,et al.  Projection based image and video inpainting using wavelets , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[25]  Eli Shechtman,et al.  Space-time video completion , 2004, CVPR 2004.

[26]  Tai-Yun Kim,et al.  A method for original image recovery for caption areas in video , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[27]  Hans-Peter Seidel,et al.  Image restoration using multiresolution texture synthesis and image inpainting , 2003, Proceedings Computer Graphics International 2003.

[28]  Yasuyuki Matsushita,et al.  Video Completion by Motion Field Transfer , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[29]  Tsung-Han Tsai,et al.  Progressive Videotext Regions Inpainting Based on Edge Detection and Statistic Method , 2006, IMECS.

[30]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.