VIDEO COMPLETION USING H IERARCHICAL M OTION ESTIMATION AND COLOR COMPENSATION

Video completion is used for repairing damaged or missing data in a video sequence. This paper proposes a video completion method usinghierarchical motion estimation and color compensation.The proposed method consists of three steps:hierarchical motion estimation, source patch search, and colorassignment. If an input video contains brightness change, existing video completion methods give poor res ults. For overcoming this drawback, we apply two methods: color normalization for motion estimation and source patch search, and color compensation for color assignment.First of all, we construct spatio-temporal Gaussian pyramid for reducing the computati on time. We estimate the motion of the input video using color information of each pyramid level.The proposed method uses normalized RGB color in motion estimation . Then, we search the source patch using normalized color information and motion information extracted in the first step. Finally, color compensation using color transform is used for color assignment. Experimental results show that the proposed method gives better visual results than conventional video completion methods.

[1]  Timothy K. Shih,et al.  Spatial-Temporal Method for Image Restoration in Aged Motion Pictures , 2007, 21st International Conference on Advanced Information Networking and Applications (AINA '07).

[2]  Tsung-Han Tsai,et al.  Advertisement video completion using hierarchical model , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[3]  Brendan J. Frey,et al.  Video Epitomes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  S. Kamel,et al.  Moving object removal in video sequence and background restoration using kalman filter , 2008, 2008 International Symposium on Telecommunications.

[5]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[7]  Rong-Chi Chang,et al.  Application of inpainting technology to video restoration , 2008, 2008 First IEEE International Conference on Ubi-Media Computing.

[8]  Jin-Hong Kim,et al.  Video completion using robust motion estimation and color compensation , 2011, 2011 IEEE International Conference on Consumer Electronics (ICCE).

[9]  Timothy K. Shih,et al.  Video falsifying by motion interpolation and inpainting , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Nam Ik Cho,et al.  Panorama Mosaic Optimization for Mobile Camera Systems , 2007, IEEE Transactions on Consumer Electronics.

[11]  Baoxin Li,et al.  Video Inpainting for Largely Occluded Moving Human , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[12]  Rae-Hong Park,et al.  Red-eye detection and correction using inpainting in digital photographs , 2009, IEEE Transactions on Consumer Electronics.

[13]  Assaf Zomet,et al.  Learning how to inpaint from global image statistics , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[14]  Xiaochun Cao,et al.  Video Completion for Perspective Camera Under Constrained Motion , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[15]  Sang Uk Lee,et al.  An object inpainting algorithm for multi-view video sequences , 2008, 2008 15th IEEE International Conference on Image Processing.

[16]  Eli Shechtman,et al.  Space-time video completion , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[17]  Guillermo Sapiro,et al.  Video Inpainting Under Constrained Camera Motion , 2007, IEEE Transactions on Image Processing.

[18]  Gui Yun Tian,et al.  Colour correction for panoramic imaging , 2002, Proceedings Sixth International Conference on Information Visualisation.

[19]  Polina Golland,et al.  Motion from Color , 1997, Comput. Vis. Image Underst..

[20]  Yingen Xiong,et al.  Fast panorama stitching for high-quality panoramic images on mobile phones , 2010, IEEE Transactions on Consumer Electronics.

[21]  Tao Ding,et al.  A Rank Minimization Approach to Video Inpainting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[22]  Tai-Pang Wu,et al.  Video repairing under variable illumination using cyclic motions , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Yasuyuki Matsushita,et al.  Video Completion by Motion Field Transfer , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).