Non-parametric Patch based Video Matting

In computer vision, matting is the process of accurate foreground estimation in images and videos. In this paper we presents a novel patch based approach to video matting relying on non-parametric statistics to represent image variations in appearance. This overcomes the limitation of parametric algorithms which only rely on strong colour correlation between the nearby pixels. Initially we construct a clean background by utilising the foreground object’s movement across the background. For a given frame, a trimap is constructed using the background and the last frame’s trimap. A patch-based approach is used to estimate the foreground colour for every unknown pixel and finally the alpha matte is extracted. Quantitative evaluation shows that the technique performs better, in terms of the accuracy and the required user interaction, than the current state-of-the-art parametric approaches.

[1]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[2]  Guillermo Sapiro,et al.  Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting , 2009, International Journal of Computer Vision.

[3]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[4]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Andrew W. Fitzgibbon,et al.  Image-Based Rendering Using Image-Based Priors , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.

[7]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[8]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..

[9]  John M. Hannah,et al.  Alpha channel estimation in high resolution images and image sequences , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[10]  Sung Yong Shin,et al.  On pixel-based texture synthesis by non-parametric sampling , 2006, Comput. Graph..

[11]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[12]  Michael F. Cohen,et al.  An iterative optimization approach for unified image segmentation and matting , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[13]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[14]  Jian Sun,et al.  Poisson matting , 2004, ACM Trans. Graph..

[15]  Jian Sun,et al.  Video object cut and paste , 2005, SIGGRAPH 2005.

[16]  Carlo Tomasi,et al.  Alpha estimation in natural images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[17]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.