Towards Temporally-Coherent Video Matting

Extracting temporally-coherent alpha mattes in video is an important but challenging problem in post-production. Previous video matting systems are highly sensitive to initial conditions and image noise, thus cannot reliably produce stable alpha mattes without temporal jitter. In this paper we propose an improved video matting system which contains two new components: (1) an accurate trimap propagation mechanism for setting up the initial matting conditions in a temporally-coherent way; and (2) a temporal matte filter which can improve the temporal coherence of the mattes while maintaining the matte structures on individual frames. Experimental results show that compared with previous methods, the two new components lead to alpha mattes with better temporal coherence.

[1]  Michael F. Cohen,et al.  Image and Video Matting: A Survey , 2007, Found. Trends Comput. Graph. Vis..

[2]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Paulo Villegas,et al.  Perceptually-weighted evaluation criteria for segmentation masks in video sequences , 2004, IEEE Transactions on Image Processing.

[4]  Yuanjie Zheng,et al.  Learning based digital matting , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[6]  Manuel Menezes de Oliveira Neto,et al.  Shared Sampling for Real‐Time Alpha Matting , 2010, Comput. Graph. Forum.

[7]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[8]  Jong-Chul Yoon,et al.  Temporally coherent video matting , 2010, Graph. Model..

[9]  Jian Sun,et al.  Video object cut and paste , 2005, SIGGRAPH 2005.

[10]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, ACM Trans. Graph..

[11]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[12]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[14]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .