Temporally coherent video matting

Existing video matting approaches determine the alpha matte sequence frame-by-frame, which lead to flickering near the boundary of the foreground region. We reduce this effect by considering video data as a spatio-temporal cube, and extending a robust matting algorithm to a 3D solver. Our results demonstrate consistent and visually pleasing alpha mattes, and tend to preserve temporal coherence better than previous techniques.

[1]  Wojciech Matusik,et al.  Natural video matting using camera arrays , 2006, SIGGRAPH '06.

[2]  Tomoyuki Nishita,et al.  Extracting depth and matte using a color-filtered aperture , 2008, SIGGRAPH Asia '08.

[3]  Bo Thiesson,et al.  Image and Video Segmentation by Anisotropic Kernel Mean Shift , 2004, ECCV.

[4]  Harry Shum,et al.  Pop-up light field: An interactive image-based modeling and rendering system , 2004, TOGS.

[5]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Qingxiong Yang,et al.  Automatic Natural Video Matting with Depth , 2007 .

[7]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[8]  Carlo Tomasi,et al.  Alpha estimation in natural images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, ACM Trans. Graph..

[10]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[11]  Michael F. Cohen,et al.  Image and Video Matting: A Survey , 2007, Found. Trends Comput. Graph. Vis..

[12]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[14]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[15]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[16]  Frédo Durand,et al.  Defocus video matting , 2005, SIGGRAPH 2005.

[17]  Guillermo Sapiro,et al.  A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[18]  John M. Hannah,et al.  Alpha channel estimation in high resolution images and image sequences , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[19]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[20]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[21]  Robert T. Collins,et al.  Shape constrained figure-ground segmentation and tracking , 2009, CVPR.

[22]  Andrew Fitzgibbon,et al.  Bayesian video matting using learnt image priors , 2004, CVPR 2004.

[23]  Paulo Villegas,et al.  Perceptually-weighted evaluation criteria for segmentation masks in video sequences , 2004, IEEE Transactions on Image Processing.

[24]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .