Video matting via opacity propagation

In this paper, a novel video matting method based on opacity propagation is proposed. It adopts a two-step framework by first segmenting each frame into bilayer and generating trimaps, and then applying matting algorithm to extract the final alpha mattes. In the bilayer segmentation step, an opacity propagation algorithm is used to predict the foreground object in the following frame. It propagates the opacity values of the former frame to the current, and generates an Opacity Map (OM), which is in most cases much more accurate than the previously used Probability Map (PM). OM is used to extract the foreground object by combining it with the graph cut algorithm. And then an accurate trimap is generated by a trimap refinement method based on a Local Gaussian Mixture Model (LGMM). At last, the opacity propagation is again applied to the unknown regions of the trimap to generate the accurate alpha mattes in a temporally coherent way. The experiment results demonstrate the effectiveness of our method.

[1]  Baozong Yuan,et al.  Better Foreground Segmentation for Static Cameras via New Energy Form and Dynamic Graph-cut , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[2]  Rüdiger Westermann,et al.  RANDOM WALKS FOR INTERACTIVE ALPHA-MATTING , 2005 .

[3]  Chang-Su Kim,et al.  Real-time segmentation of objects from video sequences with non-stationary backgrounds using spatio-temporal coherence , 2008, 2008 15th IEEE International Conference on Image Processing.

[4]  Michael F. Cohen,et al.  An iterative optimization approach for unified image segmentation and matting , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[5]  David Corrigan,et al.  Video Matting Using Motion Extended GrabCut , 2008 .

[6]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[7]  Takeshi Naemura,et al.  Real-Time Video Matting Based on Bilayer Segmentation , 2009, ACCV.

[8]  Guillermo Sapiro,et al.  Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting , 2009, International Journal of Computer Vision.

[9]  Scott Cohen,et al.  Simultaneous foreground, background, and alpha estimation for image matting , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[11]  Jong-Chul Yoon,et al.  Temporally coherent video matting , 2010, Graph. Model..

[12]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[13]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[14]  Pushmeet Kohli,et al.  A spatially varying PSF-based prior for alpha matting , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Vladimir Vezhnevets,et al.  Improvements of Bayesian Matting , 2007 .

[16]  Ying Wu,et al.  L1 matting , 2010, 2010 IEEE International Conference on Image Processing.

[17]  Xiangyang Xue,et al.  Quick matting: A matting method based on pixel spread and propagation , 2010, 2010 IEEE International Conference on Image Processing.

[18]  Abhinav Gupta,et al.  A novel approach to video matting using automated scribbling by motion analysis , 2008, 2008 IEEE Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems.

[19]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[20]  Renaud Keriven,et al.  Trimap Segmentation for Fast and User-Friendly Alpha Matting , 2005, VLSM.

[21]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[22]  Zhenjiang Miao,et al.  Temporally consistent video matting based on bilayer segmentation , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[23]  Toby Sharp,et al.  High resolution matting via interactive trimap segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Jian Sun,et al.  Fast matting using large kernel matting Laplacian matrices , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Michael F. Cohen,et al.  Image and Video Matting: A Survey , 2007, Found. Trends Comput. Graph. Vis..

[26]  Michael Cohen,et al.  Soft scissors: an interactive tool for realtime high quality matting , 2007, SIGGRAPH 2007.

[27]  Jian Sun,et al.  Video object cut and paste , 2005, SIGGRAPH 2005.

[28]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, ACM Trans. Graph..

[29]  Harry Shum,et al.  Pop-up light field: An interactive image-based modeling and rendering system , 2004, TOGS.

[30]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .