Spectral Video Matting

We present a new, simple-to-use and rapid approach to video matting, the process of pulling a highquality alpha matte from a video sequence. Our approach builds upon techniques in natural image matting, namely spectral matting, and optical flow computation. No additional hardware, despite a single camera, is needed, and only very few and intuitive user interactions are required for foreground estimation. For certain scenes the approach is able to estimate the alpha matte for a video, consisting of up to 102 frames, without any user interaction at all.

[1]  Xinhua Zhuang,et al.  Motion-partitioned adaptive block matching for video compression , 1995, Proceedings., International Conference on Image Processing.

[2]  Michael F. Cohen,et al.  Image and Video Matting , 2008 .

[3]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[4]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..

[5]  Michael Cohen,et al.  Soft scissors: an interactive tool for realtime high quality matting , 2007, SIGGRAPH 2007.

[6]  Jian Sun,et al.  Video object cut and paste , 2005, SIGGRAPH 2005.

[7]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, ACM Trans. Graph..

[8]  Gary R. Bradski,et al.  Learning OpenCV - computer vision with the OpenCV library: software that sees , 2008 .

[9]  Yair Weiss,et al.  Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Wei Chen,et al.  Easy Matting ‐ A Stroke Based Approach for Continuous Image Matting , 2006, Comput. Graph. Forum.

[11]  Dani Lischinski,et al.  Spectral Matting , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[13]  Michael F. Cohen,et al.  Monocular Video Foreground/Background Segmentation by Tracking Spatial-Color Gaussian Mixture Models , 2007, 2007 IEEE Workshop on Motion and Video Computing (WMVC'07).

[14]  Michael F. Cohen,et al.  Image and Video Matting: A Survey , 2007, Found. Trends Comput. Graph. Vis..

[15]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[16]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[17]  Rüdiger Westermann,et al.  RANDOM WALKS FOR INTERACTIVE ALPHA-MATTING , 2005 .

[18]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[19]  Kevin J. Lang.  Fixing two weaknesses of the Spectral Method , 2005, NIPS.

[20]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[21]  Michael F. Cohen,et al.  An iterative optimization approach for unified image segmentation and matting , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[22]  William A. Barrett,et al.  Interactive segmentation of image volumes with Live Surface , 2007, Comput. Graph..

[23]  Michael Isard,et al.  Active Contours: The Application of Techniques from Graphics, Vision, Control Theory and Statistics to Visual Tracking of Shapes in Motion , 2000 .

[24]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.

[25]  Jianbo Shi,et al.  Multiclass spectral clustering , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Michael Gleicher,et al.  This document was created with FrameMaker 4.0.4 Image Snapping , 2022 .

[27]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.