Interactive video cutout

We present an interactive system for efficiently extracting foreground objects from a video. We extend previous min-cut based image segmentation techniques to the domain of video with four new contributions. We provide a novel painting-based user interface that allows users to easily indicate the foreground object across space and time. We introduce a hierarchical mean-shift preprocess in order to minimize the number of nodes that min-cut must operate on. Within the min-cut we also define new local cost functions to augment the global costs defined in earlier work. Finally, we extend 2D alpha matting methods designed for images to work with 3D video volumes. We demonstrate that our matting approach preserves smoothness across both space and time. Our interactive video cutout system allows users to quickly extract foreground objects from video sequences for use in a variety of applications including compositing onto new backgrounds and NPR cartoon style rendering.

[1]  Leonard McMillan,et al.  Proscenium: a framework for spatio-temporal video editing , 2003, ACM Multimedia.

[2]  D. Comaniciu,et al.  The variable bandwidth mean shift and data-driven scale selection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[3]  Carlo Tomasi,et al.  Alpha estimation in natural images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[5]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[6]  Michael Gleicher,et al.  This document was created with FrameMaker 4.0.4 Image Snapping , 2022 .

[7]  David Salesin,et al.  Interactive digital photomontage , 2004, SIGGRAPH 2004.

[8]  Kenji Mase,et al.  Interactive video cubism , 1999, NPIVM '99.

[9]  Adam Finkelstein,et al.  Stylized video cubes , 2002, SCA '02.

[10]  Huitao Luo,et al.  Spatial temporal active contour interpolation for semi-automatic video object generation , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[11]  William A. Barrett,et al.  Image Editing with Intelligent Paint , 2002, Eurographics.

[12]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, SIGGRAPH 2004.

[14]  Darrel Greenhill,et al.  Segmenting film sequences using active surfaces , 1997, Proceedings of International Conference on Image Processing.

[15]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[16]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[17]  Michael Cohen,et al.  Video tooning , 2004, SIGGRAPH 2004.

[18]  Patrick Pérez,et al.  JetStream: probabilistic contour extraction with particles , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[19]  John Collomosse,et al.  Stroke Surfaces: A Spatio-temporal Framework for Temporally Coherent Non- photorealistic Animations , 2003 .

[20]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[21]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[22]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[23]  Chunhong Pan,et al.  An Iterative Bayesian Approach for Digital Matting , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[24]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.