Efficient Video Cutout by Paint Selection

Video cutout refers to extracting moving objects from videos, which is an important step in many video editing tasks. Recent algorithms have limitations in terms of efficiency, interaction style, and robustness. This paper presents a novel method for progressive video cutout with less user interaction and fast feedback. By exploring local and compact features, an optimization is constructed based on a graph model which establishes spatial and temporal relationship of neighboring patches in video frames. This optimization enables an efficient solution for progressive video cutout using graph cuts. Furthermore, a sampling-based method for temporally coherent matting is proposed to further refine video cutout results. Experiments demonstrate that our video cutout by paint selection is more intuitive and efficient for users than previous stroke-based methods, and thus could be put into practical use.

[1]  Jin Wei,et al.  Timeline Editing of Objects in Video , 2013, IEEE Transactions on Visualization and Computer Graphics.

[2]  Michael F. Cohen,et al.  Image and Video Matting: A Survey , 2007, Found. Trends Comput. Graph. Vis..

[3]  Leo Grady,et al.  A multilevel banded graph cuts method for fast image segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[4]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, ACM Trans. Graph..

[5]  Ralph R. Martin,et al.  Online Video Stream Abstraction and Stylization , 2011, IEEE Transactions on Multimedia.

[6]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Kun Xu,et al.  Efficient antialiased edit propagation for images and videos , 2012, Comput. Graph..

[8]  Xiangxu Meng,et al.  Discontinuity-aware video object cutout , 2012, ACM Trans. Graph..

[9]  Harry Shum,et al.  Lazy snapping , 2004, ACM Trans. Graph..

[10]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, SIGGRAPH 2009.

[11]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Shi-Min Hu,et al.  Efficient affinity-based edit propagation using K-D tree , 2009, ACM Trans. Graph..

[13]  Harry Shum,et al.  Paint selection , 2009, ACM Trans. Graph..

[14]  Qionghai Dai,et al.  A Progressive Tri-level Segmentation Approach for Topology-Change-Aware Video Matting , 2013, Comput. Graph. Forum.

[15]  Scott Cohen,et al.  Temporally coherent and spatially accurate video matting , 2014, Comput. Graph. Forum.

[16]  Yun Zhang,et al.  Video Brush: A Novel Interface for Efficient Video Cutout , 2011, Comput. Graph. Forum.

[17]  Jian Sun,et al.  A global sampling method for alpha matting , 2011, CVPR 2011.

[18]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, ACM Trans. Graph..

[19]  Ralph R. Martin,et al.  Internet visual media processing: a survey with graphics and vision applications , 2013, The Visual Computer.

[20]  Shi-Min Hu,et al.  Motion-Aware Gradient Domain Video Composition , 2013, IEEE Transactions on Image Processing.

[21]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[22]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..

[23]  Jue Wang,et al.  Towards Temporally-Coherent Video Matting , 2011, MIRAGE.

[24]  Hua Huang,et al.  RepSnapping: Efficient Image Cutout for Repeated Scene Elements , 2011, Comput. Graph. Forum.

[25]  Harry Shum,et al.  Video object cut and paste , 2005, ACM Trans. Graph..

[26]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, SIGGRAPH 2004.

[27]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..