Video Brush: A Novel Interface for Efficient Video Cutout

We present Video Brush, a novel interface for interactive video cutout. Inspired by the progressive selection scheme in images, our interface is designed to select video objects by painting on successive frames as the video plays. The video objects are progressively selected by solving the graph‐cut based local optimization according to the strokes drawn by the brush on each painted frame. In order to provide users interactive feedback, we accelerate 3D graph‐cut by efficient graph building and multi‐level banded graph‐cut. Experimental results show that our novel interface is both intuitive and efficient for video cutout.

[1]  Andrew V. Goldberg,et al.  A new approach to the maximum flow problem , 1986, STOC '86.

[2]  Harry Shum,et al.  Video object cut and paste , 2005, ACM Trans. Graph..

[3]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[4]  Richard M. Karp,et al.  Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems , 1972, Combinatorial Optimization.

[5]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[6]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Zeev Farbman,et al.  Coordinates for instant image cloning , 2009, ACM Trans. Graph..

[8]  Guillermo Sapiro,et al.  Video SnapCut: robust video object cutout using localized classifiers , 2009, SIGGRAPH 2009.

[9]  Manuel Menezes de Oliveira Neto,et al.  Shared Sampling for Real‐Time Alpha Matting , 2010, Comput. Graph. Forum.

[10]  William A. Barrett,et al.  Interactive segmentation of image volumes with Live Surface , 2007, Comput. Graph..

[11]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Leo Grady,et al.  A Seeded Image Segmentation Framework Unifying Graph Cuts And Random Walker Which Yields A New Algorithm , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[13]  Jian Sun,et al.  Video object cut and paste , 2005, SIGGRAPH 2005.

[14]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[15]  Esa Rahtu,et al.  Segmenting Salient Objects from Images and Videos , 2010, ECCV.

[16]  Harry Shum,et al.  Paint selection , 2009, ACM Trans. Graph..

[17]  Guillermo Sapiro,et al.  Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting , 2009, International Journal of Computer Vision.

[18]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Fabio Pellacini,et al.  AppProp: all-pairs appearance-space edit propagation , 2008, ACM Trans. Graph..

[20]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[21]  David A. Forsyth,et al.  Generalizing motion edits with Gaussian processes , 2009, ACM Trans. Graph..

[22]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, ACM Trans. Graph..

[23]  Leo Grady,et al.  A multilevel banded graph cuts method for fast image segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[24]  Fabio Pellacini,et al.  AppProp: all-pairs appearance-space edit propagation , 2008, SIGGRAPH 2008.

[25]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[26]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[27]  Shi-Min Hu,et al.  RepFinder: finding approximately repeated scene elements for image editing , 2010, SIGGRAPH 2010.

[28]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[29]  Shi-Min Hu,et al.  Efficient affinity-based edit propagation using K-D tree , 2009, SIGGRAPH 2009.

[30]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[31]  Guillermo Sapiro,et al.  Dynamic Color Flow: A Motion-Adaptive Color Model for Object Segmentation in Video , 2010, ECCV.

[32]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Shi-Min Hu,et al.  Sketch2Photo: internet image montage , 2009, ACM Trans. Graph..

[34]  Shi-Min Hu,et al.  RepFinder: finding approximately repeated scene elements for image editing , 2010, ACM Trans. Graph..

[35]  Michael F. Cohen,et al.  Image and Video Matting: A Survey , 2007, Found. Trends Comput. Graph. Vis..

[36]  Yuri Boykov,et al.  A Scalable graph-cut algorithm for N-D grids , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Jian Sun,et al.  Poisson matting , 2004, ACM Trans. Graph..

[38]  Heung-Yeung Shum,et al.  Paint selection , 2009, SIGGRAPH 2009.

[39]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.