Video Object Extraction via MRF-Based Contour Tracking

Video object segmentation is a critical task in multimedia analysis and editing. Normally, the user provides some hints of foreground and background, then the target object is extracted from the video sequence. Most previous methods are either computation-expensive or labor-intensive, and approaches that assume static background have limited applications. In this letter, we propose a novel video segmentation system that integrates Markov random field-based contour tracking with graph-cut image segmentation. The contour tracking propagates the shape of the target object, whereas the graph-cut refines the shape and improves the accuracy of video segmentation. Experimental results show that our segmentation system is efficient and requires less key-frames and user interactions.

[1]  Michael Cohen,et al.  Soft scissors: an interactive tool for realtime high quality matting , 2007, SIGGRAPH 2007.

[2]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2008 .

[3]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[4]  Roberto Cipolla,et al.  Multi-view stereo via volumetric graph-cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields , 2006, ECCV.

[6]  Maneesh Agrawala,et al.  Interactive video cutout , 2005, SIGGRAPH 2005.

[7]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Mo Chen,et al.  Progressive cut , 2006, MM '06.

[9]  Jian Sun,et al.  Video object cut and paste , 2005, SIGGRAPH 2005.

[10]  Guillermo Sapiro,et al.  A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[11]  Takeo Igarashi,et al.  As-rigid-as-possible shape manipulation , 2005, ACM Trans. Graph..

[12]  Jean Ponce,et al.  Segmentation by transduction , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Guillermo Sapiro,et al.  Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting , 2009, International Journal of Computer Vision.

[14]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[15]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, SIGGRAPH 2004.

[16]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[17]  A. Criminisi,et al.  Bilayer Segmentation of Live Video , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Roland T. Chin,et al.  On the Detection of Dominant Points on Digital Curves , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Harry Shum,et al.  Background Cut , 2006, ECCV.

[20]  Leo Grady,et al.  A Seeded Image Segmentation Framework Unifying Graph Cuts And Random Walker Which Yields A New Algorithm , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[21]  David Salesin,et al.  Keyframe-based tracking for rotoscoping and animation , 2004, ACM Trans. Graph..

[22]  William A. Barrett,et al.  Intelligent scissors for image composition , 1995, SIGGRAPH.

[23]  Scott Schaefer,et al.  Image deformation using moving least squares , 2006, ACM Trans. Graph..

[24]  Jian Sun,et al.  Lazy snapping , 2004, SIGGRAPH 2004.

[25]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[26]  Mubarak Shah,et al.  Object based segmentation of video using color, motion and spatial information , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.