An Attempt to Segment Foreground in Dynamic Scenes

In general, human behavior analysis relies on a sequence of human segments, e.g. gait recognition aims to address human identification based on people's manners of walking, and thus relies on the segmented silhouettes. Background subtraction is the most widely used approach to segment foreground, while dynamic scenes make it difficult to work. In this paper, we propose to combine Mean-Shift-based tracking with adaptive scale and Graphcuts-based segmentation with label propagation. The average precision on a number of sequences is 0.82, and the average recall is 0.72. Besides, our method only requires weak user interaction and is computationally efficient. We compare our method with its variant without label propagation, as well as GrabCut. For the tracking module only, we compare Mean Shift with several state-of-the-art methods (i.e. OnlineBoost, SemiBoost, MILTrack, FragTrack).

[1]  Qunsheng Peng,et al.  Transductive segmentation of live video with non-stationary background , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Harry Shum,et al.  Background Cut , 2006, ECCV.

[3]  Antti Oulasvirta,et al.  Computer Vision – ECCV 2006 , 2006, Lecture Notes in Computer Science.

[4]  Bohyung Han,et al.  Efficient extraction of human motion volumes by tracking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Yogesh Rathi,et al.  Multi-Object Tracking Through Clutter Using Graph Cuts , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[6]  Horst Bischof,et al.  Semi-supervised On-Line Boosting for Robust Tracking , 2008, ECCV.

[7]  Horst Bischof,et al.  On-line Boosting and Vision , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[8]  Yogesh Rathi,et al.  Tracking Through Clutter Using Graph Cuts , 2007, BMVC.

[9]  A. Criminisi,et al.  Bilayer Segmentation of Live Video , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[10]  Guillermo Sapiro,et al.  Dynamic Color Flow: A Motion-Adaptive Color Model for Object Segmentation in Video , 2010, ECCV.

[11]  W. Eric L. Grimson,et al.  Gait analysis for recognition and classification , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[12]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[13]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Thomas Deselaers,et al.  ClassCut for Unsupervised Class Segmentation , 2010, ECCV.

[15]  Jian Sun,et al.  Video object cut and paste , 2005, SIGGRAPH 2005.

[16]  Patrick Pérez,et al.  Detection and segmentation of moving objects in highly dynamic scenes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Roberto Cipolla,et al.  Label propagation in video sequences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  Bohyung Han,et al.  Extracting Moving People from Internet Videos , 2008, ECCV.

[20]  Pushmeet Kohli,et al.  PoseCut: Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph-Cuts , 2006, ECCV.

[21]  Ehud Rivlin,et al.  Robust Fragments-based Tracking using the Integral Histogram , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[22]  Ming-Hsuan Yang,et al.  Visual tracking with online Multiple Instance Learning , 2009, CVPR.

[23]  Jitendra Malik,et al.  Tracking as Repeated Figure/Ground Segmentation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[25]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[26]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[27]  Mei Han,et al.  Efficient hierarchical graph-based video segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Olivier Juan,et al.  Active Graph Cuts , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).