Highly Consistent Sequential Segmentation

This paper deals with segmentation of image sequences in an unsupervised manner with the goal of getting highly consistent segmentation results from frame-to-frame. We first introduce a segmentation method that uses results of the previous frame as initialization and significantly improves consistency in comparison to a single frame based approach. We also find correspondences between the segmented regions from one frame to the next to further increase consistency. This matching step is based on a modified version of an efficient partial shape matching method which allows identification of similar parts of regions despite topology changes like merges and splits. We use the identified matched parts to define a partial matching cost which is then used as input to pairwise graph matching. Experiments demonstrate that we can achieve highly consistent segmentations for diverse image sequences, even allowing to track manually initialized moving and static objects.

[1]  Narendra Ahuja,et al.  Matching images under unstable segmentations , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Daniel Cremers,et al.  Fast and exact solution of Total Variation models on the GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[4]  Dimitris N. Metaxas,et al.  ]Video object segmentation by hypergraph cut , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Hongbin Zha,et al.  Computer Vision - ACCV 2009, 9th Asian Conference on Computer Vision, Xi'an, China, September 23-27, 2009, Revised Selected Papers, Part III , 2010, Asian Conference on Computer Vision.

[6]  Jitendra Malik,et al.  From contours to regions: An empirical evaluation , 2009, CVPR.

[7]  Martial Hebert,et al.  A spectral technique for correspondence problems using pairwise constraints , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[8]  William Brendel,et al.  Video object segmentation by tracking regions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[9]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[10]  Hayko Riemenschneider,et al.  Efficient Partial Shape Matching of Outer Contours , 2009, ACCV.

[11]  David Salesin,et al.  Video object annotation, navigation, and composition , 2008, UIST '08.

[12]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[13]  Daniel DeMenthon,et al.  SPATIO-TEMPORAL SEGMENTATION OF VIDEO BY HIERARCHICAL MEAN SHIFT ANALYSIS , 2002 .

[14]  Xavier Bresson,et al.  Fast Global Minimization of the Active Contour/Snake Model , 2007, Journal of Mathematical Imaging and Vision.

[15]  Haibin Ling,et al.  Using the inner-distance for classification of articulated shapes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  Jitendra Malik,et al.  Motion segmentation and tracking using normalized cuts , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  Horst Bischof,et al.  Saliency driven total variation segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[18]  Joshua D. Schwartz,et al.  Hierarchical Matching of Deformable Shapes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Serge J. Belongie,et al.  What went where , 2003, CVPR 2003.

[20]  Serge J. Belongie,et al.  What went where [motion segmentation] , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..