Unsupervised pixel-level video foreground object segmentation via shortest path algorithm

Unsupervised video object segmentation is to automatically segment the foreground object in the video without any prior knowledge. In this paper, we propose an object-level method to extract the foreground object in the video. We firstly generate all the object-like regions as the segmentation candidates. Then based on the corresponding map between the successive frames, the video segmentation problem is converted to corresponding graph model, which selects the most corresponding object region from each frame. The shortest path algorithm is explored to get a global optimum solution for this graph. To obtain a better result, we also introduce a global foreground model to restrict the selected candidates. Finally, we utilize the selected candidates to obtain a more precise pixel-level foreground object segmentation. Compared with the state-of-the-art object-level methods, our method does not only guarantee the continuity of segmentation result, but also works well even under the cases of fast motion and occlusion.

[1]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[2]  Longin Jan Latecki,et al.  Maximum weight cliques with mutex constraints for video object segmentation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Tao Mei,et al.  EMS: Energy Minimization Based Video Scene Segmentation , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[4]  Stephen Lin,et al.  Object-Based Multiple Foreground Video Co-segmentation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  C. Schmid,et al.  Category-Specific Video Summarization , 2014, ECCV.

[6]  Yong Jae Lee,et al.  Discovering important people and objects for egocentric video summarization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[8]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Mei Han,et al.  Efficient hierarchical graph-based video segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Qingshan Liu,et al.  Temporal spectral residual: fast motion saliency detection , 2009, ACM Multimedia.

[11]  Kristen Grauman,et al.  Boundary preserving dense local regions , 2011, CVPR 2011.

[12]  James M. Rehg,et al.  Motion Coherent Tracking with Multi-label MRF optimization , 2010, BMVC.

[13]  Yong Jae Lee,et al.  Key-segments for video object segmentation , 2011, 2011 International Conference on Computer Vision.

[14]  William Brendel,et al.  Video object segmentation by tracking regions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Mubarak Shah,et al.  Video Object Segmentation through Spatially Accurate and Temporally Dense Extraction of Primary Object Regions , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Xiaochun Cao,et al.  Cluster-Based Co-Saliency Detection , 2013, IEEE Transactions on Image Processing.

[18]  Jitendra Malik,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Segmentation of Moving Objects by Long Term Video Analysis , 2022 .

[19]  Xiaochun Cao,et al.  Video object segmentation with shortest path , 2012, ACM Multimedia.

[20]  Derek Hoiem,et al.  Category Independent Object Proposals , 2010, ECCV.

[21]  Esa Rahtu,et al.  Segmenting Salient Objects from Images and Videos , 2010, ECCV.

[22]  Xian-Sheng Hua,et al.  Optimized video scene segmentation , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[23]  Jitendra Malik,et al.  Object Segmentation by Long Term Analysis of Point Trajectories , 2010, ECCV.

[24]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Cristian Sminchisescu,et al.  Constrained parametric min-cuts for automatic object segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  C. V. Jawahar,et al.  Video retrieval by mimicking poses , 2012, ICMR '12.

[27]  Stephen Lin,et al.  Object-based RGBD image co-segmentation with mutex constraint , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Stanley T. Birchfield,et al.  Adaptive fragments-based tracking of non-rigid objects using level sets , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[29]  Mohan M. Trivedi,et al.  Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Thomas Deselaers,et al.  What is an object? , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Shaogang Gong,et al.  International Journal of Computer Vision (The original publication is available at www.springerlink.com) Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding , 2009 .

[32]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[33]  Thomas Brox,et al.  Object segmentation in video: A hierarchical variational approach for turning point trajectories into dense regions , 2011, 2011 International Conference on Computer Vision.