Online Video Object Segmentation via Boundary-Constrained Low-Rank Sparse Representation

Graphcut-based algorithm is adopted in many video object segmentation systems because different terms can be probabilistically fused together in a framework. Constructing spatio-temporal coherences is an important stage in segmentation systems. However, many steps are involved when computing a key term with good discriminative power. If the cascade steps are adopted, the inaccurate output of the previous step will definitely affect the next step, leading to inaccurate segmentation. In this paper, a key term that is computed by a single framework referred to as boundary-constrained low-rank sparse representation (BCLRSR) is proposed to achieve the accurate segmentation. By treating the elements as linear combinations of dictionary templates, low-rank sparse optimization is adopted to achieve the spatio-temporal saliency. For adding the spatial information to the low-rank sparse model, a boundary constraint is adopted in the framework as a Laplacian regularization. A BCLRSR saliency is then obtained by the represented coefficients, which measure the similarity between the elements in the current frame and the ones in the dictionary. At last, the object is segmented by minimizing the energy function, which is formalized by the spatio-temporal coherences. The experiments on some public datasets show that our proposed algorithm outperforms the state-of-the-art methods.

[1]  Donald B. Johnson,et al.  Efficient Algorithms for Shortest Paths in Sparse Networks , 1977, J. ACM.

[2]  Atsushi Nakazawa,et al.  Motion Coherent Tracking Using Multi-label MRF Optimization , 2012, International Journal of Computer Vision.

[3]  Michael J. Black,et al.  Video Segmentation via Object Flow , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Yael Pritch,et al.  Saliency filters: Contrast based filtering for salient region detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Robert T. Collins,et al.  Online Figure-ground Segmentation with Edge Pixel Classification , 2008, BMVC.

[6]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[7]  Shuicheng Yan,et al.  SOLD: Sub-optimal low-rank decomposition for efficient video segmentation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Liang Lin,et al.  Robust Region Grouping via Internal Patch Statistics , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Jitendra Malik,et al.  Learning to segment moving objects in videos , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Patrick Pérez,et al.  Geodesic image and video editing , 2010, TOGS.

[11]  Mei Xie,et al.  Figure/ground video segmentation via low-rank sparse learning , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[12]  Haibin Ling,et al.  Real time robust L1 tracker using accelerated proximal gradient approach , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Mubarak Shah,et al.  Video Object Segmentation through Spatially Accurate and Temporally Dense Extraction of Primary Object Regions , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  James M. Rehg,et al.  Combining Self Training and Active Learning for Video Segmentation , 2011, BMVC.

[15]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Huchuan Lu,et al.  Saliency Detection via Graph-Based Manifold Ranking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Enhua Wu,et al.  Multilevel Model for Video Object Segmentation Based on Supervision Optimization , 2019, IEEE Transactions on Multimedia.

[18]  Dongmei Zhou,et al.  Online video object segmentation via LRS representation , 2019, IET Comput. Vis..

[19]  Yong Jae Lee,et al.  Key-segments for video object segmentation , 2011, 2011 International Conference on Computer Vision.

[20]  Xiaogang Wang,et al.  Unsupervised Video Matting via Sparse and Low-Rank Representation , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ming-Hsuan Yang,et al.  Hierarchical Convolutional Features for Visual Tracking , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[22]  Peyman Milanfar,et al.  Static and space-time visual saliency detection by self-resemblance. , 2009, Journal of vision.

[23]  Mei Han,et al.  Efficient hierarchical graph-based video segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Kristen Grauman,et al.  Supervoxel-Consistent Foreground Propagation in Video , 2014, ECCV.

[25]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  James M. Rehg,et al.  Video Segmentation by Tracking Many Figure-Ground Segments , 2013, 2013 IEEE International Conference on Computer Vision.

[27]  Jitendra Malik,et al.  Object Segmentation by Long Term Analysis of Point Trajectories , 2010, ECCV.

[28]  Nenghai Yu,et al.  Non-negative low rank and sparse graph for semi-supervised learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Bo Han,et al.  TouchCut: Fast image and video segmentation using single-touch interaction , 2014, Comput. Vis. Image Underst..

[30]  Vittorio Ferrari,et al.  Fast Object Segmentation in Unconstrained Video , 2013, 2013 IEEE International Conference on Computer Vision.

[31]  Huchuan Lu,et al.  Robust Superpixel Tracking , 2014, IEEE Transactions on Image Processing.

[32]  Wei Li,et al.  Single and Multiple Object Tracking Using a Multi-Feature Joint Sparse Representation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Guillermo Sapiro,et al.  A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[34]  Tinghuai Wang,et al.  Probabilistic Motion Diffusion of Labeling Priors for Coherent Video Segmentation , 2012, IEEE Transactions on Multimedia.

[35]  A. Criminisi,et al.  Bilayer Segmentation of Live Video , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[36]  Huchuan Lu,et al.  Superpixel tracking , 2011, 2011 International Conference on Computer Vision.

[37]  Liming Zhang,et al.  Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Andrew Blake,et al.  GeoS: Geodesic Image Segmentation , 2008, ECCV.

[39]  Junbin Gao,et al.  Laplacian Regularized Low-Rank Representation and Its Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Ruigang Yang,et al.  Saliency-Aware Video Object Segmentation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Weisheng Li,et al.  Video saliency detection based on low-level saliency fusion and saliency-aware geodesic , 2019, J. Electronic Imaging.

[42]  Ying Wu,et al.  A unified approach to salient object detection via low rank matrix recovery , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  S. Süsstrunk,et al.  Frequency-tuned salient region detection , 2009, CVPR 2009.

[44]  Cristian Sminchisescu,et al.  Efficient Closed-Form Solution to Generalized Boundary Detection , 2012, ECCV.

[45]  Narendra Ahuja,et al.  Low-Rank Sparse Learning for Robust Visual Tracking , 2012, ECCV.

[46]  Haibin Ling,et al.  Robust Visual Tracking and Vehicle Classification via Sparse Representation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[48]  Arvind Ganesh,et al.  Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix , 2009 .

[49]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[50]  Feng Zhou,et al.  Time-Mapping Using Space-Time Saliency , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Xiaochun Cao,et al.  Cluster-Based Co-Saliency Detection , 2013, IEEE Transactions on Image Processing.

[52]  Scott Cohen,et al.  Geodesic graph cut for interactive image segmentation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[53]  Zhixun Su,et al.  Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation , 2011, NIPS.