Fast Calculation Method of Video Saliency Based on Temporal and Spatial Edge-Preserving Filter

How to detect video saliency accurately is an very challenging problem in computer vision, and has gained great research and application values. A fast video saliency detection method, which based on spatio-temporal edge-preserving filter, was proposed in this paper. Firstly, object appearance features and motion features were extracted in video frames. Secondly, a new spatio-temporal edge-preserving filter was constructed by combining object temporal information in video. Then, the video saliency detection is formulated as an energy minimization problem, which fuses the appearance and motion features, and stand out salient object in video uniformly by using the spatio-temporal edge-preserving filter. Finally, extensive experiments, including comparisons analysis with three video saliency detection approaches and components analysis of the proposed approach, on three datasets suggest that the proposed approach nearly outperforms other approaches in both accuracy and efficiency.

[1]  Qian Li,et al.  Predictive Video Saliency Detection , 2012, CCPR.

[2]  Yan Liu,et al.  Video Saliency Detection via Dynamic Consistent Spatio-Temporal Attention Modelling , 2013, AAAI.

[3]  Liang Lin,et al.  PISA: Pixelwise Image Saliency by Aggregating Complementary Appearance Contrast Measures With Edge-Preserving Coherence , 2015, IEEE Transactions on Image Processing.

[4]  Minh N. Do,et al.  Cross-based local multipoint filtering , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[6]  Asha Iyer,et al.  Components of bottom-up gaze allocation in natural images , 2005, Vision Research.

[7]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[8]  Vladimir Zlokolica,et al.  Salient Motion Features for Video Quality Assessment , 2011, IEEE Transactions on Image Processing.

[9]  Peyman Milanfar,et al.  Static and space-time visual saliency detection by self-resemblance. , 2009, Journal of vision.

[10]  Nicolas Riche,et al.  Dynamic Saliency Models and Human Attention: A Comparative Study on Videos , 2012, ACCV.

[11]  Hailin Jin,et al.  Fast Edge-Preserving PatchMatch for Large Displacement Optical Flow , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Lihi Zelnik-Manor,et al.  Learning Video Saliency from Human Gaze Using Candidate Selection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.