Video saliency detection using 3D shearlet transform

Recently, visual saliency detection has received great interest. As most video saliency detection models are based on spatiotemporal mechanism, we firstly give a simple introduction of it in this paper. After discussing issues to be addressed, we present a novel framework for video saliency detection based on 3D discrete shearlet transform. Instead of measuring saliency by fusing spatial and temporal saliency maps, the proposed model regards video as three-dimensional data. By decomposing the video with 3D discrete shearlet transform and reconstructing it on multi-scales, this multi-scale saliency detection model obtains a number of feature blocks to describe the video. Based on each feature block, every a number of successive feature maps are taken as a whole, and the global contrast is calculated to obtain the saliency maps. By fusing all the saliency maps of different levels, the saliency map is generated for each video frame. This novel framework is very simple, and experimental results on ten videos show that the proposed model outperforms lots existing models.

[1]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[2]  Yu-Chiang Frank Wang,et al.  Exploring Visual and Motion Saliency for Automatic Video Object Extraction , 2013, IEEE Transactions on Image Processing.

[3]  Laura Igual,et al.  Robust gait-based gender classification using depth cameras , 2013, EURASIP Journal on Image and Video Processing.

[4]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[5]  Wonjun Kim,et al.  Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[7]  Yannis Avrithis,et al.  Spatiotemporal saliency for video classification , 2009, Signal Process. Image Commun..

[8]  Sudeep Sarkar,et al.  REM: relational entropy-based measure of saliency , 2010, ICVGIP '10.

[9]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[10]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[11]  Pooran Singh Negi,et al.  3-D Discrete Shearlet Transform and Video Processing , 2012, IEEE Transactions on Image Processing.

[12]  Yang Li,et al.  A saliency detection model using shearlet transform , 2014, Multimedia Tools and Applications.

[13]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  C. Koch,et al.  Models of bottom-up and top-down visual attention , 2000 .

[15]  D. Labate,et al.  Sparse Multidimensional Representations using Anisotropic Dilation and Shear Operators , 2006 .

[16]  Junji Yamato,et al.  Saliency-based video segmentation with graph cuts and sequentially updated priors , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[17]  Sven J. Dickinson,et al.  TurboPixels: Fast Superpixels Using Geometric Flows , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Weisi Lin,et al.  A Video Saliency Detection Model in Compressed Domain , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Bo Wu,et al.  A unified framework for spatiotemporal salient region detection , 2013, EURASIP J. Image Video Process..

[20]  Feng Zhou,et al.  Time-Mapping Using Space-Time Saliency , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Lizhuang Ma,et al.  Temporally Coherent Video Saliency Using Regional Dynamic Contrast , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Lihi Zelnik-Manor,et al.  Learning Video Saliency from Human Gaze Using Candidate Selection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Sudeep Sarkar,et al.  Relational entropy-based saliency detection in images and videos , 2012, 2012 19th IEEE International Conference on Image Processing.

[24]  Wonjun Kim,et al.  Spatiotemporal Saliency Detection Using Textural Contrast and Its Applications , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Ivan V. Bajic,et al.  Saliency-Aware Video Compression , 2014, IEEE Transactions on Image Processing.

[26]  Ruxandra Tapu,et al.  Video structuring: From pixels to visual entities , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[27]  Lihi Zelnik-Manor,et al.  Context-aware saliency detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.