Motion saliency detection for compressed videos

Abstract. Motion saliency detection in a compressed domain is crucial for various video applications, including retargeting, surveillance, object checking, and segmentation. The goal of this paper is to improve the performances of an existing motion saliency detection model in a compressed domain developed by Fang et al. Specifically, we improve the detection accuracy of motion center-surround features by dynamically fitting the parameters of a Gaussian distribution model. Besides, the parameters for the distribution of distance in horizontal and vertical directions are obtained separately instead of treating them together. In addition, the motion importance features are exploited to strengthen the performance of detection. Experimental results demonstrate that the proposed motion saliency detection method outperforms the existing approaches in both a pixel and compressed domain.

[1]  Aidong Men,et al.  A novel video saliency map detection model in compressed domain , 2015, MILCOM 2015 - 2015 IEEE Military Communications Conference.

[2]  Ivan V. Bajic,et al.  Comparison of visual saliency models for compressed video , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[3]  Dwarikanath Mahapatra,et al.  Coherency Based Spatio-Temporal Saliency Detection for Video Object Segmentation , 2014, IEEE Journal of Selected Topics in Signal Processing.

[4]  Aniruddha Sinha,et al.  Region-of-interest based compressed domain video transcoding scheme , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[6]  Wei Chen,et al.  Region-of-Interest intra prediction for H.264/AVC error resilience , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[7]  Chang-Su Kim,et al.  Video saliency detection based on spatiotemporal feature learning , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[8]  Weisi Lin,et al.  A Video Saliency Detection Model in Compressed Domain , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Mihai Mitrea,et al.  Visual saliency in MPEG-4 AVC video stream , 2015, Electronic Imaging.

[10]  Chang-Su Kim,et al.  Compressed domain video saliency detection using global and local spatiotemporal features , 2016, J. Vis. Commun. Image Represent..

[11]  Nicolas Riche,et al.  Saliency and Human Fixations: State-of-the-Art and Study of Comparison Metrics , 2013, 2013 IEEE International Conference on Computer Vision.

[12]  Aidong Men,et al.  Video saliency detection incorporating temporal information in compressed domain , 2015, Signal Process. Image Commun..

[13]  Alan C. Bovik,et al.  Point-of-gaze analysis reveals visual search strategies , 2004, IS&T/SPIE Electronic Imaging.

[14]  Deepu Rajan,et al.  Salient Motion Detection in Compressed Domain , 2013, IEEE Signal Processing Letters.

[15]  Yuan F. Zheng,et al.  Object Tracking in Structured Environments for Video Surveillance Applications , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[17]  Ivan V. Bajic,et al.  Eye-Tracking Database for a Set of Standard Video Sequences , 2012, IEEE Transactions on Image Processing.

[18]  Chang-Su Kim,et al.  Spatiotemporal Saliency Detection for Video Sequences Based on Random Walk With Restart , 2015, IEEE Transactions on Image Processing.

[19]  Yongjun Li,et al.  A fast and efficient saliency detection model in video compressed-domain for human fixations prediction , 2017, Multimedia Tools and Applications.

[20]  Peyman Milanfar,et al.  Static and space-time visual saliency detection by self-resemblance. , 2009, Journal of vision.

[21]  Liming Zhang,et al.  A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression , 2010, IEEE Transactions on Image Processing.

[22]  Asha Iyer,et al.  Components of bottom-up gaze allocation in natural images , 2005, Vision Research.

[23]  A. Coutrot,et al.  How saliency, faces, and sound influence gaze in dynamic social scenes. , 2014, Journal of vision.

[24]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[25]  Zhou Wang,et al.  Video saliency incorporating spatiotemporal cues and uncertainty weighting , 2013, ICME.

[26]  Chia-Wen Lin,et al.  Maintaining Temporal Coherence in Video Retargeting Using Mosaic-Guided Scaling , 2011, IEEE Transactions on Image Processing.

[27]  Shan Li,et al.  An Efficient Spatiotemporal Attention Model and Its Application to Shot Matching , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Lucas Paletta,et al.  Novelty-based Spatiotemporal Saliency Detection for Prediction of Gaze in Egocentric Video , 2016, IEEE Signal Processing Letters.

[29]  Dwarikanath Mahapatra,et al.  Illumination invariant tracking in office environments using neurobiology-saliency based particle filter , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[30]  Aidong Men,et al.  Visual saliency detection based on mutual information in compressed domain , 2015, 2015 Visual Communications and Image Processing (VCIP).

[31]  John A. Swets,et al.  Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers , 1996 .

[32]  Aidong Men,et al.  Video saliency detection based on mutual information and background prior in compressed domain , 2015, 2015 IEEE/CIC International Conference on Communications in China (ICCC).

[33]  Ivan V. Bajic,et al.  Compressed-domain correlates of human fixations in dynamic scenes , 2015, Multimedia Tools and Applications.

[34]  Sabine Süsstrunk,et al.  Saliency detection for content-aware image resizing , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[35]  Wonjun Kim,et al.  Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[36]  Mihai Mitrea,et al.  HEVC saliency map computation , 2016, HVEI.

[37]  Mubarak Shah,et al.  Visual attention detection in video sequences using spatiotemporal cues , 2006, MM '06.

[38]  Weisi Lin,et al.  Saliency Detection in the Compressed Domain for Adaptive Image Retargeting , 2012, IEEE Transactions on Image Processing.