Coherency Based Spatio-Temporal Saliency Detection for Video Object Segmentation

Extracting moving and salient objects from videos is important for many applications like surveillance and video retargeting. In this paper we use spatial and temporal coherency information to segment salient objects in videos. While many methods use motion information from videos, they do not exploit coherency information which has the potential to give more accurate saliency maps. Spatial coherency maps identify regions belonging to regular objects, while temporal coherency maps identify regions with high coherent motion. The two coherency maps are combined to obtain the final spatio-temporal map identifying salient regions. Experimental results on public datasets show that our method outperforms two competing methods in segmenting moving objects from videos.

[1]  Ying Sun,et al.  Rigid Registration of Renal Perfusion Images Using a Neurobiology-Based Visual Saliency Model , 2010, EURASIP J. Image Video Process..

[2]  R. Abrams,et al.  Motion Onset Captures Attention , 2003, Psychological science.

[3]  Shang-Hong Lai,et al.  Fusing generic objectness and visual saliency for salient object detection , 2011, 2011 International Conference on Computer Vision.

[4]  Huchuan Lu,et al.  Saliency Detection via Dense and Sparse Reconstruction , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[6]  Dwarikanath Mahapatra,et al.  Nonrigid Registration of Dynamic Renal MR Images Using a Saliency Based MRF Model , 2008, MICCAI.

[7]  King Ngi Ngan,et al.  Saliency model-based face segmentation and tracking in head-and-shoulder video sequences , 2008, J. Vis. Commun. Image Represent..

[8]  Daniel Cohen-Or,et al.  Non-homogeneous Content-driven Video-retargeting , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Sheng-Wen Shih,et al.  Dynamic visual saliency modeling based on spatiotemporal analysis , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[10]  J.-L. Wu,et al.  Video Adaptation for Small Display Based on Content Recomposition , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[12]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[14]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[15]  Atsushi Nakazawa,et al.  Motion Coherent Tracking Using Multi-label MRF Optimization , 2012, International Journal of Computer Vision.

[16]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[17]  James M. Rehg,et al.  Motion Coherent Tracking with Multi-label MRF optimization , 2010, BMVC.

[18]  Lihi Zelnik-Manor,et al.  Context-aware saliency detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[20]  A. L. Yarbus,et al.  Eye Movements and Vision , 1967, Springer US.

[21]  Vibhav Vineet,et al.  Efficient Salient Region Detection with Soft Image Abstraction , 2013, 2013 IEEE International Conference on Computer Vision.

[22]  Dwarikanath Mahapatra,et al.  Illumination invariant tracking in office environments using neurobiology-saliency based particle filter , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[23]  Yan Liu,et al.  Video Saliency Detection via Dynamic Consistent Spatio-Temporal Attention Modelling , 2013, AAAI.

[24]  Xiaogang Wang,et al.  Person Re-identification by Salience Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[25]  L. R. Dice Measures of the Amount of Ecologic Association Between Species , 1945 .

[26]  Dwarikanath Mahapatra,et al.  Integrating Segmentation Information for Improved MRF-Based Elastic Image Registration , 2012, IEEE Transactions on Image Processing.

[27]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[28]  Dwarikanath Mahapatra,et al.  MRF-Based Intensity Invariant Elastic Registration of Cardiac Perfusion Images Using Saliency Information , 2011, IEEE Transactions on Biomedical Engineering.

[29]  Jingdong Wang,et al.  Salient Object Detection: A Discriminative Regional Feature Integration Approach , 2013, International Journal of Computer Vision.

[30]  Nuno Vasconcelos,et al.  Spatiotemporal Saliency in Dynamic Scenes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Lihi Zelnik-Manor,et al.  Learning Video Saliency from Human Gaze Using Candidate Selection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Esa Rahtu,et al.  Segmenting Salient Objects from Images and Videos , 2010, ECCV.

[34]  Dwarikanath Mahapatra,et al.  Registration of dynamic renal MR images using neurobiological model of saliency , 2008, 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[35]  James W. Davis,et al.  Background-subtraction using contour-based fusion of thermal and visible imagery , 2007, Comput. Vis. Image Underst..

[36]  Liming Zhang,et al.  A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression , 2010, IEEE Transactions on Image Processing.

[37]  Stefan Winkler,et al.  Motion saliency outweighs other low-level features while watching videos , 2008, Electronic Imaging.

[38]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, ACM Trans. Graph..

[39]  Nuno Vasconcelos,et al.  Integrated learning of saliency, complex features, and object detectors from cluttered scenes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[40]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[41]  Dwarikanath Mahapatra,et al.  Joint Registration and Segmentation of Dynamic Cardiac Perfusion Images Using MRFs , 2010, MICCAI.

[42]  Wonjun Kim,et al.  Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  A. L. I︠A︡rbus Eye Movements and Vision , 1967 .

[44]  Dwarikanath Mahapatra,et al.  Orientation Histograms as Shape Priors for Left Ventricle Segmentation Using Graph Cuts , 2011, MICCAI.

[45]  B. Tatler,et al.  Yarbus, eye movements, and vision , 2010, i-Perception.

[46]  Kai-Kuang Ma,et al.  Adaptive rood pattern search for fast block-matching motion estimation , 2002, IEEE Trans. Image Process..