Perceptual synoptic view of pixel, object and semantic based attributes of video

Display Omitted The need for various object and semantics level attributes is described in detail.Attention model based on object and semantics level attributes is proposed.Key frames are selected based on the proposed model.Key frames are fused to give perceptual synopsis. For a scene, what are the object and semantic based attributes, other than the pixel based attributes, and how do they affect our attentional selection are some of the questions we need to address. We studied the effects of various attributes on our attentional perspective. We described a new saliency prediction model that accounts for different pixel-level attributes as color, contrast and intensity; object level attributes such as size, shape of objects and semantic level attributes as motion and speed of objects. We quantified these attributes based on motion contrast, motion energy and motion chromism. With this in view, we examined the problem of information prioritizing and filtering with emphasis on directing this exercise using object and semantic based attributes of the human attention model. We have evaluated proposed approach on different types of videos for their quantitative and qualitative comparison. The promising results create a gateway for synopsis view.

[1]  D. Regan,et al.  Visual perception of changing size: The effect of object size , 1979, Vision Research.

[2]  Changsheng Xu,et al.  Cross-Domain Feature Learning in Multimedia , 2015, IEEE Transactions on Multimedia.

[3]  Mark Wexler,et al.  Depth Affects Where We Look , 2008, Current Biology.

[4]  Jiang Peng,et al.  Keyframe-Based Video Summary Using Visual Attention Clues , 2010 .

[5]  D. Cohen-Or,et al.  Action synopsis: pose selection and illustration , 2005, SIGGRAPH 2005.

[6]  Lie Lu,et al.  A generic framework of user attention model and its application in video summarization , 2005, IEEE Trans. Multim..

[7]  P. J. Narayanan,et al.  Interactive Video Manipulation Using Object Trajectories and Scene Backgrounds , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Walter Bender,et al.  Salient Stills: Process and Practice , 1996, IBM Syst. J..

[9]  L G Williams,et al.  The effects of target specification on objects fixated during visual search. , 1967, Acta psychologica.

[10]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[11]  P Reinagel,et al.  Natural scene statistics at the centre of gaze. , 1999, Network.

[12]  Jacob L. Orquin,et al.  A review of the findings and theories on surface size effects on visual attention , 2013, Front. Psychol..

[13]  Kerry Hourigan,et al.  Wake transition of a rolling sphere , 2011, J. Vis..

[14]  L. Itti,et al.  Visual causes versus correlates of attentional selection in dynamic scenes , 2006, Vision Research.

[15]  Qionghai Dai,et al.  WBSMDA: Within and Between Score for MiRNA-Disease Association prediction , 2016, Scientific Reports.

[16]  Alan C. Bovik,et al.  Saliency Prediction on Stereoscopic Videos , 2014, IEEE Transactions on Image Processing.

[17]  Christopher G. Healey,et al.  Visualizing data with motion , 2005, VIS 05. IEEE Visualization, 2005..

[18]  M. Corbetta,et al.  Selective and divided attention during visual discriminations of shape, color, and speed: functional anatomy by positron emission tomography , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[19]  Daniel Bullock,et al.  How Position, Velocity, and Temporal Information Combine in the Prospective Control of Catching: Data and Model , 2005, Journal of Cognitive Neuroscience.

[20]  Letitia R. Naigles,et al.  The Shape Bias is Affected by Differing Similarity Among Objects. , 2012, Cognitive development.

[21]  L. G. Williams,et al.  The effect of target specification on objects fixated during visual search , 1966 .

[22]  Weisi Lin,et al.  Scene-Based Movie Summarization Via Role-Community Networks , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  I Kovács,et al.  A closed curve is much more than an incomplete one: effect of closure in figure-ground segmentation. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Shuo Wang,et al.  Predicting human gaze beyond pixels. , 2014, Journal of vision.

[25]  Yongdong Zhang,et al.  A Highly Parallel Framework for HEVC Coding Unit Partitioning Tree Decision on Many-core Processors , 2014, IEEE Signal Processing Letters.

[26]  Yael Pritch,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2008 1 Non-Chronological Video , 2022 .

[27]  G. Hauske,et al.  Object and scene analysis by saccadic eye-movements: an investigation with higher-order statistics. , 2000, Spatial vision.

[28]  LinLin Shen,et al.  Visual-Patch-Attention-Aware Saliency Detection , 2015, IEEE Transactions on Cybernetics.

[29]  Hanspeter Pfister,et al.  Video Snapshots: Creating High-Quality Images from Video Clips , 2012, IEEE Transactions on Visualization and Computer Graphics.

[30]  Yongdong Zhang,et al.  Efficient Parallel Framework for HEVC Motion Estimation on Many-Core Processors , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Zoe Kourtzi,et al.  Shape Saliency Modulates Contextual Processing in the Human Lateral Occipital Complex , 2004, Journal of Cognitive Neuroscience.

[32]  Sung Wook Baik,et al.  Feature aggregation based visual attention model for video summarization , 2014, Comput. Electr. Eng..

[33]  Antonio Fernández-Caballero,et al.  Dynamic visual attention model in image sequences , 2007, Image Vis. Comput..

[34]  Jenq-Neng Hwang,et al.  Content-Based Attention Ranking Using Visual and Contextual Attention Model for Baseball Videos , 2009, IEEE Transactions on Multimedia.

[35]  N. Mackworth,et al.  Cognitive determinants of fixation location during picture viewing. , 1978, Journal of experimental psychology. Human perception and performance.

[36]  Pietro Perona,et al.  On the usefulness of attention for object recognition , 2004 .

[37]  Ding Gangyi,et al.  A Computable Visual Attention Model for Video Skimming , 2008, 2008 Tenth IEEE International Symposium on Multimedia.

[38]  K. S. Venkatesh,et al.  Non user interaction content summarization , 2014, 2014 19th International Conference on Digital Signal Processing.