Semiautomatic visual-attention modeling and its application to video compression

This research aims to sufficiently increase the quality of visual-attention modeling to enable practical applications. We found that automatic models are significantly worse at predicting attention than even single-observer eye tracking. We propose a semiautomatic approach that requires eye tracking of only one observer and is based on time consistency of the observer's attention. Our comparisons showed the high objective quality of our proposed approach relative to automatic methods and to the results of single-observer eye tracking with no postprocessing. We demonstrated the practical applicability of our proposed concept to the task of saliency-based video compression.

[1]  A. L. Yarbus Eye Movements During Perception of Complex Objects , 1967 .

[2]  Dmitriy Vatolin,et al.  Fast video super-resolution via classification , 2008, 2008 15th IEEE International Conference on Image Processing.

[3]  Liming Zhang,et al.  A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression , 2010, IEEE Transactions on Image Processing.

[4]  B. Tatler,et al.  Looking and Acting: Vision and eye movements in natural behaviour , 2009 .

[5]  Narciso García,et al.  NAMA3DS1-COSPAD1: Subjective video quality assessment database on coding conditions introducing freely available high quality 3D stereoscopic sequences , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[6]  Hadi Hadizadeh,et al.  Visual Saliency in Video Compression and Transmission , 2013 .

[7]  Martin D. Levine,et al.  Visual Saliency Based on Scale-Space Analysis in the Frequency Domain , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  John K. Tsotsos,et al.  Saliency Based on Information Maximization , 2005, NIPS.

[10]  Laurent Itti,et al.  Visual attention guided bit allocation in video compression , 2011, Image Vis. Comput..

[11]  Lihi Zelnik-Manor,et al.  Saliency for image manipulation , 2013, The Visual Computer.

[12]  Vladimir Zlokolica,et al.  Salient Motion Features for Video Quality Assessment , 2011, IEEE Transactions on Image Processing.

[13]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[14]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[15]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[16]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[17]  Santanu Chaudhury,et al.  Visual saliency guided video compression algorithm , 2013, Signal Process. Image Commun..

[18]  Ivan V. Bajic,et al.  Saliency-preserving video compression , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[19]  Song-Hai Zhang,et al.  Saliency-Based Fidelity Adaptation Preprocessing for Video Coding , 2011, Journal of Computer Science and Technology.

[20]  Mary M Hayhoe,et al.  Spatial memory and saccadic targeting in a natural task. , 2005, Journal of vision.

[21]  A. Hyvärinen,et al.  Spatial frequency tuning in human retinotopic visual areas. , 2008, Journal of vision.

[22]  Nicolas Riche,et al.  Rare: A new bottom-up saliency model , 2012, 2012 19th IEEE International Conference on Image Processing.

[23]  Michael F. Land,et al.  How our eyes question the world , 2009 .

[24]  Lihi Zelnik-Manor,et al.  Context-Aware Saliency Detection , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Itu-T and Iso Iec Jtc Advanced video coding for generic audiovisual services , 2010 .

[26]  Peyman Milanfar,et al.  Nonparametric bottom-up saliency detection by self-resemblance , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[27]  Frédo Durand,et al.  A Benchmark of Computational Models of Saliency to Predict Human Fixations , 2012 .

[28]  Christof Koch,et al.  Learning a saliency map using fixated locations in natural scenes. , 2011, Journal of vision.

[29]  Alexei A. Efros,et al.  Data-driven visual similarity for cross-domain image matching , 2011, ACM Trans. Graph..

[30]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.