Fusion of Multiple Visual Cues for Visual Saliency Extraction from Wearable Camera Settings with Strong Motion

In this paper we are interested in the saliency of visual content from wearable cameras. The subjective saliency in wearable video is studied first due to the psycho-visual experience on this content. Then the method for objective saliency map computation with a specific contribution based on geometrical saliency is proposed. Fusion of spatial, temporal and geometric cues in an objective saliency map is realized by the multiplicative operator. Resulting objective saliency maps are evaluated against the subjective maps with promising results, highlighting interesting performance of proposed geometric saliency model.

[1]  Scott Daly,et al.  Engineering observations from spatiovelocity and spatiotemporal visual models , 1998, Electronic Imaging.

[2]  Bärbel Mertsching,et al.  Fast and Robust Generation of Feature Maps for Region-Based Visual Attention , 2008, IEEE Transactions on Image Processing.

[3]  Alex Pentland,et al.  Visual contextual awareness in wearable computing , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[4]  Patrick Lambert,et al.  A color-action perceptual approach to the classification of animated movies , 2011, ICMR '11.

[5]  David S Wooding,et al.  Eye movements of large populations: II. Deriving regions of interest, coverage, and similarity using fixation maps , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[6]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[7]  Jenny Benois-Pineau,et al.  Scene similarity measure for video content segmentation in the framework of a rough indexing paradigm , 2006, Int. J. Intell. Syst..

[8]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[9]  Jenny Benois-Pineau,et al.  Detection of moving foreground objects in videos with strong camera motion , 2011, Pattern Analysis and Applications.

[10]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[11]  Nathalie Guyader,et al.  Modelling Spatio-Temporal Saliency to Predict Gaze Direction for Short Videos , 2009, International Journal of Computer Vision.

[12]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[13]  Dominique Barba,et al.  Cartes de Saillance Spatio-Temporelle basées Contrastes de Couleur et Mouvement Relatif , 2009 .

[14]  O. Meur,et al.  Predicting visual fixations on video based on low-level visual features , 2007, Vision Research.

[15]  Thomas Martinetz,et al.  Variability of eye movements when viewing dynamic natural scenes. , 2010, Journal of vision.

[16]  L. Itti.  Author address: , 1999 .

[17]  Matthai Philipose,et al.  Egocentric recognition of handled objects: Benchmark and analysis , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[18]  Donald C. Hood,et al.  Sensitivity to Light , 1986 .

[19]  J. Todd,et al.  The effects of viewing angle, camera angle, and sign of surface curvature on the perception of three-dimensional shape from texture. , 2007, Journal of vision.

[20]  Jenny Benois-Pineau,et al.  Human Daily Activities Indexing in Videos from Wearable Cameras for Monitoring of Patients with Dementia Diseases , 2010, 2010 20th International Conference on Pattern Recognition.

[21]  Yong Jae Lee,et al.  Discovering important people and objects for egocentric video summarization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[23]  Ofer Hadar,et al.  A metric for no-reference video quality assessment for HD TV delivery based on saliency maps , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[24]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .