Visual interestingness in image sequences

Interestingness is said to be the power of attracting or holding one's attention (because something is unusual or exciting, etc.). We, as humans, have the great capacity to direct our visual attention and judge the interestingness of a scene. Consider for example the image sequence in the figure on the right. The spider in front of the camera or the snow on the lens are examples of events that deviate from the context since they violate the expectations, and therefore are considered interesting. On the other hand, weather changes or a camera shift, do not raise human attention considerably, even though large regions of the image are influenced. In this work we firstly investigate what humans consider as "interesting" in image sequences. Secondly we propose a computer vision algorithm to automatically spot these interesting events. To this end, we integrate multiple cues inspired by cognitive concepts and discuss why and to what extent the automatic discovery of visual interestingness is possible.

[1]  Allan Hanbury,et al.  Affective image classification using features inspired by psychology and art theory , 2010, ACM Multimedia.

[2]  David C. Hogg,et al.  Learning the Distribution of Object Trajectories for Event Recognition , 1995, BMVC.

[3]  A. Chen,et al.  An examination of situational interest and its sources. , 2001, The British journal of educational psychology.

[4]  L. Itti,et al.  A brief and selective history of attention , 2005 .

[5]  Luc Van Gool,et al.  Hunting Nessie - Real-time abnormality detection from webcams , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[6]  J. Henderson.  Human gaze control during real-world scene perception , 2003, Trends in Cognitive Sciences.

[7]  Jianxiong Xiao,et al.  What makes an image memorable? , 2011, CVPR 2011.

[8]  Robert Pless,et al.  Consistent Temporal Variations in Many Outdoor Scenes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[10]  Luc Van Gool,et al.  Temporal Relations in Videos for Unsupervised Activity Analysis , 2011, BMVC.

[11]  Walter Schneider,et al.  Controlled and Automatic Human Information Processing: 1. Detection, Search, and Attention. , 1977 .

[12]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[13]  Daphna Weinshall,et al.  Beyond Novelty Detection: Incongruent Events, When General and Specific Classifiers Disagree , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  P. Silvia.  Exploring the Psychology of Interest , 2006 .

[15]  Tom Schaul,et al.  Coherence Progress: A Measure of Interestingness Based on Fixed Compressors , 2011, AGI.

[16]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[17]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[18]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[19]  Jürgen Schmidhuber,et al.  Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes , 2008, ABiALS.

[20]  P. Valdez,et al.  Effects of color on emotions. , 1994, Journal of experimental psychology. General.

[21]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[22]  Pierre Baldi,et al.  A principled approach to detecting surprising events in video , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Vicente Ordonez,et al.  High level describable attributes for predicting aesthetics and interestingness , 2011, CVPR 2011.

[24]  W. McD..  Grundzüge der physiologischen Psychologie , 1902, Nature.

[25]  Harry W. Agius,et al.  Video summarisation: A conceptual framework and survey of the state of the art , 2008, J. Vis. Commun. Image Represent..

[26]  D. Berlyne.  Conflict, arousal, and curiosity , 2014 .

[27]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[28]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[30]  Erik G. Learned-Miller,et al.  Online domain adaptation of a pre-trained cascade of classifiers , 2011, CVPR 2011.

[31]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[32]  Jianbo Shi,et al.  Detecting unusual activity in video , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..