What Makes a Beautiful Landscape Beautiful: Adjective Noun Pairs Attention by Eye-Tracking and Gaze Analysis

This paper asks the questions: what makes a beautiful landscape beautiful, what makes a damaged building look damaged? It tackles the challenge to understand which regions of Adjective Noun Pairs (ANP) images attract attention when observed by a human subject. We employ eye-tracking techniques to record the gaze over a set of multiple ANPs images and derive regions of interests for these ANPs. Our contribution is to study eye fixation pattern in the context of ANPs and their characteristics between being objective or subjective on the one hand and holistic vs. localizable on the other hand. Our finding indicate that subjects who differ in their assessment of ANP labels also have different eye fixation pattern. Further, we provide insights about ANP attention during implicit and explicit ANP assessment.

[1]  Andreas Dengel,et al.  Automatic Detection of CSA Media by Multi-modal Feature Fusion for Law Enforcement Support , 2014, ICMR.

[2]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[3]  Yifan Peng,et al.  Studying Relationships between Human Gaze, Description, and Computer Vision , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Tao Chen,et al.  DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks , 2014, ArXiv.

[5]  Deb Roy,et al.  Visual memory augmentation: using eye gaze as an attention filter , 2004, Eighth International Symposium on Wearable Computers.

[6]  Dong Liu,et al.  Towards a comprehensive computational model foraesthetic assessment of videos , 2013, MM '13.

[7]  Jiebo Luo,et al.  Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks , 2015, AAAI.

[8]  Adrian Ulges,et al.  Visual Concept Learning from Weakly Labeled Web Videos , 2010, Video Search and Mining.

[9]  Rongrong Ji,et al.  SentiBank: large-scale ontology and classifiers for detecting sentiment and emotions in visual content , 2013, ACM Multimedia.

[10]  Andreas Dengel,et al.  A robust realtime reading-skimming classifier , 2012, ETRA.

[11]  Tao Chen,et al.  Object-Based Visual Sentiment Concept Analysis and Application , 2014, ACM Multimedia.

[12]  Rongrong Ji,et al.  Large-scale visual sentiment ontology and detectors using adjective noun pairs , 2013, ACM Multimedia.

[13]  Li-Jia Li,et al.  Visual Sentiment Prediction with Deep Convolutional Neural Networks , 2014, ArXiv.