Estimation of Attentiveness of People Watching TV Based on Their Emotional Behaviors

We propose a method for estimating the attentiveness of viewers watching TV on the basis of their emotional behaviors. Such behaviors are usually hard to perceive and have in the past required contact-type equipment to measure them. We devised a novel attentiveness estimator that uses three visual features that are correlated with emotional behaviors: the viewer's head motion, blink interval, and eye movement. The main contribution of this paper is that the method automatically obtains these features from ordinary noncontact-type sensors. In addition, the method considers not only the viewer's emotional behaviors but also content features which affect the behaviors. The temporal window pattern in a video sequence is used to describe temporally local features. Experimental results indicate the possibility of recognizing TV viewers' attentiveness from certain visible emotional behaviors measured by common home electronics.

[1]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[2]  L. Itti,et al.  Visual causes versus correlates of attentional selection in dynamic scenes , 2006, Vision Research.

[3]  Daniel McDuff,et al.  Predicting online media effectiveness based on smile responses gathered over the Internet , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[4]  D. Morris.  Manwatching : a field guide to human behaviour , 1977 .

[5]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[6]  Simon Dobri.  Emotion Recognition using Linear Transformations in Combination with Video , 2009 .

[7]  Björn W. Schuller,et al.  Emotion representation, analysis and synthesis in continuous space: A survey , 2011, Face and Gesture 2011.

[8]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Surendra Ranganath,et al.  Robust Attentive Behavior Detection by Non-linear Head Pose Embedding and Estimation , 2006, ECCV.

[10]  Yunde Jia,et al.  Audio-visual emotion recognition using Boltzmann Zippers , 2012, ICIP.

[11]  Christof Koch,et al.  Predicting human gaze using low-level saliency combined with face detection , 2007, NIPS.

[12]  Miwa Nakanishi,et al.  User Characteristic-Based Information-Providing Service for Museum with Optical See-Through Head-Mounted Display: Does It Evoke Enthusiasm? , 2011, HCI.

[13]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  R. S. Ramakrishna,et al.  Vision-based eye-gaze tracking for human computer interface , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[15]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  M. Argyle.  Bodily communication, 2nd ed. , 1988 .

[17]  Keiichi Kitajo,et al.  Synchronization of spontaneous eyeblinks while viewing video stories , 2009, Proceedings of the Royal Society B: Biological Sciences.

[18]  Albert Ali Salah,et al.  Design and implementation of an affect-responsive interactive photo frame , 2011, Journal on Multimodal User Interfaces.

[19]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.