Attention Monitoring Based on Temporal Signal-Behavior Structures

In this paper, we discuss our system that estimates user attention to displayed content signals with temporal analysis of their exhibited behavior. Detecting user attention and controlling contents are key issues in our “networked interaction therapy system,” which effectively attracts the attention of memory-impaired people. In our proposed system, user behavior, including facial movements and body motions (“beat actions”), is detected with vision-based methods. User attention to the displayed content is then estimated based on the on/off facial orientation from a display system and body motions synchronous to auditorial signals. This attention monitoring mechanism design is derived from observations of actual patients. Estimated attention level can be used for content control to attract more attention of the viewers to the display system. Experimental results suggest that the content switching mechanism effectively attracts user interest.

[1]  Shigeru Akamatsu,et al.  Automatic detection of human faces in natural scene images by use of a skin color model and of invariant moments , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[2]  Jakub Segen,et al.  A camera-based system for tracking people in real time , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[3]  N. Tetsutani,et al.  Real-time Detection of Between-the-Eyes with a Circle Frequency Filter , 2002 .

[4]  Masataka Goto,et al.  An Audio-based Real-time Beat Tracking System for Music With or Without Drum-sounds , 2001 .

[5]  Atsushi Nakazawa,et al.  Detecting dance motion structure through music analysis , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[6]  N. Tetsutani,et al.  Networked Interaction Therapy: Relieving Stress in Memory-Impaired People and Their Family Members , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[7]  Dmitry O. Gorodnichy,et al.  On importance of nose for face tracking , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[8]  Alexander Zelinsky,et al.  3-D facial pose and gaze point estimation using a robust real-time tracking paradigm , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[9]  Jake K. Aggarwal,et al.  Tracking human motion using multiple cameras , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[10]  Janice Loh,et al.  Technology applied to address difficulties of Alzheimer patients and their partners , 2004 .

[11]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Eric D. Scheirer,et al.  Tempo and beat analysis of acoustic musical signals. , 1998, The Journal of the Acoustical Society of America.

[13]  Alexander H. Waibel,et al.  A real-time face tracker , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[14]  Berry Eggen,et al.  Proceedings of the conference on Dutch directions in HCI , 2004 .