A Concept of Bimodal Visual Emotion Recognition in Computer Users

Touchless measurement of affects in computer users is gaining much interests in current man-machine interactions. In this work we present the concept of bimodal visual emotion recognition in computer users. Our idea builds on two different paradigms: a pulse detection based on face image processing and an analysis of scanpath features using eyetracking. The concept is supported by mutual correspondence of the two different methods, while both originate from a video frame sequence possibly acquired with a single sensor. Besides the novel concept, we also put forward several suggestions for future work.

[1]  Davis E. King.  Max-Margin Object Detection , 2015, ArXiv.

[2]  Piotr Augustyniak,et al.  Distant Measurement of Plethysmographic Signal in Various Lighting Conditions Using Configurable Frame-Rate Camera , 2016 .

[3]  Shaogang Gong,et al.  Dynamic Vision - From Images to Face Recognition , 2000 .

[4]  Matti Pietikäinen,et al.  Remote Heart Rate Measurement from Face Videos under Realistic Situations , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Huaizu Jiang,et al.  Face Detection with the Faster R-CNN , 2016, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[6]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[7]  Jaromir Przybylo,et al.  Vision Based Facial Action Recognition System for People with Disabilities , 2012, ITIB.

[8]  Rosalind W. Picard,et al.  Non-contact, automated cardiac pulse measurements using video imaging and blind source separation , 2022 .

[9]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[11]  Zhengyou Zhang,et al.  A Survey of Recent Advances in Face Detection , 2010 .

[12]  Piotr Augustyniak,et al.  Eyetracking-based assessment of affect-related decay of human performance in visual tasks , 2018, Future Gener. Comput. Syst..

[13]  L. O. Svaasand,et al.  Remote plethysmographic imaging using ambient light. , 2008, Optics express.

[14]  Stefanos Zafeiriou,et al.  A survey on face detection in the wild: Past, present and future , 2015, Comput. Vis. Image Underst..