A review of emotion recognition methods based on keystroke dynamics and mouse movements

The paper describes the approach based on using standard input devices, such as keyboard and mouse, as sources of data for the recognition of users' emotional states. A number of systems applying this idea have been presented focusing on three categories of research problems, i.e. collecting and labeling training data, extracting features and training classifiers of emotions. Moreover the advantages and examples of combining standard input devices with other sources of information on human emotions have been also described. Eventually some conclusions from the review have been drawn.

[1]  Maria Virvou,et al.  Towards Improving Visual-Facial Emotion Recognition through Use of Complementary Keyboard-Stroke Pattern Information , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[2]  Mariusz Szwoch,et al.  FEEDB: A multimodal database of facial expressions and emotions , 2013, 2013 6th International Conference on Human System Interactions (HSI).

[3]  Issa Traoré,et al.  Detecting Computer Intrusions Using Behavioral Biometrics , 2005, PST.

[4]  Robert A. Sottilare,et al.  Passively Classifying Student Mood and Performance within Intelligent Tutors , 2012, J. Educ. Technol. Soc..

[5]  Michal R. Wróbel,et al.  Emotions in the software development process , 2013, 2013 6th International Conference on Human System Interactions (HSI).

[6]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[7]  Areej Alhothali,et al.  Modeling User Affect Using Interaction Events , 2011 .

[8]  R. Likert “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[9]  Claudia Picardi,et al.  Keystroke analysis of free text , 2005, TSEC.

[10]  Björn W. Schuller,et al.  Multimodal emotion recognition in audiovisual communication , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[11]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Venu Govindaraju,et al.  Behavioural biometrics: a survey and classification , 2008, Int. J. Biom..

[13]  Maja Pantic,et al.  Gaze-X: adaptive affective multimodal interface for single-user office scenarios , 2006, ICMI '06.

[14]  Hosub Lee,et al.  Towards unobtrusive emotion recognition for affective social communication , 2012, 2012 IEEE Consumer Communications and Networking Conference (CCNC).

[15]  Andrew Sears,et al.  Automated stress detection using keystroke and linguistic features: An exploratory study , 2009, Int. J. Hum. Comput. Stud..

[16]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[17]  M. Sasikumar,et al.  Recognising Emotions from Keyboard Stroke Pattern , 2010 .

[18]  Hairong Lv,et al.  Emotion recognition based on pressure sensor keyboards , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[19]  Gintautas Dzemyda,et al.  Web-based Biometric Computer Mouse Advisory System to Analyze a User's Emotions and Work Productivity , 2011, Engineering applications of artificial intelligence.

[20]  Carla E. Brodley,et al.  User re-authentication via mouse movements , 2004, VizSEC/DMSEC '04.

[21]  Fabian Monrose,et al.  Keystroke dynamics as a biometric for authentication , 2000, Future Gener. Comput. Syst..

[22]  Claudia Picardi,et al.  User authentication through keystroke dynamics , 2002, TSEC.

[23]  Regan L. Mandryk,et al.  Identifying emotional states using keystroke dynamics , 2011, CHI.