Applicability of RF-based methods for emotion recognition: A survey

Human emotion recognition has attracted a lot of research in recent years. However, conventional methods for sensing human emotions are either expensive or privacy intrusive. In this paper, we explore a connection between emotion recognition and RF-based activity recognition that can lead to a novel ubiquitous emotion sensing technology. We discuss the latest literature from both domains, highlight the potential of body movements for accurate emotion detection and focus on how emotion recognition could be done using inexpensive, less privacy intrusive, device-free RF sensing methods. Applications include environment and crowd behaviour tracking in real time, assisted living, health monitoring, or also domestic appliance control. As a result of this survey, we propose RF-based device free recognition for emotion detection based on body movements. However, it requires overcoming challenges, such as accuracy, to outperform classical methods.

[1]  Wioleta Szwoch,et al.  Using physiological signals for emotion recognition , 2013, International Conference on Human System Interaction.

[2]  K. Scherer,et al.  Bodily expression of emotion , 2009 .

[3]  Hairong Lv,et al.  Emotion recognition based on pressure sensor keyboards , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[4]  Elizabeth A. Crane,et al.  Motion Capture and Emotion: Affect Detection in Whole Body Movement , 2007, ACII.

[5]  Andrea Kleinsmith,et al.  Affective Body Expression Perception and Recognition: A Survey , 2013, IEEE Transactions on Affective Computing.

[6]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[7]  Dian Tjondronegoro,et al.  Facial Expression Recognition Using Facial Movement Features , 2011, IEEE Transactions on Affective Computing.

[8]  Loïc Kessous,et al.  Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis , 2010, Journal on Multimodal User Interfaces.

[9]  Marc Leman,et al.  Toward E-Motion-Based Music Retrieval a Study of Affective Gesture Recognition , 2012, IEEE Transactions on Affective Computing.

[10]  Khaled A. Harras,et al.  WiGest: A ubiquitous WiFi-based gesture recognition system , 2014, 2015 IEEE Conference on Computer Communications (INFOCOM).

[11]  Xiaonan Guo,et al.  RASS: A Real-Time, Accurate, and Scalable System for Tracking Transceiver-Free Objects , 2013, IEEE Trans. Parallel Distributed Syst..

[12]  Gang Zhou,et al.  RadioSense: Exploiting Wireless Communication Patterns for Body Sensor Network Activity Recognition , 2012, 2012 IEEE 33rd Real-Time Systems Symposium.

[13]  R. D. Walk,et al.  Perception of emotion from body posture , 1986 .

[14]  Lijun Yin,et al.  Static and dynamic 3D facial expression recognition: A comprehensive survey , 2012, Image Vis. Comput..

[15]  E. Tolosa,et al.  Structural correlates of facial emotion recognition deficits in Parkinson's disease patients , 2012, Neuropsychologia.

[16]  Gerhard Tröster,et al.  The telepathic phone: Frictionless activity recognition from WiFi-RSSI , 2014, 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[17]  Nadia Bianchi-Berthouze,et al.  Continuous Recognition of Player's Affective Body Expression as Dynamic Quality of Aesthetic Experience , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[18]  Shuangquan Wang,et al.  A review on radio based activity recognition , 2015, Digit. Commun. Networks.

[19]  Agata Kolakowska,et al.  A review of emotion recognition methods based on keystroke dynamics and mouse movements , 2013, 2013 6th International Conference on Human System Interactions (HSI).

[20]  Shashidhar G. Koolagudi,et al.  Emotion recognition from speech: a review , 2012, International Journal of Speech Technology.

[21]  Loïc Kessous,et al.  Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech , 2008, Affect and Emotion in Human-Computer Interaction.

[22]  Dana Kulic,et al.  Discriminative functional analysis of human movements , 2013, Pattern Recognit. Lett..

[23]  Moustafa Youssef,et al.  Nuzzer: A Large-Scale Device-Free Passive Localization System for Wireless Environments , 2009, IEEE Transactions on Mobile Computing.

[24]  B. Gelder,et al.  Why bodies? Twelve reasons for including bodily expressions in affective neuroscience , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[25]  Tsutomu Miyasato,et al.  Multimodal human emotion/expression recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[26]  Thierry Pun,et al.  Multimodal Emotion Recognition in Response to Videos , 2012, IEEE Transactions on Affective Computing.

[27]  J. Montepare,et al.  The Use of Body Movements and Gestures as Cues to Emotions in Younger and Older Adults , 1999 .

[28]  Mike Thelwall,et al.  Twitter, MySpace, Digg: Unsupervised Sentiment Analysis in Social Media , 2012, TIST.

[29]  Yusheng Ji,et al.  Leveraging RF-channel fluctuation for activity recognition: Active and passive systems, continuous and RSSI-based signal features , 2013, MoMM '13.

[30]  Y. Trope,et al.  Body Cues, Not Facial Expressions, Discriminate Between Intense Positive and Negative Emotions , 2012, Science.

[31]  K. Scherer,et al.  Emotion expression in body action and posture. , 2012, Emotion.

[32]  Divya Sharma,et al.  Body area networks: A survey , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[33]  Michelle Karg,et al.  Body Movements for Affective Expression: A Survey of Automatic Recognition and Generation , 2013, IEEE Transactions on Affective Computing.

[34]  Fakhri Karray,et al.  Survey on speech emotion recognition: Features, classification schemes, and databases , 2011, Pattern Recognit..

[35]  Yusheng Ji,et al.  Monitoring Attention Using Ambient FM Radio Signals , 2014, IEEE Pervasive Computing.

[36]  Hatice Gunes,et al.  Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space , 2011, IEEE Transactions on Affective Computing.

[37]  Jie Yang,et al.  E-eyes: device-free location-oriented activity identification using fine-grained WiFi signatures , 2014, MobiCom.

[38]  Khaled A. Harras,et al.  WiGest demo: A ubiquitous WiFi-based gesture recognition system , 2015, 2015 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[39]  Lionel M. Ni,et al.  RASS: A real-time, accurate and scalable system for tracking transceiver-free objects , 2011, 2011 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[40]  Lijun Yin,et al.  3D facial behaviour analysis and understanding , 2012, Image Vis. Comput..

[41]  Marc Schröder,et al.  Issues in emotion-oriented computing – towards a shared understanding , 2006 .

[42]  Arthur C. Graesser,et al.  AUTOMATIC DETECTION OF LEARNER'S AFFECT FROM GROSS BODY LANGUAGE , 2009, Appl. Artif. Intell..

[43]  S. Strejilevich,et al.  Theory of mind and facial emotion recognition in euthymic bipolar I and bipolar II disorders , 2011, Psychiatry Research.

[44]  Hatice Gunes,et al.  Bi-modal emotion recognition from expressive face and body gestures , 2007, J. Netw. Comput. Appl..

[45]  Shwetak N. Patel,et al.  Whole-home gesture recognition using wireless signals , 2013, MobiCom.

[46]  Fadel Adib,et al.  See through walls with WiFi! , 2013, SIGCOMM.

[47]  Michelle Karg,et al.  Recognition of Affect Based on Gait Patterns , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[48]  Shyamnath Gollakota,et al.  Bringing Gesture Recognition to All Devices , 2014, NSDI.

[49]  Guillaume Chanel,et al.  Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[50]  Amit P. Sheth,et al.  Harnessing Twitter "Big Data" for Automatic Emotion Identification , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[51]  F. Mast,et al.  How Mood States Affect Information Processing During Facial Emotion Recognition: An Eye Tracking Study , 2011 .