Emotion Recognition Using Wearables: A Systematic Literature Review - Work-in-progress

Wearables like smartwatches or wrist bands equipped with pervasive sensors enable us to monitor our physiological signals. In this study, we address the question whether they can help us to recognize our emotions in our everyday life for ubiquitous computing. Using the systematic literature review, we identified crucial research steps and discussed the main limitations and problems in the domain.

[1]  D. Kahneman.  Thinking, Fast and Slow , 2011 .

[2]  Michele Magno,et al.  DeepEmote: Towards multi-layer neural networks in a low power wearable multi-sensors bracelet , 2017, 2017 7th IEEE International Workshop on Advances in Sensors and Interfaces (IWASI).

[3]  Kristof Van Laerhoven,et al.  Wearable-Based Affect Recognition—A Review , 2019, Sensors.

[4]  Antonio Fernández-Caballero,et al.  Emotion Detection in Aging Adults Through Continuous Monitoring of Electro-Dermal Activity and Heart-Rate Variability , 2019, IWINAC.

[5]  Tao Xu,et al.  Learning Emotions EEG-based Recognition and Brain Activity: A Survey Study on BCI for Intelligent Tutoring System , 2018, ANT/SEIT.

[6]  Eman M. G. Younis,et al.  Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection , 2019, Inf. Fusion.

[7]  Michiko Ohkura,et al.  Emotion monitoring from physiological signals for service robots in the living space , 2010, ICCAS 2010.

[8]  Zhu Wang,et al.  EmotionSense: Emotion Recognition Based on Wearable Wristband , 2018, 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[9]  Ricardo Gutierrez-Osuna,et al.  Development and Evaluation of an Ambulatory Stress Monitor Based on Wearable Sensors , 2012, IEEE Transactions on Information Technology in Biomedicine.

[10]  Han Wen Guo,et al.  Short-term analysis of heart rate variability for emotion recognition via a wearable ECG device , 2015, 2015 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS).

[11]  L. Bai,et al.  The Development of Native Chinese Affective Picture System-A pretest in 46 College Students , 2005 .

[12]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[13]  R. Plutchik.  Emotions and Life: Perspectives from Psychology, Biology, and Evolution , 2002 .

[14]  Abdulmotaleb El Saddik,et al.  iAware: A Real-Time Emotional Biofeedback System Based on Physiological Signals , 2018, IEEE Access.

[15]  Corinne Ancourt,et al.  Survey on AI-Based Multimodal Methods for Emotion Detection , 2019, High-Performance Modelling and Simulation for Big Data Applications.

[16]  Bernardo Nugroho Yahya,et al.  A Framework for Real Time Emotion Recognition Based on Human ANS Using Pervasive Device , 2018, 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC).

[17]  Xi Lu,et al.  "It sounds like she is sad": Introducing a Biosensing Prototype that Transforms Emotions into Real-time Music and Facilitates Social Interaction , 2019, CHI Extended Abstracts.

[18]  Alan S. Cowen,et al.  Self-report captures 27 distinct categories of emotion bridged by continuous gradients , 2017, Proceedings of the National Academy of Sciences.

[19]  R. Lazarus.  Emotion and Adaptation , 1991 .

[20]  Carlos Carrascosa,et al.  Intelligent Wristbands for the Automatic Detection of Emotional States for the Elderly , 2018, IDEAL.

[21]  Cheng He,et al.  An Emotion Recognition System Based on Physiological Signals Obtained by Wearable Sensors , 2017 .

[22]  David M. Fresco,et al.  Emotions and Life: Perspectives from Psychology, Biology, and Evolution , 2005 .

[23]  Carlos Carrascosa,et al.  Emotions Detection on an Ambient Intelligent System Using Wearable Devices , 2019, AfCAI.

[24]  Jennifer Healey,et al.  Affsens: a mobile platform for capturing affect in context , 2018, MobileHCI Adjunct.

[25]  David Pollreisz,et al.  A simple algorithm for emotion recognition, using physiological signals of a smart watch , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[26]  Sten Hanke,et al.  Emotion Recognition from Physiological Signal Analysis: A Review , 2019, BRAINS/WS-AFFIN@AmI.

[27]  Yang Wu,et al.  Emotional Evaluation Based on SVM , 2017 .

[28]  Amir Jalilifard,et al.  Brain-TV connection: Toward establishing emotional connection with smart TVs , 2017, 2017 IEEE Region 10 Humanitarian Technology Conference (R10-HTC).

[29]  Xiangmin Xu,et al.  Emotion Recognition Using Frontal EEG in VR Affective Scenes , 2019, 2019 IEEE MTT-S International Microwave Biomedical Conference (IMBioC).

[30]  Kristof Van Laerhoven,et al.  Labelling Affective States "in the Wild": Practical Guidelines and Lessons Learned , 2018, UbiComp/ISWC Adjunct.

[31]  Nicu Sebe,et al.  A Quality Adaptive Multimodal Affect Recognition System for User-Centric Multimedia Indexing , 2016, ICMR.

[32]  A. Grabowska,et al.  The Nencki Affective Picture System (NAPS): Introduction to a novel, standardized, wide-range, high-quality, realistic picture database , 2013, Behavior research methods.

[33]  Maulik R. Kamdar,et al.  PRISM: A Data-Driven Platform for Monitoring Mental Health , 2016, PSB.

[34]  M. Z. Soroush,et al.  A Review on EEG Signals Based Emotion Recognition , 2017 .

[35]  Renlai Zhou,et al.  Gender Differences in Emotional Response: Inconsistency between Experience and Expressivity , 2016, PloS one.

[36]  L. Fernández-Aguilar,et al.  Emotional Differences in Young and Older Adults: Films as Mood Induction Procedure , 2018, Front. Psychol..

[37]  Andry Rakotonirainy,et al.  Long Short Term Memory Hyperparameter Optimization for a Neural Network Based Emotion Recognition Framework , 2018, IEEE Access.

[38]  Nico Pallamin,et al.  Emotion Recognition Using Physiological Signals: Laboratory vs. Wearable Sensors , 2017, AHFE.

[39]  Grzegorz J. Nalepa,et al.  Analysis and Use of the Emotional Context with Wearable Devices for Games and Intelligent Assistants , 2019, Sensors.

[40]  Christine L. Lisetti,et al.  Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals , 2004, EURASIP J. Adv. Signal Process..

[41]  Makoto Mizukawa,et al.  Emotion Recognition Using Biological Signal in Intelligent Space , 2011, HCI.

[42]  Kristof Van Laerhoven,et al.  Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection , 2018, ICMI.

[43]  Mohammad H. Mahoor,et al.  A wavelet-based approach to emotion classification using EDA signals , 2018, Expert Syst. Appl..

[44]  José Manuel Pastor,et al.  A Distributed Architecture for Multimodal Emotion Identification , 2015, PAAMS.

[45]  Min Chen,et al.  SCAI-SVSC: Smart clothing for effective interaction with a sustainable vital sign collection , 2018, Future Gener. Comput. Syst..

[46]  Tadashi Okoshi,et al.  Comparing ESM Timings for Emotional Estimation Model with Fine Temporal Granularity , 2018, UbiComp/ISWC Adjunct.

[47]  J. Lerner,et al.  Emotion and decision making. , 2015, Annual review of psychology.

[48]  Minh-Son Dao,et al.  HealthyClassroom - A Proof-of-Concept Study for Discovering Students' Daily Moods and Classroom Emotions to Enhance a Learning-teaching Process using Heterogeneous Sensors , 2018, ICPRAM.

[49]  Marco Maier,et al.  DeepFlow: Detecting Optimal User Experience From Physiological Data Using Deep Neural Networks , 2019, AAMAS.

[50]  Christine L. Lisetti,et al.  Categorizing Autonomic Nervous System (ANS) Emotional Signals using Bio-Sensors for HRI within the MAUI Paradigm , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[51]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[52]  B.A. Kitchenham,et al.  Systematic Reviews , 2011, Diabetes Care.

[53]  Zhenqi Li,et al.  A Review of Emotion Recognition Using Physiological Signals , 2018, Sensors.

[54]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.