Discriminating affective state intensity using physiological responses

In this paper we explore if physiological signals obtained from a person through wearable sensors permit the correct interpretation of human affective states. A crucial aspect within this field of research is the capability of inducing real-life emotions in laboratory environments. To this end we designed a very strict and regulated experimental protocol. We consider two affective states: a relaxation state and a stressful one. To induce these two states we adopt audio tracks of natural or day-life sounds and math calculations. Moreover, to study different intensities of effectively induced relaxation, we consider two different audio players: a traditional pair of headphones and the Spherison Sound6D pillowⒸ, a special device that provides a complete spherical immersion in what users are listening to. We consider as physiological signals the Galvanic Skin Response (GSR) and the Photoplethysmography (PPG), as they are sensitive measures for emotional arousal. After an extensive analysis on our experimental data, we demonstrate that GSR and PPG signals can successfully distinguish relaxation and stressful states. Moreover, the same physiological signals can discriminate affective state intensity, especially when relaxation is induced adopting the Sound6D technology.

[1]  Stefan Winkler,et al.  ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors , 2018, IEEE Transactions on Affective Computing.

[2]  Francesca Gasparini,et al.  Cognitive and Physiological Response for Health Monitoring in an Ageing Population: A Multi-modal System , 2019, INSCI.

[3]  G. Scott Vercoe,et al.  The affective remixer: personalized music arranging , 2006, CHI Extended Abstracts.

[4]  Lanlan Chen,et al.  Detecting driving stress in physiological signals based on multimodal feature analysis and kernel classifiers , 2017, Expert Syst. Appl..

[5]  Antonio Artés-Rodríguez,et al.  Individual performance calibration using physiological stress signals , 2015, ArXiv.

[6]  Febus Reidj G. Cruz,et al.  Portable Stress Level Detector based on Galvanic Skin Response, Heart Rate, and Body Temperature , 2018, 2018 IEEE 10th International Conference on Humanoid, Nanotechnology, Information Technology,Communication and Control, Environment and Management (HNICEM).

[7]  Akane Sano,et al.  Stress Recognition Using Wearable Sensors and Mobile Phones , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[8]  S. Wiens,et al.  Stress Recovery during Exposure to Nature Sound and Environmental Noise , 2010, International journal of environmental research and public health.

[9]  Tahira Reid,et al.  Dynamic Data Driven Approach for Modeling Human Error , 2015, ICCS.

[10]  J. Russell,et al.  An approach to environmental psychology , 1974 .

[11]  Christine L. Lisetti,et al.  Emotion recognition from physiological signals using wireless sensors for presence technologies , 2004, Cognition, Technology & Work.

[12]  Emer P. Doheny,et al.  SHIMMER™: An extensible platform for physiological signal capture , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[13]  Alfredo Raglio,et al.  Music Therapy Interventions in Parkinson’s Disease: The State-of-the-Art , 2015, Front. Neurol..

[14]  Nilanjan Dey,et al.  Medical cyber-physical systems: A survey , 2018, Journal of Medical Systems.

[15]  Cem Ersoy,et al.  Stress detection in daily life scenarios using smart phones and wearable sensors: A survey , 2019, J. Biomed. Informatics.

[16]  Stefania Bandini,et al.  Personalized Music Experience for the Wellbeing of Elderly People , 2019, INSCI.

[17]  Paulo Menezes,et al.  Human Emotions and Physiological Signals: A Classroom Experiment , 2016, Int. J. Online Eng..

[18]  Nicole Novielli,et al.  Emotion detection using noninvasive low cost sensors , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII).

[19]  Rosalind W. Picard Affective Computing for HCI , 1999, HCI.

[20]  Egon L. van den Broek,et al.  Tune in to your emotions: a robust personalized affective music player , 2012, User Modeling and User-Adapted Interaction.

[21]  Seyed Kamaledin Setarehdan,et al.  Stress assessment by means of heart rate derived from functional near-infrared spectroscopy , 2018, Journal of biomedical optics.

[22]  Manfred Clynes,et al.  Sentics: The touch of emotions , 1977 .

[23]  Aamir Saeed Malik,et al.  Mitigation of stress: new treatment alternatives , 2018, Cognitive Neurodynamics.

[24]  A. Grabowska,et al.  The Nencki Affective Picture System (NAPS): Introduction to a novel, standardized, wide-range, high-quality, realistic picture database , 2013, Behavior research methods.

[25]  Enzo Pasquale Scilingo,et al.  Recognizing Emotions Induced by Affective Sounds through Heart Rate Variability , 2015, IEEE Transactions on Affective Computing.

[26]  Konstantinos Trochidis,et al.  Modeling Affective Responses to Music Using Audio Signal Analysis and Physiology , 2015, CMMR.

[27]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[28]  José Manuel Pastor,et al.  Electrodermal Activity Sensor for Classification of Calm/Distress Condition , 2017, Sensors.

[29]  O. V. Ramana Murthy,et al.  Stress Detection in Working People , 2017 .

[30]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[31]  Subramanian Ramanathan,et al.  DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses , 2015, IEEE Transactions on Affective Computing.

[32]  Wessel Kraaij,et al.  The SWELL Knowledge Work Dataset for Stress and User Modeling Research , 2014, ICMI.

[33]  Elmar Nöth,et al.  Real-time Recognition of the Affective User State with Physiological Signals , 2022 .

[34]  Johannes Wagner,et al.  From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[35]  J. Russell A circumplex model of affect. , 1980 .

[36]  N. Cliff Dominance statistics: Ordinal analyses to answer ordinal questions. , 1993 .

[37]  Li-Wei Ko,et al.  Develop a personalized intelligent music selection system based on heart rate variability and machine learning , 2017, Multimedia Tools and Applications.

[38]  K. Scherer,et al.  The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significance , 2011, Behavior research methods.

[39]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2012, Springer Berlin Heidelberg.

[40]  Hayley Guess,et al.  Alzheimer’s disease and the impact of music therapy a systematic literature review , 2017 .

[41]  Changwoo Yoon,et al.  Multilevel mental stress detection using ultra-short pulse rate variability series , 2020, Biomed. Signal Process. Control..

[42]  Jorge Gonçalves,et al.  Measuring the Effects of Stress on Mobile Interaction , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[43]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[44]  Stefania Bandini,et al.  Towards Affective Walkability for Healthy Ageing in the Future of the Cities , 2019, AI*AAL@AI*IA.

[45]  R. Plutchik A GENERAL PSYCHOEVOLUTIONARY THEORY OF EMOTION , 1980 .

[46]  P. Lang International Affective Picture System (IAPS) : Technical Manual and Affective Ratings , 1995 .

[47]  Monique,et al.  The Effect of Music Therapy for Patients With Huntington's Disease: A Systematic Literature Review , 2015 .

[48]  A. Vargha,et al.  A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong , 2000 .

[49]  P. Ekman An argument for basic emotions , 1992 .

[50]  D. Watson,et al.  Toward a consensual structure of mood. , 1985, Psychological bulletin.

[51]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[52]  Juan C. Quiroz,et al.  Emotion Recognition Using Smart Watch Sensor Data: Mixed-Design Study , 2018, JMIR mental health.

[53]  Gerhard Tröster,et al.  Discriminating Stress From Cognitive Load Using a Wearable EDA Device , 2010, IEEE Transactions on Information Technology in Biomedicine.

[54]  Javier Hernandez,et al.  Call Center Stress Recognition with Person-Specific Models , 2011, ACII.