A knowledge discovery and visualisation method for unearthing emotional states from physiological data

In this paper we propose a knowledge discovery and visualisation method for unearthing emotional states from physiological data typically available from wearable devices. In addition we investigate the viability of using a limited set of wearable sensors to extract decision tree rules which are representative of physiological changes taking place during emotional changes. Our method utilised a fusion of pre-processing and classification techniques using decision trees to discover logic rules relating to the valence and arousal emotional dimensions. This approach normalised the signal data in a manner that enabled accurate classification and generated logic rules for knowledge discovery. Furthermore, the use of three target classes for the emotional dimensions was effective at denoising the data and further enhancing classification and useful rule extraction. There are three key contributions in this work, firstly an exploration and validation of our knowledge discovery methodology, secondly successful extraction of high accuracy rules derived from physiological data and thirdly knowledge discovery and visualisation of relationships within-participant physiological data that can be inferred relating to emotions. Additionally, this work may be utilised in areas such as the medical sciences where interpretable rules are required for knowledge discovery.

[1]  David I W Phillips,et al.  Heart-rate and blood-pressure variability during psychophysiological tasks involving speech: influence of respiration. , 2007, Psychophysiology.

[2]  Rosalind W. Picard Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..

[3]  Franziska Völckner,et al.  Temperature and emotions: Effects of physical temperature on responses to emotional advertising , 2017 .

[4]  Kenneth McGarry,et al.  A survey of interestingness measures for knowledge discovery , 2005, The Knowledge Engineering Review.

[5]  Peter Robinson,et al.  Real-Time Recognition of Affective States from Nonverbal Features of Speech and Its Application for Public Speaking Skill Analysis , 2011, IEEE Transactions on Affective Computing.

[6]  Hussein A. Abbass,et al.  Introducing Data Mining and Knowledge Discovery , 2002 .

[7]  David Tien,et al.  Data mining and knowledge discovery from physiological sensors , 2019, PETRA.

[8]  Rafael A. Calvo,et al.  Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications , 2010, IEEE Transactions on Affective Computing.

[9]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[10]  Elena Smets,et al.  Large-scale wearable data reveal digital phenotypes for daily-life stress detection , 2018, npj Digital Medicine.

[11]  Terence K. L. Hui,et al.  Coverage of Emotion Recognition for Common Wearable Biosensors , 2018, Biosensors.

[12]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[13]  Pasin Israsena,et al.  EEG-Based Emotion Recognition Using Deep Learning Network with Principal Component Based Covariate Shift Adaptation , 2014, TheScientificWorldJournal.

[14]  R. Adolphs Neural systems for recognizing emotion , 2002, Current Opinion in Neurobiology.

[15]  Md Zahidul Islam,et al.  Measuring Rule Retention in Anonymized Data - When One Measure Is Not Enough , 2017, Trans. Data Priv..

[16]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[17]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Arun Ross,et al.  Score normalization in multimodal biometric systems , 2005, Pattern Recognit..

[19]  Md Zahidul Islam,et al.  ForEx++: A New Framework for Knowledge Discovery from Decision Forests , 2017, Australas. J. Inf. Syst..

[20]  Marta Zaraska The Warmth of Friendship, the Chill of Betrayal , 2017 .

[21]  E. Fallen,et al.  Hidden rhythms in the heart rate record: a primer on neurocardiology. , 2000, Clinical and investigative medicine. Medecine clinique et experimentale.

[22]  Olga Sourina,et al.  Real-time EEG-based emotion monitoring using stable features , 2015, The Visual Computer.

[23]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[24]  J. Ruiz-Rodríguez,et al.  Innovative continuous non-invasive cuffless blood pressure monitoring based on photoplethysmography technology , 2013, Intensive Care Medicine.

[25]  Dan Wang,et al.  Modeling Physiological Data with Deep Belief Networks. , 2013, International journal of information and education technology.

[26]  Akash Kumar Bhoi,et al.  Fingertip Pulse Wave (PPG signal) Analysis and Heart Rate Detection , 2012 .

[27]  Mohammad H. Mahoor,et al.  DISFA: A Spontaneous Facial Action Intensity Database , 2013, IEEE Transactions on Affective Computing.

[28]  H. Critchley Review: Electrodermal Responses: What Happens in the Brain , 2002 .

[29]  Amy Loutfi,et al.  Data Mining for Wearable Sensors in Health Monitoring Systems: A Review of Recent Trends and Challenges , 2013, Sensors.

[30]  Turgay Kaya,et al.  Wavelet-based study of valence–arousal model of emotions on EEG signals with LabVIEW , 2016, Brain Informatics.

[31]  Howard J. Hamilton,et al.  Interestingness measures for data mining: A survey , 2006, CSUR.