Feature Extraction and Selection for Pain Recognition Using Peripheral Physiological Signals

In pattern recognition, the selection of appropriate features is paramount to both the performance and the robustness of the system. Over-reliance on machine learning-based feature selection methods can, therefore, be problematic; especially when conducted using small snapshots of data. The results of these methods, if adopted without proper interpretation, can lead to sub-optimal system design or worse, the abandonment of otherwise viable and important features. In this work, a deep exploration of pain-based emotion classification was conducted to better understand differences in the results of the related literature. In total, 155 different time domain and frequency domain features were explored, derived from electromyogram (EMG), skin conductance levels (SCL), and electrocardiogram (ECG) readings taken from the 85 subjects in response to heat-induced pain. To address the inconsistency in the optimal feature sets found in related works, an exhaustive and interpretable feature selection protocol was followed to obtain a generalizable feature set. Associations between features were then visualized using a topologically-informed chart, called Mapper, of this physiological feature space, including synthesis and comparison of results from previous literature. This topological feature chart was able to identify key sources of information that led to the formation of five main functional feature groups: signal amplitude and power, frequency information, nonlinear complexity, unique, and connecting. These functional groupings were used to extract further insight into observable autonomic responses to pain through a complementary statistical interaction analysis. From this chart, it was observed that EMG and SCL derived features could functionally replace those obtained from ECG. These insights motivate future work on novel sensing modalities, feature design, deep learning approaches, and dimensionality reduction techniques.

[1]  M. Bradley,et al.  Affective reactions to acoustic stimuli. , 2000, Psychophysiology.

[2]  P. Y. Lum,et al.  Extracting insights from the shape of complex data using topology , 2013, Scientific Reports.

[3]  Rosalind W. Picard,et al.  Multi-task neural networks for personalized pain recognition from physiological signals , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW).

[4]  M. Bradley,et al.  Emotion, Motivation, and Anxiety: Brain Mechanisms and Psychophysiology the Motivational Organization of Emotion Patterns of Human Emotion Emotion and Perception the Psychophysiology of Picture Processing Neural Imaging: Motivation in the Visual Cortex Motivational Circuits in the Brain , 2022 .

[5]  M. Opper,et al.  Statistical mechanics of Support Vector networks. , 1998, cond-mat/9811421.

[6]  Facundo Mémoli,et al.  Topological Methods for the Analysis of High Dimensional Data Sets and 3D Object Recognition , 2007, PBG@Eurographics.

[7]  Abdul Wahab,et al.  EEG analysis for understanding stress based on affective model basis function , 2011, 2011 IEEE 15th International Symposium on Consumer Electronics (ISCE).

[8]  Gustavo Moreira da Silva,et al.  Automatic pain quantification using autonomic parameters , 2014 .

[9]  Pornchai Phukpattaranont,et al.  Feature reduction and selection for EMG signal classification , 2012, Expert Syst. Appl..

[10]  Zhenqi Li,et al.  A Review of Emotion Recognition Using Physiological Signals , 2018, Sensors.

[11]  M. Bradley,et al.  Brain potentials in affective picture processing: covariation with autonomic arousal and affective report , 2000, Biological Psychology.

[12]  Karl J. Friston,et al.  Analytic measures for quantification of arousal from spontaneous skin conductance fluctuations , 2010, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[13]  Ayoub Al-Hamadi,et al.  Automatic Pain Recognition from Video and Biomedical Signals , 2014, 2014 22nd International Conference on Pattern Recognition.

[14]  Pablo Laguna,et al.  Human emotion recognition using heart rate variability analysis with spectral bands based on respiration , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[15]  T. Wager,et al.  Distinct Brain Systems Mediate the Effects of Nociceptive Input and Self-Regulation on Pain , 2015, PLoS biology.

[16]  F. Shaffer,et al.  An Overview of Heart Rate Variability Metrics and Norms , 2017, Front. Public Health.

[17]  Mahyar Hamedi,et al.  Robust Facial Expression Recognition for MuCI: A Comprehensive Neuromuscular Signal Analysis , 2018, IEEE Transactions on Affective Computing.

[18]  Aamir Saeed Malik,et al.  Pupillary behavior in positive and negative emotions , 2013, 2013 IEEE International Conference on Signal and Image Processing Applications.

[19]  Mahesh Sooriyabandara,et al.  HealthyOffice: Mood recognition at work using smartphones and wearable sensors , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[20]  Sylvia D. Kreibig,et al.  Autonomic nervous system activity in emotion: A review , 2010, Biological Psychology.

[21]  Patrick Thiam,et al.  Adaptive confidence learning for the personalization of pain intensity estimation systems , 2017, Evol. Syst..

[22]  Edmund C. Lalor,et al.  Estimation of coherence using the median is robust against EEG artefacts , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[23]  A. Barreto,et al.  Stress Detection in Computer Users Based on Digital Signal Processing of Noninvasive Physiological Variables , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[24]  J. H. Ward Hierarchical Grouping to Optimize an Objective Function , 1963 .

[25]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[26]  P. Petta,et al.  Computational models of emotion , 2010 .

[27]  C. Darwin The Expression of the Emotions in Man and Animals , .

[28]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[29]  Kevin B. Englehart,et al.  A wavelet-based continuous classification scheme for multifunction myoelectric control , 2001, IEEE Transactions on Biomedical Engineering.

[30]  A. Phinyomark,et al.  Application of Wavelet Analysis in EMG Feature Extraction for Pattern Classification , 2011 .

[31]  Subramanian Ramanathan,et al.  DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses , 2015, IEEE Transactions on Affective Computing.

[32]  H. Traue,et al.  Pain Intensity Recognition Rates via Biopotential Feature Patterns with Support Vector Machines , 2015, PloS one.

[33]  Xiang Ji,et al.  Arousal Recognition Using Audio-Visual Features and FMRI-Based Brain Response , 2015, IEEE Transactions on Affective Computing.

[34]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[35]  Angkoon Phinyomark,et al.  EMG feature evaluation for improving myoelectric pattern recognition robustness , 2013, Expert Syst. Appl..

[36]  W. Hubert,et al.  Autonomic, neuroendocrine, and subjective responses to emotion-inducing film stimuli. , 1991, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[37]  Zhizhong Wang,et al.  Detecting nonlinearity of action surface EMG signal , 2001 .

[38]  Leontios J. Hadjileontiadis,et al.  Emotion Recognition From EEG Using Higher Order Crossings , 2010, IEEE Transactions on Information Technology in Biomedicine.

[39]  Zahra Khalili,et al.  Emotion recognition system using brain and peripheral signals: Using correlation dimension to improve the results of EEG , 2009, 2009 International Joint Conference on Neural Networks.

[40]  R. Boostani,et al.  A Novel Classification Strategy to Distinguish Five Levels of Pain Using the EEG Signal Features , 2018, IEEE Transactions on Affective Computing.

[41]  H.L. Kennedy,et al.  A New Statistical Measure of Signal Similarity , 2007, 2007 Information, Decision and Control.

[42]  Angkoon Phinyomark,et al.  Topological Data Analysis of Biomedical Big Data , 2018, Signal Processing and Machine Learning for Biomedical Big Data.

[43]  Arthur C. Graesser,et al.  AUTOMATIC DETECTION OF LEARNER'S AFFECT FROM GROSS BODY LANGUAGE , 2009, Appl. Artif. Intell..

[44]  J. Richman,et al.  Physiological time-series analysis using approximate entropy and sample entropy. , 2000, American journal of physiology. Heart and circulatory physiology.

[45]  Yiwen Zhao,et al.  Physiological Signals Based Quantitative Evaluation Method of the Pain , 2014 .

[46]  Erik Scheme,et al.  Navigating features: a topologically informed chart of electromyographic features space , 2017, Journal of The Royal Society Interface.

[47]  Mingzhe Jiang,et al.  Ultra-short-term analysis of heart rate variability for real-time acute pain monitoring with wearable electronics , 2017, 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[48]  D. Mobbs,et al.  Affective Neuroscience: Past, Present, and Future , 2009 .

[49]  Angkoon Phinyomark,et al.  Feature extraction of the first difference of EMG time series for EMG pattern recognition , 2014, Comput. Methods Programs Biomed..

[50]  W. James II.—WHAT IS AN EMOTION ? , 1884 .

[51]  Olaf Sporns,et al.  Towards a new approach to reveal dynamical organization of the brain using topological data analysis , 2018, Nature Communications.

[52]  Zheru Chi,et al.  Facial Expression Recognition in Video with Multiple Feature Fusion , 2018, IEEE Transactions on Affective Computing.

[53]  Julien Penders,et al.  Trapezius muscle EMG as predictor of mental stress , 2013, TECS.

[54]  K. Scherer,et al.  Vocal expression of affect , 2005 .

[55]  Srinivasan Murali,et al.  A wearable device for physical and emotional health monitoring , 2015, 2015 Computing in Cardiology Conference (CinC).

[56]  Angkoon Phinyomark,et al.  An Investigation of Temporally Inspired Time Domain Features for Electromyographic Pattern Recognition , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[57]  Ayoub Al-Hamadi,et al.  The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system , 2013, 2013 IEEE International Conference on Cybernetics (CYBCO).

[58]  L. Becerra,et al.  The Pain Imaging Revolution: Advancing Pain Into the 21st Century , 2010, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[59]  Ligang Zhang,et al.  Synchronous prediction of arousal and valence using LSTM network for affective video content analysis , 2017, 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[60]  Tarmo Lipping,et al.  Comparison of entropy and complexity measures for the assessment of depth of sedation , 2006, IEEE Transactions on Biomedical Engineering.

[61]  Jianda Han,et al.  Physiological Signal-Based Method for Measurement of Pain Intensity , 2017, Front. Neurosci..

[62]  Ahmad R. Sharafat,et al.  Reliable emotion recognition system based on dynamic adaptive fusion of forehead biopotentials and physiological signals , 2015, Comput. Methods Programs Biomed..

[63]  Richard J Bodnar,et al.  Opioid supraspinal analgesic synergy between the amygdala and periaqueductal gray in rats , 1998, Brain Research.

[64]  Richard J Davidson,et al.  Test-retest reliability of voluntary emotion regulation. , 2009, Psychophysiology.

[65]  Pramod K. Varshney,et al.  Performance of mutual information similarity measure for registration of multitemporal remote sensing images , 2003, IEEE Trans. Geosci. Remote. Sens..

[66]  P Leconte,et al.  Comparative study of methods of measuring acute pain intensity in an ED. , 1998, The American journal of emergency medicine.

[67]  Pau-Choo Chung,et al.  Representative Segment-Based Emotion Analysis and Classification with Automatic Respiration Signal Segmentation , 2012, IEEE Transactions on Affective Computing.

[68]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Markus Kächele,et al.  Bio-Visual Fusion for Person-Independent Recognition of Pain Intensity , 2015, MCS.

[70]  J. Russell A circumplex model of affect. , 1980 .

[71]  Rafael A. Calvo,et al.  Detecting Naturalistic Expressions of Nonbasic Affect Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[72]  Fakhri Karray,et al.  Fuzzy entropy: a brief survey , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).

[73]  Andrea Pinna,et al.  Physiological-Based Emotion Detection and Recognition in a Video Game Context , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[74]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[75]  P. Ekman,et al.  Unmasking the face : a guide to recognizing emotions from facial clues , 1975 .

[76]  Saeid Sanei,et al.  A time-frequency approach for EEG signal segmentation , 2014 .

[77]  Juan J. Flores,et al.  Induction of Emotional States in Educational Video Games Through a Fuzzy Control System , 2018, IEEE Transactions on Affective Computing.

[78]  Mohamed Chetouani,et al.  Hilbert-Huang transform based physiological signals analysis for emotion recognition , 2009, 2009 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT).

[79]  Maja Pantic,et al.  Fully Automatic Recognition of the Temporal Phases of Facial Actions , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[80]  Semyon Slobounov,et al.  Application of a novel measure of EEG non-stationarity as ‘Shannon- entropy of the peak frequency shifting’ for detecting residual abnormalities in concussed individuals , 2011, Clinical Neurophysiology.

[81]  M. Berger,et al.  Pain in elderly people with severe dementia: A systematic review of behavioural pain assessment tools , 2006, BMC geriatrics.

[82]  G. Carlsson,et al.  Topology based data analysis identifies a subgroup of breast cancers with a unique mutational profile and excellent survival , 2011, Proceedings of the National Academy of Sciences.