Analysis of multimodal physiological signals within and between individuals to predict psychological challenge vs. threat

Abstract Challenge and threat characterize distinct patterns of physiological response to a motivated performance task where the response patterns vary as a function of an individual's evaluation of task demands relative to his/her available resources to cope with the demands. Challenge and threat responses during motivated performance have been used to understand psychological, behavioral, and biological phenomena across many motivated performance domains. In this study, we aimed to investigate individual and group-level variations in physiological responding across a series of motivated performance tasks that vary in difficulty. The proposed approach is motivated by documented individual differences in physiological responses observed in motivated performance tasks, such that we first focus on individual differences in physiological responses rather than group-level comparisons. Then, through our analysis of individuals we identify sub-groups (i.e., clusters) of individuals that share common physiological patterns across tasks of varying difficulty and we perform across-subject analysis within each cluster. This is distinct from existing studies which typically do not examine individual vs. subgroup-specific patterns of physiological activity. Such an approach enables us to identify patterns in physiological responses that can be used to predict self-reported judgments of challenge vs. threat with higher accuracy in each subgroup compared to an analysis that includes the entire sample population as a single group. Specifically, three hypotheses were tested: (H1) individuals will have different sets of physiological patterns (features) across tasks of varying difficulty; (H2) there will be subgroups of individuals who share common salient physiological features across the subgroup clusters that differentiate their physiological responding across tasks of varying difficulty; and (H3) the accuracy of predicting self-reported judgments of challenge vs. threat across individuals will be higher within each subgroup with shared salient physiological features than across all subgroups or the entire sample with all computed features. To test these hypotheses, we developed an integrated analytic framework for multimodal physiological data analysis. We employed data from an existing experiment in which participants completed three mental arithmetic tasks of increasing difficulty during which different modalities of physiological data were collected. Analyses revealed three subgroups of participants who shared common features that best differentiated their within-individual physiological response patterns across tasks. Support vector machine (SVM) classifiers were trained using both shared features within each group and all computed features to predict challenge vs. threat states. Results showed that, the within-group classification model using group common features achieved higher self-report prediction accuracy compared to an alternative model trained on data from all participants without feature selection.

[1]  Hlaing Minn,et al.  Real-time sleep quality assessment using single-lead ECG and multi-stage SVM classifier , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[2]  Matthew K Nock,et al.  Mind over matter: reappraising arousal improves cardiovascular and cognitive responses to stress. , 2012, Journal of experimental psychology. General.

[3]  Richard L. Hazlett,et al.  Incorporating Facial EMG Emotion Measures as Feedback in the Software Design Process , 2005 .

[4]  Jeremy P Jamieson,et al.  Reappraising Stress Arousal Improves Performance and Reduces Evaluation Anxiety in Classroom Exam Situations , 2016 .

[5]  Jennifer G. Dy,et al.  Emotion Fingerprints or Emotion Populations? A Meta-Analytic Investigation of Autonomic Features of Emotion Categories , 2018, Psychological bulletin.

[6]  Jim Blascovich,et al.  Subjective, physiological, and behavioral effects of threat and challenge appraisal. , 1993 .

[7]  R. Kelsey,et al.  Cardiovascular reactivity and adaptation to recurrent psychological stress: replication and extension. , 2004, Psychophysiology.

[8]  Tsutomu Miyasato,et al.  Emotion recognition from audiovisual information , 1998, 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175).

[9]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[10]  W. J. Tompkins,et al.  Motion Artifact from Spot and Band Electrodes During Impedance Cardiography , 1986, IEEE Transactions on Biomedical Engineering.

[11]  Yue Wang,et al.  A three-stage decision framework for multi-subject emotion recognition using physiological signals , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[12]  Ryan O. Murphy,et al.  Using Skin Conductance in Judgment and Decision Making Research , 2011 .

[13]  J Blascovich,et al.  Cardiovascular reactivity and adaptation to recurrent psychological stress: effects of prior task exposure. , 1999, Psychophysiology.

[14]  B. Alpert,et al.  Reliability of cardiovascular reactivity to stress: internal consistency. , 2007, Psychophysiology.

[15]  P. Ekman Facial expression and emotion. , 1993, The American psychologist.

[16]  K. H. Kim,et al.  Emotion recognition system using short-term monitoring of physiological signals , 2004, Medical and Biological Engineering and Computing.

[17]  Maciej Behnke,et al.  Successful performance and cardiovascular markers of challenge and threat: A meta-analysis. , 2018, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[18]  J Blascovich,et al.  Cognitive and physiological antecedents of threat and challenge appraisal. , 1997, Journal of personality and social psychology.

[19]  Jim Blascovich,et al.  Challenge and Threat During Social Interactions With White and Black Men , 2002 .

[20]  J. Blascovich,et al.  How attributional ambiguity shapes physiological and emotional responses to social rejection and acceptance. , 2008, Journal of Personality and Social Psychology.

[21]  Conor Heneghan,et al.  Automated processing of the single-lead electrocardiogram for the detection of obstructive sleep apnoea , 2003, IEEE Transactions on Biomedical Engineering.

[22]  J Blascovich,et al.  Cardiovascular reactivity and adaptation to recurrent psychological stress: the moderating effects of evaluative observation. , 2000, Psychophysiology.

[23]  A. W. Goertz Left Ventricular Ejection Time and End-Systolic Pressure Revisited , 1995 .

[24]  Jim Blascovich,et al.  Threatened by the unexpected: physiological responses during social interactions with expectancy-violating partners. , 2007, Journal of personality and social psychology.

[25]  J Blascovich,et al.  Perceiver threat in social interactions with stigmatized others. , 2001, Journal of personality and social psychology.

[26]  Simon S. Young,et al.  Computerized Data Acquisition and Analysis for the Life Sciences: A Hands-on Guide , 2001 .

[27]  Mark D. Seery The Biopsychosocial Model of Challenge and Threat: Using the Heart to Measure the Mind , 2013 .

[28]  Lisa Feldman Barrett,et al.  Cardiovascular patterns associated with threat and challenge appraisals: a within-subjects analysis. , 2002, Psychophysiology.

[29]  P. Jaccard THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.1 , 1912 .

[30]  R. Kelsey,et al.  An evaluation of the ensemble averaged impedance cardiogram. , 1990, Psychophysiology.

[31]  Jim Blascovich,et al.  The Biopsychosocial Model of Arousal Regulation , 1996 .

[32]  Jim Blascovich,et al.  Challenge and threat responses during downward and upward social comparisons , 2001 .

[33]  J. Fahrenberg,et al.  Methodological guidelines for impedance cardiography. , 1990, Psychophysiology.

[34]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[35]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Dongrui Wu,et al.  Inductive Transfer Learning for Handling Individual Differences in Affective Computing , 2011, ACII.

[37]  D. Pennell,et al.  Normalized left ventricular systolic and diastolic function by steady state free precession cardiovascular magnetic resonance. , 2006, Journal of cardiovascular magnetic resonance : official journal of the Society for Cardiovascular Magnetic Resonance.

[38]  Toni Schmader,et al.  Turning the knots in your stomach into bows: Reappraising arousal improves performance on the GRE. , 2010, Journal of experimental social psychology.

[39]  Mahadev D. Uplane,et al.  Digital elliptic filter application for noise reduction in ECG signal , 2005 .

[40]  Dennis C. Tkach,et al.  Study of stability of time-domain features for electromyographic pattern recognition , 2010, Journal of NeuroEngineering and Rehabilitation.

[41]  H. Gray,et al.  Why Egalitarianism Might Be Good for Your Health , 2007, Psychological science.

[42]  R. Dienstbier,et al.  Arousal and physiological toughness: implications for mental and physical health. , 1989, Psychological review.

[43]  Mikhail Kuznetsov,et al.  Filtering the surface EMG signal: Movement artifact and baseline noise contamination. , 2010, Journal of biomechanics.

[44]  P. Welch The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[45]  Isabel M. Santos,et al.  EEG-based Subject Independent Affective Computing Models , 2015, INNS Conference on Big Data.