Real time emotion aware applications: A case study employing emotion evocative pictures and neuro-physiological sensing enhanced by Graphic Processor Units

In this paper the feasibility of adopting Graphic Processor Units towards real-time emotion aware computing is investigated for boosting the time consuming computations employed in such applications. The proposed methodology was employed in analysis of encephalographic and electrodermal data gathered when participants passively viewed emotional evocative stimuli. The GPU effectiveness when processing electroencephalographic and electrodermal recordings is demonstrated by comparing the execution time of chaos/complexity analysis through nonlinear dynamics (multi-channel correlation dimension/D2) and signal processing algorithms (computation of skin conductance level/SCL) into various popular programming environments. Apart from the beneficial role of parallel programming, the adoption of special design techniques regarding memory management may further enhance the time minimization which approximates a factor of 30 in comparison with ANSI C language (single-core sequential execution). Therefore, the use of GPU parallel capabilities offers a reliable and robust solution for real-time sensing the user's affective state.

[1]  Kostas Karpouzis,et al.  3rd international workshop on affective interaction in natural environments (AFFINE) , 2010, ACM Multimedia.

[2]  E. Hudlicka.  AFFECTIVE COMPUTING FOR GAME DESIGN , 2008 .

[3]  Sravanti L. Sanivarapu.  Emotion , 2020, Indian journal of psychiatry.

[4]  Manuel Ujaldon,et al.  Parallel 3D fast wavelet transform on manycore GPUs and multicore CPUs , 2010, ICCS.

[5]  A. Damasio,et al.  Emotion, decision making and the orbitofrontal cortex. , 2000, Cerebral cortex.

[6]  M. Bradley,et al.  Emotion and motivation I: defensive and appetitive reactions in picture processing. , 2001, Emotion.

[7]  Thomas Wehrle,et al.  Emotion and Facial Expression , 1999, IWAI.

[8]  Burkhard Zink,et al.  A general relativistic evolution code on CUDA architectures , 2008 .

[9]  Christine L. Lisetti,et al.  Emotion recognition from physiological signals using wireless sensors for presence technologies , 2004, Cognition, Technology & Work.

[10]  Panagiotis D. Bamidis,et al.  Towards emotion aware computing: A study of arousal modulation with multichannel event-related potentials, delta oscillatory activity and skin conductivity responses , 2008, 2008 8th IEEE International Conference on BioInformatics and BioEngineering.

[11]  Loïc Kessous,et al.  Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis , 2010, Journal on Multimodal User Interfaces.

[12]  Konstantina S. Nikita,et al.  Comparison of fractal dimension estimation algorithms for epileptic seizure onset detection , 2008, BIBE.

[13]  Charalampos Bratsas,et al.  On the Classification of Emotional Biosignals Evoked While Viewing Affective Pictures: An Integrated Data-Mining-Based Approach for Healthcare Applications , 2010, IEEE Transactions on Information Technology in Biomedicine.

[14]  P. Lang.  International Affective Picture System (IAPS) : Technical Manual and Affective Ratings , 1995 .

[15]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  L. Rothkrantz,et al.  Toward an affect-sensitive multimodal human-computer interaction , 2003, Proc. IEEE.

[17]  P. McOwan,et al.  Affect Recognition for Interactive Companions , 2008 .

[18]  P. Ktonas,et al.  Comparison of fractal dimension estimation algorithms for epileptic seizure onset detection , 2008, 2008 8th IEEE International Conference on BioInformatics and BioEngineering.

[19]  Paul Pauli,et al.  Modulation of event-related brain potentials during affective picture processing: a complement to startle reflex and skin conductance response? , 2004, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[20]  Panagiotis D. Bamidis,et al.  Affective computing in the era of contemporary neurophysiology and health informatics , 2004, Interact. Comput..

[21]  K. Berridge,et al.  Affective neuroscience of pleasure: reward in humans and animals , 2008, Psychopharmacology.

[22]  Niilo Saranummi,et al.  Information technology in biomedicine , 2002, IEEE Trans. Biomed. Eng..

[23]  Ehud Sharlin,et al.  Robot expressionism through cartooning , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[24]  Verdi March,et al.  Survey on Parallel Programming Model , 2008, NPC.

[25]  Karine Sergerie,et al.  The role of the amygdala in emotional processing: A quantitative meta-analysis of functional neuroimaging studies , 2008, Neuroscience & Biobehavioral Reviews.

[26]  Rob Reilly,et al.  An Affective Module for an Intelligent Tutoring System , 2002, Intelligent Tutoring Systems.

[27]  Christophe Jaillet,et al.  MultiGPU computing using MPI or OpenMP , 2010, Proceedings of the 2010 IEEE 6th International Conference on Intelligent Computer Communication and Processing.

[28]  Kipton Barros,et al.  Blasting through lattice calculations using CUDA , 2008, 0810.5365.

[29]  Kenneth A. Hawick,et al.  Auto-generation of Parallel Finite-Differencing Code for MPI, TBB and CUDA , 2011, 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum.

[30]  Charalampos Bratsas,et al.  An Integrated Approach to Emotion Recognition for Advanced Emotional Intelligence , 2009, HCI.

[31]  Christine L. Lisetti,et al.  Emotion Recognition from Physiological Signals for User Modeling of Affect , 2003 .

[32]  Chrysa D. Lithari,et al.  Are Females More Responsive to Emotional Stimuli? A Neurophysiological Study Across Arousal and Valence Dimensions , 2009, Brain Topography.

[33]  M. Bradley,et al.  Motivated attention: Affect, activation, and action. , 1997 .

[34]  Panagiotis D. Bamidis,et al.  Description and future trends of ICT solutions offered towards independent living: the case of LLM project , 2009, PETRA '09.

[35]  M. Bradley,et al.  Emotion, Motivation, and Anxiety: Brain Mechanisms and Psychophysiology the Motivational Organization of Emotion Patterns of Human Emotion Emotion and Perception the Psychophysiology of Picture Processing Neural Imaging: Motivation in the Visual Cortex Motivational Circuits in the Brain , 2022 .

[36]  Charalampos Bratsas,et al.  Toward Emotion Aware Computing: An Integrated Approach Using Multichannel Neurophysiological Recordings and Affective Visual Stimuli , 2010, IEEE Transactions on Information Technology in Biomedicine.

[37]  Rosalind W. Picard.  Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..

[38]  R. Dolan,et al.  Distinct spatial frequency sensitivities for processing faces and emotional expressions , 2003, Nature Neuroscience.

[39]  Jonathan Klein,et al.  This computer responds to user frustration: Theory, design, and results , 2002, Interact. Comput..

[40]  M. Junghöfer,et al.  Attention and emotion: an ERP analysis of facilitated emotional stimulus processing , 2003, Neuroreport.

[41]  Panagiotis D. Bamidis,et al.  Accelerating biomedical signal processing algorithms with parallel programming on graphic processor units , 2009, 2009 9th International Conference on Information Technology and Applications in Biomedicine.

[42]  Eva Hudlicka,et al.  To feel or not to feel: The role of affect in human-computer interaction , 2003, Int. J. Hum. Comput. Stud..

[43]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[44]  M. Bradley,et al.  Brain potentials in affective picture processing: covariation with autonomic arousal and affective report , 2000, Biological Psychology.

[45]  鈴木 聡.  Media Equation 研究の背景と動向 , 2011 .

[46]  Peter J. Lang,et al.  Attention and Orienting : Sensory and Motivational Processes , 1997 .