Feasible Affect Recognition in Advertising Based on Physiological Responses from Wearable Sensors

This is an extension from a selected paper from JSAI2019. Recent studies in affective computing have facilitated and stimulated the development of systems and sensors that can recognize and interpret human affects. Affective computing has been applied in various domains, and one of the applied domains is in the marketing area to increase the consumers’ appeal and attraction. In particular, advertisements (ads) can convey amounts of information in a short time. Therefore, using physiological responses can help to acquire a user’s feedback and obtain an advantage. This study proposes non-invasive affect recognition in each scene of an advertising video using electroencephalogram (EEG), electrocardiogram (ECG) and eye-tracking. The preliminary analysis of EEG shows the relationship between scene feeling score and emotional affects regarding physiological responses. Hence, we also trained two types of recognition models: window recognition and sequence learning. The models learned from the physiological responses and questionnaires on a user’s preference in each ad scene.

[1]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[2]  Nikos E. Mastorakis,et al.  Multilayer perceptron and neural networks , 2009 .

[3]  Nico Pallamin,et al.  Emotion Recognition Using Physiological Signals: Laboratory vs. Wearable Sensors , 2017, AHFE.

[4]  L. Trainor,et al.  Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions , 2001 .

[5]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[6]  L. Trainor,et al.  Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions , 2001 .

[7]  Laura Chamberlain,et al.  What is "neuromarketing"? A discussion and agenda for future research. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[8]  Zhenqi Li,et al.  A Review of Emotion Recognition Using Physiological Signals , 2018, Sensors.

[9]  Xiaohui Yuan,et al.  A Survey on Eye-Gaze Tracking Techniques , 2013, ArXiv.

[10]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[11]  Areej Al-Wabil,et al.  Review and Classification of Emotion Recognition Based on EEG Brain-Computer Interface System Research: A Systematic Review , 2017 .

[12]  Doug White SASci.  Queuing Theory , 1997 .

[13]  M. Arthanari,et al.  ECG Feature Extraction Techniques - A Survey Approach , 2010, ArXiv.

[14]  Marti A. Hearst.  Intelligent Connections: Battling with GA-Joe. , 1998 .

[15]  Yue Wang,et al.  Subject-independent emotion recognition based on physiological signals: a three-stage decision method , 2017, BMC Medical Informatics and Decision Making.

[16]  W. Cannon.  The James-Lange theory of emotions: a critical examination and an alternative theory. By Walter B. Cannon, 1927. , 1927, The American journal of psychology.

[17]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[18]  Gora Chand Nandi,et al.  Feature Extraction and Selection for Emotion Recognition from Electrodermal Activity , 2019, IEEE Transactions on Affective Computing.

[19]  Miyoung Kim,et al.  A Review on the Computational Methods for Emotional State Estimation from the Human EEG , 2013, Comput. Math. Methods Medicine.

[20]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.