Automatic Recognition of Boredom in Video Games Using Novel Biosignal Moment-Based Features

This paper presents work conducted toward the biosignals-based automatic recognition of boredom, induced during video-game playing. For this purpose, common biosignal feature extraction methods were exploited and their capability to identify boredom was assessed. Moreover, for the first time, Legendre and Krawtchouk moments, as well as novel moment variations, were extracted as biosignal features and their potential toward automatic affect recognition was examined using the specific application scenario. The present analysis was conducted with ECG and GSR data collected from 19 different subjects, while boredom was naturally induced during the repetitive playing of a 3D video game. Conventional biosignal features as well as moment-based ones were found to be effective for the automatic recognition of boredom by achieving classification accuracies around 85 percent. Then, the joint use of moments and moment variations with conventional features was found to significantly improve classification accuracy by producing a maximum correct classification ratio of 94.17 percent.

[1]  Michael G. Strintzis,et al.  3D Content-Based Search Based on 3D Krawtchouk Moments , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[2]  M. Teague.  Image analysis via the general theory of moments , 1980 .

[3]  Georgios N. Yannakakis,et al.  Real-Time Game Adaptation for Optimizing Player Satisfaction , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[4]  Regan L. Mandryk,et al.  A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies , 2007, Int. J. Hum. Comput. Stud..

[5]  Guillaume Chanel,et al.  Boredom, engagement and anxiety as indicators for adaptation to difficulty in games , 2008, MindTrek '08.

[6]  Shan Li,et al.  Complex Zernike Moments Features for Shape-Based Image Retrieval , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[7]  Jason Williams,et al.  Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System , 2004, ADS.

[8]  Huazhong Shu,et al.  Efficient Legendre moment computation for grey level images , 2014, Pattern Recognit..

[9]  Peta Wyeth,et al.  GameFlow: a model for evaluating player enjoyment in games , 2005, CIE.

[10]  Venkataramana Appala,et al.  Fast Computation of Inverse Krawtchouk Moment Transform using Clenshaw's Recurrence Formula , 2007, 2007 IEEE International Conference on Image Processing.

[11]  K. R. Ramakrishnan,et al.  Fast computation of Legendre and Zernike moments , 1995, Pattern Recognit..

[12]  Dimitrios Tzovaras,et al.  Gait Recognition Using Compact Feature Extraction Transforms and Depth Information , 2007, IEEE Transactions on Information Forensics and Security.

[13]  Raveendran Paramesran,et al.  Image analysis by Krawtchouk moments , 2003, IEEE Trans. Image Process..

[14]  H. Nagaraja,et al.  Heart rate variability: origins, methods, and interpretive caveats. , 1997, Psychophysiology.

[15]  A. Damasio,et al.  Basic emotions are associated with distinct patterns of cardiorespiratory activity. , 2006, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[16]  Randolph R. Cornelius,et al.  The science of emotion: Research and tradition in the psychology of emotion. , 1997 .

[17]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[19]  Cheng Wang,et al.  Image analysis by modified Legendre moments , 2007, Pattern Recognit..

[20]  Mohammad Soleymani,et al.  Short-term emotion assessment in a recall paradigm , 2009, Int. J. Hum. Comput. Stud..

[21]  Christos D. Katsis,et al.  Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[22]  S. Lajevardi,et al.  Zernike moments for facial expression recognition , 2009 .

[23]  Wei Shen,et al.  On Geometric and Orthogonal Moments , 2000, Int. J. Pattern Recognit. Artif. Intell..

[24]  Christine L. Lisetti,et al.  Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals , 2004, EURASIP J. Adv. Signal Process..

[25]  Rene F. Swarttouw,et al.  The Askey-scheme of hypergeometric orthogonal polynomials and its q-analogue Report Fac , 1996, math/9602214.

[26]  Rosalind W. Picard.  Affective Computing , 1997 .

[27]  J. Andreassi.  Psychophysiology: Human Behavior and Physiological Response , 1980 .

[28]  K. H. Kim,et al.  Emotion recognition system using short-term monitoring of physiological signals , 2004, Medical and Biological Engineering and Computing.

[29]  Georgios N. Yannakakis,et al.  Preference Learning for Cognitive Modeling: A Case Study on Entertainment Preferences , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[30]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Changchun Liu,et al.  An empirical study of machine learning techniques for affect recognition in human–robot interaction , 2006, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Roland T. Chin,et al.  On Image Analysis by the Methods of Moments , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  G. Breithardt,et al.  Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. , 1996 .

[34]  A. Malliani,et al.  Heart rate variability. Standards of measurement, physiological interpretation, and clinical use , 1996 .

[35]  Petra Sundström,et al.  Exploring the Affective Loop , 2005 .

[36]  Johannes Wagner,et al.  From Physiological Signals to Emotions: Implementing and Comparing Selected Methods for Feature Extraction and Classification , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[37]  Elisabeth André,et al.  Emotion recognition based on physiological changes in music listening , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Guobao Wang,et al.  Parallel recursive computation of the inverse Legendre moment transforms for signal and image reconstruction , 2004, IEEE Signal Processing Letters.

[39]  K. Scherer.  Voice, Stress, and Emotion , 1986 .

[40]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Guillaume Chanel,et al.  Emotion Assessment: Arousal Evaluation Using EEG's and Peripheral Physiological Signals , 2006, MRCS.