From Automatic Sound Analysis of Gameplay Footage [Echos] to the Understanding of Player Experience [Ethos]: an Interdisciplinary Approach to Feedback- Based Gameplay Metrics

In line with the ICMC|SMC|2014 conference theme "from digital echos to virtual ethos", and the conference interdisciplinary main objective; the present paper is seeking to demonstrate that the sound feedback stream produced by videogames when activated by players (echos) can be automatically analyzed in order to study how sound can, not only, describe a gameplay perfor-mance, but also help to understand player experience and emotions (ethos). To do so, the present paper illustrates how sound processing algorithms can be applied in the game studies discipline in order to assess and understand better how players engage with videogames. The present paper proposes to adapt the Feedback-based Gameplay Metrics method, successfully applied to the analysis of gameplay footage video stream [17], to the sound stream, via the automatic detection of musical sequences and speech segment.

[1]  Gaël Richard,et al.  Combined Supervised and Unsupervised Approaches for Automatic Segmentation of Radiophonic Audio Streams , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[2]  Georg Lauteren,et al.  The Pleasure of the Playable Text: Towards an Aesthetic Theory of Computer Games , 2002, CGDC Conf..

[3]  Anders Drachen,et al.  Methods for Evaluating Gameplay Experience in a Serious Gaming Context , 2010, Int. J. Comput. Sci. Sport.

[4]  Gareth Schott,et al.  Situating Gaming as a Sonic Experience: The acoustic ecology of First-Person Shooters , 2007, DiGRA Conference.

[5]  J. Newman.  The myth of the ergodic videogame , 2002 .

[6]  Guillaume Gravier,et al.  Corpus description of the ESTER Evaluation Campaign for the Rich Transcription of French Broadcast News , 2004, LREC.

[7]  Avery Wang,et al.  An Industrial Strength Audio Search Algorithm , 2003, ISMIR.

[8]  Mia Consalvo.  Hot Dates and Fairy-Tale Romances: Studying Sexuality in Video Games , 2013 .

[9]  Pedro Cano,et al.  A review of algorithms for audio fingerprinting , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[10]  Gareth Schott,et al.  Feedback-based gameplay metrics: measuring player experience via automatic visual analysis , 2012, IE '12.

[11]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[12]  Alessandro Canossa.  Play-Persona: Modeling Player Behaviour in Computer Games , 2009 .

[13]  James Newman,et al.  The Myth of the Ergodic Videogame - Some thoughts on player-character relationships in videogames , 2002, Game Stud..

[14]  Alessandro Canossa,et al.  Towards gameplay analysis via gameplay metrics , 2009, MindTrek '09.

[15]  Ana M. Barbancho,et al.  Humming Method for Content-Based Music Information Retrieval , 2011, ISMIR.

[16]  Kristine Jørgensen,et al.  On the Functional Aspects of Computer Game Audio , 2006 .

[17]  Xavier Serra,et al.  Chroma Binary Similarity and Local Alignment Applied to Cover Song Identification , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[18]  Karolien Poels,et al.  Playability and Player Experience Research [Panel Abstracts] , 2009, DiGRA Conference.

[19]  Grant Tavinor.  The Art of Videogames , 2009 .

[20]  Alessandro Canossa,et al.  Analyzing spatial user behavior in computer games using geographic information systems , 2009, MindTrek '09.

[21]  Bruce Phillips,et al.  Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems , 2008, CHI.

[22]  M S Waterman,et al.  Identification of common molecular subsequences. , 1981, Journal of molecular biology.

[23]  Christoph Klimmt,et al.  Explaining the enjoyment of playing video games: the role of competition , 2003, ICEC.

[24]  Guillaume Gravier,et al.  The ESTER phase II evaluation campaign for the rich transcription of French broadcast news , 2005, INTERSPEECH.

[25]  Nicola Orio,et al.  MUSIC RETRIEVAL (Foundations and Trends(R) in Information Retrieval) , 2006 .

[26]  Pejman Mirza-Babaei,et al.  Biometric Storyboards: visualising meaningful gameplay events , 2011 .

[27]  Lennart E. Nacke,et al.  Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment , 2009 .

[28]  Julien Allali,et al.  Detection of Near-Duplicate Musical Documents from a Multi-Level Comparison of Tonal Information , 2011 .

[29]  Alessandro Canossa,et al.  Arrrgghh!!!: blending quantitative and qualitative methods to detect player frustration , 2011, FDG.

[30]  J. Matias Kivikangas,et al.  Psychophysiology of flow experience : An explorative study , 2006 .

[31]  Lennart E. Nacke,et al.  Flow and immersion in first-person shooters: measuring the player's gameplay experience , 2008, Future Play.

[32]  Chia-Hung Yeh,et al.  Movie story intensity representation through audiovisual tempo analysis , 2009, Multimedia Tools and Applications.