Towards a new ITU-T recommendation for subjective methods evaluating gaming QoE

This paper reports on activities in Study Group 12 of the International Telecommunication Union (ITU-T SG12) to define a new Recommendation on subjective evaluation methods for gaming Quality of Experience (QoE). It first resumes the structure and content of the current draft which has been proposed to ITU-T SG12 in September 2014 and then critically discusses potential gaming content and evaluation methods for inclusion into the upcoming Recommendation. The aim is to start a discussion amongst experts on potential evaluation methods and their limitations, before finalizing a Recommendation. Such a recommendation might in the end be applied by non-expert users, hence wrong decisions in the evaluation design could negatively affect gaming QoE throughout the evaluation.

[1]  R. Ryan,et al.  Perceived locus of causality and internalization: examining reasons for acting in two domains. , 1989, Journal of personality and social psychology.

[2]  P. Costa,et al.  Revised NEO Personality Inventory (NEO-PI-R) and NEO-Five-Factor Inventory (NEO-FFI) , 1992 .

[3]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[4]  Cathleen A. Towey.  Flow , 2000, My Wilderness.

[5]  Nick Yee,et al.  Motivations for Play in Online Games , 2006, Cyberpsychology Behav. Soc. Netw..

[6]  Karolien Poels,et al.  Characterising and measuring user experiences in digital games , 2007 .

[7]  Methodology for the subjective assessment of video quality in multimedia applications ( Question , 2007 .

[8]  D. Kort,et al.  D3.3 : Game Experience Questionnaire:development of a self-report measure to assess the psychological impact of digital games , 2007 .

[9]  S. Engeser,et al.  Flow, performance and moderators of challenge-skill balance , 2008 .

[10]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[11]  Chin-Laung Lei,et al.  A crowdsourceable QoE evaluation framework for multimedia content , 2009, ACM Multimedia.

[12]  Cédric Courtois,et al.  Towards a Serious Game Experience Model: Validation, Extension and Adaptation of the GEQ for Use in an Educational Context , 2010 .

[13]  P. Costa,et al.  NEO inventories for the NEO Personality Inventory-3 (NEO-PI-3), NEO Five-Factor Inventory-3 (NEO-FFI-3), NEO Personality Inventory-Revised (NEO PI-R) : professional manual , 2010 .

[14]  Sebastian Möller,et al.  Too tired for calling? A physiological measure of fatigue caused by bandwidth limitations , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[15]  Robert Schleicher,et al.  The effects of text-to-speech system quality on emotional states and frontal alpha band power , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[16]  Peter Schelkens,et al.  Qualinet White Paper on Definitions of Quality of Experience , 2013 .

[17]  Sebastian Möller,et al.  Gaming taxonomy: An overview of concepts and evaluation methods for computer gaming QoE , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[18]  Lea Skorin-Kapov,et al.  The impact of user, system, and context factors on gaming QoE: A case study involving MMORPGs , 2013, 2013 12th Annual Workshop on Network and Systems Support for Games (NetGames).

[19]  Jan Van Looy,et al.  Challenging the Other: Exploring the Role of Opponent Gender in Digital Game Competition for Female Players , 2014, Cyberpsychology Behav. Soc. Netw..

[20]  Sebastian Möller,et al.  Towards understanding the relationship between game type and network traffic for cloud gaming , 2014, 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[21]  Sebastian Möller,et al.  Assessing the impact of display size, game type, and usage context on mobile gaming QOE , 2014, 2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX).

[22]  Richard Huskey,et al.  Attentional Capacity and Flow Experiences: Examining the Attentional Component of Synchronization Theory , 2015 .

[23]  Nicole Fruehauf.  Flow The Psychology Of Optimal Experience , 2016 .

[24]  S. Möller,et al.  Factors Influencing Gaming QoE: Lessons Learned from the Evaluation of Cloud Gaming Services , 2017 .