Evaluating the Onboarding Phase of Free-toPlay Mobile Games: A Mixed-Method Approach

The first few minutes of play, commonly referred to as the onboarding phase, of Free-to-Play mobile games typically display a substantial churn rate among new players. It is therefore vital for designers to effectively evaluate this phase to investigate its satisfaction of player expectations. This paper presents a study utilizing a lab-based mixed-methods approach in providing insights for evaluating the user experience of onboarding phases in mobile games. This includes an investigation into the contribution of physiological measures (Heart-Rate Variability and Galvanic Skin Conductance) as well as a range of self-reported proxy measures including: a) stimulated recall, engagement graphs, b) flow state survey and c) post-game experience questionnaire. These techniques were applied across 28 participants using three mobile Free-to-Play titles from different genres. This paper makes two important contributions to the games user research (GUR) domain: 1) evaluates different research techniques (e.g. physiological measures and experience graphs) in the context of mobile games; 2) provides an empirically based recommendation for design elements that result in high arousal.

[1]  Thorsten Quandt,et al.  Multiplayer: The Social Aspects of Digital Gaming , 2013 .

[2]  Lennart E. Nacke,et al.  Flow and immersion in first-person shooters: measuring the player's gameplay experience , 2008, Future Play.

[3]  Kenneth S Bordens,et al.  Research design and methods : a process approach , 2004 .

[4]  Lei Zhou,et al.  Using Physiological Measures to Evaluate User Experience of Mobile Applications , 2014, HCI.

[5]  Magy Seif El-Nasr,et al.  Game Analytics - The Basics , 2013, Game Analytics, Maximizing the Value of Player Data.

[6]  Juho Hamari,et al.  Free-to-Play Games: Professionals’ Perspectives , 2014 .

[7]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[8]  K. Kallinen,et al.  Phasic Emotional Reactions to Video Game Events: A Psychophysiological Investigation , 2006 .

[9]  Lennart E. Nacke,et al.  Games User Research and Physiological Game Evaluation , 2015, Game User Experience Evaluation.

[10]  Lennart E. Nacke,et al.  Contextual influences on mobile player experience - A game user experience model , 2013, Entertain. Comput..

[11]  Graham McAllister,et al.  Video Game Development and User Experience , 2010, Evaluating User Experience in Games.

[12]  Janne Paavilainen,et al.  Critical Acclaim and Commercial Success in Mobile Free-to-Play Games , 2016, DiGRA/FDG.

[13]  G. Lu,et al.  A comparison of photoplethysmography and ECG recording to analyse heart rate variability in healthy subjects , 2009, Journal of medical engineering & technology.

[14]  Christian Bauckhage,et al.  Predicting Purchase Decisions in Mobile Free-to-Play Games , 2015, AIIDE.

[15]  W. Boucsein.  Electrodermal activity, 2nd ed. , 2012 .

[16]  J. Matias,et al.  Review on psychophysiological methods in game research , 2010 .

[17]  Hannu Korhonen,et al.  Comparison of playtesting and expert review methods in mobile game evaluation , 2010, Fun and Games '10.

[18]  Judith Good,et al.  Weak inter-rater reliability in heuristic evaluation of video games , 2011, CHI EA '11.

[19]  Regan L. Mandryk,et al.  A continuous and objective evaluation of emotional experience with interactive play environments , 2006, CHI.

[20]  John Lyle,et al.  Stimulated recall: a report on its use in naturalistic research , 2003 .

[21]  H. Marsh,et al.  Development and Validation of a Scale to Measure Optimal Experience: The Flow State Scale , 1996 .

[22]  Randy J. Pagulayan,et al.  User-centered design in games , 2012 .

[23]  S. Puttonen,et al.  The psychophysiology of James Bond: phasic emotional responses to violent video game events. , 2008, Emotion.

[24]  David G. Novick,et al.  Why don't people read the manual? , 2006, SIGDOC '06.

[25]  Extended Games Played by Managerial Firms with Asymmetric Costs , 2014 .

[26]  Henry Been-Lirn Duh,et al.  Playing different games on different phones: an empirical study on mobile gaming , 2008, Mobile HCI.

[27]  Marilyn Tremaine.  CHI '01 Extended Abstracts on Human Factors in Computing Systems , 2001, CHI Extended Abstracts.

[28]  J. Lagopoulos.  Electrodermal activity , 2007, Acta Neuropsychiatrica.

[29]  Jeanne H. Brockmyer,et al.  The Development of the Game Engagement Questionnaire: A Measure of Engagement in Video Game Playing: Response to Reviews , 2009, Interacting with computers.

[30]  Chui Yin Wong,et al.  Socializing in mobile gaming , 2008, DIMEA.

[31]  Yu-kai Chou,et al.  Actionable Gamification: Beyond Points, Badges, and Leaderboards , 2015 .

[32]  Pejman Mirza-Babaei,et al.  Biometric storyboards : a games user research approach for improving qualitative evaluations of player experience , 2014 .

[33]  Anders Drachen,et al.  Identifying Onboarding Heuristics for Free-to-Play Mobile Games: A Mixed Methods Approach , 2016, ICEC.

[34]  Elina M. I. Koivisto,et al.  Playability heuristics for mobile games , 2006, Mobile HCI.

[35]  Andrew Curry.  Playing to win. , 2014, Diabetes forecast.

[36]  Fernando Silveira,et al.  Predicting audience responses to movie content from electro-dermal activity signals , 2013, UbiComp.

[37]  Fang Chen,et al.  Using Galvanic Skin Response (GSR) to Measure Trust and Cognitive Load in the Text-Chat Environment , 2015, CHI Extended Abstracts.

[38]  Pejman Mirza-Babaei,et al.  Understanding the Contribution of Biometrics to Games User Research , 2011, DiGRA Conference.

[39]  Janne Paavilainen,et al.  Domain-specific playability problems in social network games , 2015, Int. J. Arts Technol..

[40]  Graham McAllister,et al.  Video Game Development and User Experience , 2015, Game User Experience Evaluation.

[41]  Lennart E. Nacke,et al.  Unified visualization of quantitative and qualitative playtesting data , 2014, CHI Extended Abstracts.

[42]  Christian Bauckhage,et al.  Predicting player churn in the wild , 2014, 2014 IEEE Conference on Computational Intelligence and Games.