Quantifying engagement: Measuring player involvement in human-avatar interactions

This research investigated the merits of using an established system for rating behavioral cues of involvement in human dyadic interactions (i.e., face-to-face conversation) to measure involvement in human-avatar interactions. Gameplay audio-video and self-report data from a Feasibility Trial and Free Choice study of an effective peer resistance skill building simulation game (DRAMA-RAMA™) were used to evaluate reliability and validity of the rating system when applied to human-avatar interactions. The Free Choice study used a revised game prototype that was altered to be more engaging. Both studies involved girls enrolled in a public middle school in Central Florida that served a predominately Hispanic (greater than 80%), low-income student population. Audio-video data were coded by two raters, trained in the rating system. Self-report data were generated using measures of perceived realism, predictability and flow administered immediately after game play. Hypotheses for reliability and validity were supported: Reliability values mirrored those found in the human dyadic interaction literature. Validity was supported by factor analysis, significantly higher levels of involvement in Free Choice as compared to Feasibility Trial players, and correlations between involvement dimension sub scores and self-report measures. Results have implications for the science of both skill-training intervention research and game design.

[1]  Christian Kay,et al.  The Oxford English Dictionary Online , 2004, Lit. Linguistic Comput..

[2]  V. Manusov.  The sourcebook of nonverbal measures : going beyond words , 2004 .

[3]  Po-Yao Chao,et al.  An Empathic Avatar in a Computer-Aided Learning Program to Encourage and Persuade Learners , 2012, J. Educ. Technol. Soc..

[4]  Rafael Bidarra,et al.  In Press: Ieee Transactions on Computational Intelligence and Ai in Games Adaptivity Challenges in Games and Simulations: a Survey , 2022 .

[5]  James E. Driskell,et al.  Games, Motivation, and Learning: A Research and Practice Model , 2002 .

[6]  M. Csíkszentmihályi.  Flow: The Psychology of Optimal Experience , 1990 .

[7]  W. Nehring,et al.  Nursing Simulation: A Review of the Past 40 Years , 2009 .

[8]  Timothy Charoenying,et al.  Accountable Game Design: Structuring the Dynamics of Student Learning Interactions , 2010 .

[9]  B. Ootim,et al.  Social Support , 1986, Ageing and Society.

[10]  Katie Salen,et al.  Rules of play: game design fundamentals , 2003 .

[11]  Jeanne H. Brockmyer,et al.  The Development of the Game Engagement Questionnaire: A Measure of Engagement in Video Game Playing: Response to Reviews , 2009, Interacting with computers.

[12]  James Paul Gee,et al.  Learning by Design: Good Video Games as Learning Machines , 2005 .

[13]  William V. Wright,et al.  A Theory of Fun for Game Design , 2004 .

[14]  J. Bailenson,et al.  Virtual Humans and Persuasion: The Effects of Agency and Behavioral Realism , 2007 .

[15]  Clint A. Bowers,et al.  An Examination of Flow and Immersion in Games , 2011 .

[16]  Sheng-Chin Yu,et al.  journal homepage: www.elsevier.com/locate/compedu , 2022 .

[17]  Laura K. Guerrero,et al.  Attachment‐style differences in intimacy and involvement: A test of the four‐category model , 1996 .

[18]  Jim Blascovich,et al.  Social evaluations of embodied agents and avatars , 2011, Comput. Hum. Behav..

[19]  Richard R. Johnson.  Confounding influences on police detection of suspiciousness , 2006 .

[20]  M. Hecht,et al.  Narrative Means to Preventative Ends: A Narrative Engagement Framework for Designing Prevention Interventions , 2013, Health communication.

[21]  Newton Lee,et al.  Anna Marie Piersimoni , 2005, CIE.

[22]  Byron Reeves,et al.  Computer agents versus avatars: Responses to interactive game characters controlled by a computer or other player , 2010, Int. J. Hum. Comput. Stud..

[23]  Roger Smith,et al.  The Long History of Gaming in Military Training , 2010 .

[24]  Laura Knarr. Guerrero.  An application of attachment theory to relational messages and nonverbal involvement behaviors in romantic relationships. , 1994 .

[25]  M. Csíkszentmihályi.  Flow. The Psychology of Optimal Experience. New York (HarperPerennial) 1990. , 1990 .

[26]  Judee K. Burgoon,et al.  The Nature of Conversational Involvement and Nonverbal Encoding Patterns , 1987 .

[27]  A. McMahan.  Immersion, Engagement, and Presence: A Method for Analyzing 3-D Video Games , 2013 .

[28]  Nicole C. Krämer,et al.  "It doesn't matter what you are!" Explaining social effects of agents and avatars , 2010, Comput. Hum. Behav..

[29]  Michael J. Beatty,et al.  A Test of the Cognitive Load Hypothesis: Investigating the Impact of Number of Nonverbal Cues Coded and Length of Coding Session on Observer Accuracy , 2007 .

[30]  Marcelle Vernazza,et al.  Involvement , 1974, JAMA.

[31]  Daniel K. Mayes,et al.  Measuring Engagement in Video Games: A Questionnaire , 2001 .

[32]  Jeong Kyu Lee,et al.  Evaluating Mediated Perception of Narrative Health Messages: The Perception of Narrative Performance Scale , 2011, Communication methods and measures.

[33]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[34]  Peta Wyeth,et al.  GameFlow: a model for evaluating player enjoyment in games , 2005, CIE.

[35]  Gayle L. Nelson,et al.  Hispanic/Anglo-American differences in attributions to paralinguistic behavior☆ , 1993 .

[36]  Ivar Oswalt.  Current Applications, Trends, and Organizations in U.S. Military Simulation and Gaming , 1993 .

[37]  David J. Shernoff,et al.  Student engagement in high school classrooms from the perspective of flow theory. , 2003 .

[38]  D. Nickerson,et al.  Randomized Trial of a Peer Resistance Skill-Building Game for Hispanic Early Adolescent Girls , 2013, Nursing research.

[39]  M. Csíkszentmihályi,et al.  The effect of perceived challenges and skills on the quality of subjective experience. , 1996, Journal of personality.

[40]  Michele D. Dickey.  Engaging by design: How engagement strategies in popular computer and video games can inform instructional design , 2005 .

[41]  Michelle F. Wright,et al.  Computer Gaming and Interactive Simulations for Learning: A Meta-Analysis , 2006 .