From Objective to Subjective Difficulty Evaluation in Video Games

This paper describes our research investigating the perception of difficulty in video games, defined as players’ estimation of their chances of failure. We discuss our approach as it relates to psychophysical studies of subjective difficulty and to cognitive psychology research into the overconfidence effect. The starting point for our study was the assumption that the strong motivational pull of video games may lead players to become overconfident, and thereby underestimate their chances of failure. We design and implement a method for an experiment using three games, each representing a different type of difficulty, wherein players bet on their capacity to succeed. Our results confirm the existence of a gap between players’ actual and self-evaluated chances of failure. Specifically, players seem to underestimate high levels of difficulty. The results do not show any influence on difficulty underestimation from the players gender, feelings of self-efficacy, risk aversion or gaming habits.

[1]  R. Caillois.  Les jeux et les hommes : le masque et le vertige , 1967 .

[2]  K. Holyoak,et al.  The Cambridge handbook of thinking and reasoning , 2005 .

[3]  A. Bandura.  Self-efficacy: toward a unifying theory of behavioral change. , 1977, Psychological review.

[4]  Daniel M. Oppenheimer,et al.  Heuristics made easy: an effort-reduction framework. , 2008, Psychological bulletin.

[5]  A. Tversky,et al.  The weighing of evidence and the determinants of confidence , 1992, Cognitive Psychology.

[6]  Nilanjan Sarkar,et al.  Maintaining Optimal Challenge in Computer Games through Real-Time Physiological Feedback , 2005 .

[7]  Robin Hunicke,et al.  The case for dynamic difficulty adjustment in games , 2005, ACE '05.

[8]  S. Dornič,et al.  ON THE PROBLEMS OF PERCEIVED DIFFICULTY , 1971 .

[9]  D. Stone.  Overconfidence in Initial Self-Efficacy Judgments: Effects on Decision Processes and Performance , 1994 .

[10]  Andrew K. Przybylski,et al.  The Motivational Pull of Video Games: A Self-Determination Theory Approach , 2006 .

[11]  J. Famose,et al.  PERCEPTION DE LA DIFFICULTE, ENTROPIE ET PERFORMANCE. , 1992 .

[12]  A. Tversky,et al.  The hot hand in basketball: On the misperception of random sequences , 1985, Cognitive Psychology.

[13]  Rachel T. A. Croson,et al.  Biases in casino betting: The hot hand and the gambler’s fallacy , 2006, Judgment and Decision Making.

[14]  Brad M. Barber,et al.  Boys Will Be Boys: Gender, Overconfidence, and Common Stock Investment , 1998 .

[15]  Shinichi Nakagawa,et al.  A general and simple method for obtaining R2 from generalized linear mixed‐effects models , 2013 .

[16]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[17]  Gilad Chen,et al.  Validation of a New General Self-Efficacy Scale , 2001 .

[18]  Adam S. Goodie,et al.  The Role of Perceived Control and Overconfidence in Pathological Gambling , 2005, Journal of Gambling Studies.

[19]  A. Bandura.  GUIDE FOR CONSTRUCTING SELF-EFFICACY SCALES , 2006 .

[20]  Peta Wyeth,et al.  GameFlow: a model for evaluating player enjoyment in games , 2005, CIE.

[21]  Brian Burton,et al.  The Art of Failure , 2005 .

[22]  Agathe Guilloux,et al.  Difficulty influence on motivation over time in video games using survival analysis , 2017, FDG.

[23]  Daniel Afergan,et al.  Dynamic difficulty using brain metrics of workload , 2014, CHI.

[24]  Rachel T. A. Croson,et al.  The Gambler’s Fallacy and the Hot Hand: Empirical Data from Casinos , 2005 .

[25]  Meynaud Jean,et al.  Caillois (Roger) - Les Jeux et les Hommes (Le Masque et la Vertige) , 1958 .

[26]  Joseph H. Goldberg,et al.  Computer interface evaluation using eye movements: methods and constructs , 1999 .

[27]  Young Joon Park,et al.  Overconfidence in tournaments: evidence from the field , 2010 .

[28]  Didier Delignières,et al.  INFLUENCE OF PHYSICAL EXERCISE ON CHOICE REACTION TIME IN SPORTS EXPERTS : THE MEDIATING ROLE OF RESOURCE ALLOCATION , 2003 .

[29]  Charles A. Holt,et al.  Risk Aversion and Incentive Effects , 2002 .

[30]  Gideon Keren,et al.  On The Calibration of Probability Judgments: Some Critical Comments and Alternative Perspectives , 1997 .

[31]  Gideon Keren,et al.  Facing uncertainty in the game of bridge: A calibration study , 1987 .

[32]  Robert Zubek,et al.  MDA : A Formal Approach to Game Design and Game Research , 2004 .

[33]  Carl Gutwin,et al.  Now You Can Compete With Anyone: Balancing Players of Different Skill Levels in a First-Person Shooter Game , 2015, CHI.

[34]  Stéphane Natkin,et al.  Difficulty in videogames: an experimental validation of a formal definition , 2011, Advances in Computer Entertainment Technology.

[35]  Kim Mouridsen,et al.  Experienced poker players differ from inexperienced poker players in estimation bias and decision bias , 2010 .

[36]  Jack B. Soll,et al.  Overconfidence: It Depends on How, What, and Whom You Ask. , 1999, Organizational behavior and human decision processes.

[37]  Andrew B. Slifkin,et al.  Aiming for the future: prospective action difficulty, prescribed difficulty, and Fitts’ law , 2006, Experimental Brain Research.

[38]  B. Tversky,et al.  Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. , 2011, Psychophysiology.

[39]  Briony D. Pulford,et al.  Overconfidence : Feedback and item difficulty effects , 1997 .

[40]  Thomas W. Malone,et al.  Heuristics for designing enjoyable user interfaces: Lessons from computer games , 1982, CHI '82.

[41]  Vincent Corruble,et al.  Extending Reinforcement Learning to Provide Dynamic Game Balancing , 2005 .

[42]  M. Sujan,et al.  Skill-based versus effort-based task difficulty: A task-analysis approach to the role of specific emotions in motivating difficult actions , 2012 .

[43]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .

[44]  N. Lazzaro.  Why we Play Games: Four Keys to More Emotion without Story , 2004 .

[45]  J. Famose,et al.  VALIDATION D'UNE ECHELLE DE CATEGORIES POUR LA PERCEPTION DE LA DIFFICULTE. , 1994 .

[46]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[47]  M. Csíkszentmihályi,et al.  The Concept of Flow , 2014 .

[48]  D. Kahneman,et al.  A model of heuristic judgment , 2005 .

[49]  Paul V Stenis,et al.  Uncertainty in Games , 2013 .

[50]  A. Tversky,et al.  Subjective Probability: A Judgment of Representativeness , 1972 .

[51]  Véronique Bessière,et al.  Excès de confiance des dirigeants et décisions financières:une synthèse , 2007 .

[52]  Jesper Juul,et al.  The game, the player, the world: looking for a heart of gameness , 2010, DiGRA Conference.

[53]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[54]  Didier Delignières,et al.  A psychophysical study of difficulty rating in rock climbing. , 1993 .

[55]  Jonathan Parke,et al.  The Psychology of the Fruit Machine: The Role of Structural Characteristics (Revisited) , 2006, International Journal of Mental Health and Addiction.

[56]  H. Arkes,et al.  Two methods of reducing overconfidence , 1987 .

[57]  Staffan Björk,et al.  Game Research Methods. An Overview , 2015 .

[58]  Jesper Juul,et al.  A Casual Revolution: Reinventing Video Games and Their Players , 2009 .

[59]  Dominic D. P. Johnson,et al.  The evolution of overconfidence , 2009, Nature.

[60]  Miguel A. Vadillo,et al.  Illusion of Control , 2013, Experimental psychology.

[61]  D. Moore,et al.  The trouble with overconfidence. , 2008, Psychological review.