Deconstructing gamification: evaluating the effectiveness of continuous measurement, virtual rewards, and social comparison for promoting physical activity

Game design elements are often implemented in persuasive systems aimed to promote physical activity, a process called “gamification.” Gamification is believed to motivate users to become more active, and is commonly implemented in commercial products. However, relatively few studies rigorously evaluated the effectiveness of gamification, and they yielded contradicting findings. We set out to evaluate the effectiveness of virtual rewards and social comparison—two game elements prevalent in persuasive systems. We developed a research prototype, called “StepByStep,” aimed to promote routine walking. We created different versions of StepByStep, implemented as an application on Android-based mobile devices, and compared their effectiveness in two field studies. Study 1 showed that a quantified version of the application—offering continuous measurement of walking time, a daily goal, and real-time feedback on progress toward this goal—facilitated reflection on activity and significantly increased walking time over baseline level. Study 2 showed that gamified versions offering virtual rewards and social comparison were only as effective as the quantified version. Thus, we advise designers to facilitate reflection on meaningful aspects of physical activity by developing novel ubiquitous measures. Furthermore, our findings highlight the importance of systematic comparisons between quantified and gamified elements for better understanding their motivational affordances.

[1]  Emre Ertin,et al.  An unobtrusively wearable sensor suite for inferring the onset, causality, and consequences of stress in the field , 2011, SenSys.

[2]  Silvia Lindtner,et al.  Fish'n'Steps: Encouraging Physical Activity with an Interactive Computer Game , 2006, UbiComp.

[3]  Steven N. Blair,et al.  Accumulated versus Continuous Exercise for Health Benefit , 2009, Sports medicine.

[4]  B Stetd,et al.  TAKING STEPS TOWARD INCREASED PHYSICAL ACTIVITY: USING PEDOMETERS TO MEASURE AND MOTIVATE , 2002 .

[5]  Jonna Häkkilä,et al.  Let's all get up and walk to the North Pole: design and evaluation of a mobile wellness application , 2010, NordiCHI.

[6]  Maria Hagströmer,et al.  The descriptive epidemiology of sitting. A 20-country comparison using the International Physical Activity Questionnaire (IPAQ). , 2011, American journal of preventive medicine.

[7]  Andrew D. Miller,et al.  Designing pervasive health games for sustainability, adaptability and sociability , 2012, FDG.

[8]  E. A. Locke,et al.  Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. , 2002, The American psychologist.

[9]  I. Olkin,et al.  Using pedometers to increase physical activity and improve health: a systematic review. , 2007, JAMA.

[10]  Catrine Tudor-Locke,et al.  Why Do Pedometers Work? , 2009, Sports medicine.

[11]  I. Janssen,et al.  Systematic review of the health benefits of physical activity and fitness in school-aged children and youth , 2010, The international journal of behavioral nutrition and physical activity.

[12]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination , 2004 .

[13]  Andrew D. Miller,et al.  This is not a one-horse race: understanding player types in multiplayer pervasive health games for youth , 2012, CSCW.

[14]  R. J. Bogumil,et al.  The reflective practitioner: How professionals think in action , 1985, Proceedings of the IEEE.

[15]  William G. Griswold,et al.  Usability and Feasibility of PmEB: A Mobile Phone Application for Monitoring Real Time Caloric Balance , 2006, Mob. Networks Appl..

[16]  Scott Nicholson A User-Centered Theoretical Framework for Meaningful Gamification , 2012 .

[17]  M. Harjumaa,et al.  A Systematic Framework for Designing and Evaluating Persuasive Systems , 2008, PERSUASIVE.

[18]  Amanda E Staiano,et al.  The descriptive epidemiology of sitting among US adults, NHANES 2009/2010. , 2014, Journal of science and medicine in sport.

[19]  Aino Ahtinen,et al.  User experiences of mobile wellness applications in health promotion: User study of Wellness Diary, Mobile Coach and SelfRelax , 2009, 2009 3rd International Conference on Pervasive Computing Technologies for Healthcare.

[20]  Sean A. Munson,et al.  Exploring goal-setting, rewards, self-monitoring, and sharing to motivate physical activity , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[21]  S. Koch,et al.  On Health-enabling and Ambient-assistive Technologies , 2009, Methods of Information in Medicine.

[22]  Andrea Grimes Parker,et al.  Reflection-through-performance: personal implications of documenting health behaviors for the collective , 2014, Personal and Ubiquitous Computing.

[23]  Tingting Zhao,et al.  Exploring teenagers' motivation to exercise through technology probes , 2011, BCS HCI.

[24]  Skill Atoms as Design Lenses for User-Centered Gameful Design , .

[25]  S. Biddle,et al.  Physical activity and mental health in children and adolescents: a review of reviews , 2011, British Journal of Sports Medicine.

[26]  Harri Oinas-Kukkonen,et al.  Social interaction and reflection for behaviour change , 2014, Personal and Ubiquitous Computing.

[27]  James Fogarty,et al.  Game design principles in everyday fitness applications , 2008, CSCW.

[28]  Eric Baumer,et al.  Prescriptive persuasion and open-ended social awareness: expanding the design space of mobile health , 2012, CSCW.

[29]  D. Schoen,et al.  The Reflective Practitioner: How Professionals Think in Action , 1985 .

[30]  Arie W. Kruglanski,et al.  Classic and current social comparison research: Expanding the perspective. , 1990 .

[31]  Ananda Sen,et al.  A Meta-Analysis of Pedometer-Based Walking Interventions and Weight Loss , 2008, The Annals of Family Medicine.

[32]  L. Festinger A Theory of Social Comparison Processes , 1954 .

[33]  P. Bréchat,et al.  Health benefits of physical activity in older patients: a review , 2009, International journal of clinical practice.

[34]  B. Thomas,et al.  Usability Evaluation In Industry , 1996 .

[35]  C. Abraham,et al.  Effective techniques in healthy eating and physical activity interventions: a meta-regression. , 2009, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[36]  David W. McDonald,et al.  Goal-setting considerations for persuasive technologies that encourage physical activity , 2009, Persuasive '09.

[37]  David W. McDonald,et al.  Activity sensing in the wild: a field trial of ubifit garden , 2008, CHI.

[38]  Harri Oinas-Kukkonen,et al.  Persuasive software design patterns for social influence , 2014, Personal and Ubiquitous Computing.

[39]  Ana M. Bernardos,et al.  Activity logging using lightweight classification techniques in mobile devices , 2012, Personal and Ubiquitous Computing.

[40]  Jodi Forlizzi,et al.  A stage-based model of personal informatics systems , 2010, CHI.

[41]  Aart van Halteren,et al.  Toward a persuasive mobile application to reduce sedentary behavior , 2013, Personal and Ubiquitous Computing.

[42]  Tomas Philipson,et al.  The growth of obesity and technological change. , 2009, Economics and human biology.

[43]  E. Churchill,et al.  Badges in Social Media: A Social Psychological Perspective , 2011 .

[44]  Rebecca E. Grinter,et al.  Let's play!: mobile health games for adults , 2010, UbiComp.

[45]  Kenton O'Hara,et al.  Gamification. using game-design elements in non-gaming contexts , 2011, CHI Extended Abstracts.

[46]  Aino Ahtinen User-centered design of mobile wellness applications , 2009, Mobile HCI.

[47]  Staffan Björk,et al.  Designing gamification: creating gameful and playful experiences , 2013, CHI Extended Abstracts.

[48]  Juho Hamari,et al.  Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification , 2014, 2014 47th Hawaii International Conference on System Sciences.

[49]  Valérie Gay,et al.  A mobile rehabilitation application for the remote monitoring of cardiac patients after a heart attack or a coronary bypass surgery , 2009, PETRA '09.

[50]  B. J. Fogg,et al.  Persuasive technology: using computers to change what we think and do , 2002, UBIQ.

[51]  Isaac Wiafe,et al.  Categorizing users in behavior change support systems based on cognitive dissonance , 2014, Personal and Ubiquitous Computing.

[52]  Tatsuo Nakajima,et al.  Gamifying intelligent environments , 2011, Ubi-MUI '11.

[53]  Matthias Rauterberg,et al.  Unconscious emotions: quantifying and logging something we are not aware of , 2013, Personal and Ubiquitous Computing.

[54]  Rodrigo de Oliveira,et al.  TripleBeat: enhancing exercise performance with persuasion , 2008, Mobile HCI.

[55]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[56]  Alexandre N. Tuch,et al.  Disassembling gamification: the effects of points and meaning on user motivation and performance , 2013, CHI Extended Abstracts.

[57]  Sebastian Deterding,et al.  Situated motivational affordances of game elements: A conceptual model , 2011 .

[58]  E. Deci,et al.  Extrinsic Rewards and Intrinsic Motivation in Education: Reconsidered Once Again , 2001 .

[59]  David W. McDonald,et al.  Flowers or a robot army?: encouraging awareness & activity with personal, mobile displays , 2008, UbiComp.

[60]  James A. Landay,et al.  Design requirements for technologies that encourage physical activity , 2006, CHI.

[61]  Lennart E. Nacke,et al.  From game design elements to gamefulness: defining "gamification" , 2011, MindTrek.

[62]  John Zimmerman,et al.  I'm the mayor of my house: examining why people use foursquare - a social-driven location sharing application , 2011, CHI.

[63]  Donald A. Sch The reflective practitioner: how professionals think in action , 1983 .

[64]  R. Davison,et al.  The efficacy of accumulated short bouts versus single daily bouts of brisk walking in improving aerobic fitness and blood lipid profiles. , 1999, Health education research.

[65]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[66]  M. Fogelholm Physical activity, fitness and fatness: relations to mortality, morbidity and disease risk factors. A systematic review , 2010, Obesity reviews : an official journal of the International Association for the Study of Obesity.

[67]  D. French,et al.  What is the best way to change self-efficacy to promote lifestyle and recreational physical activity? A systematic review with meta-analysis. , 2010, British journal of health psychology.

[68]  Wanda Pratt,et al.  How to evaluate technologies for health behavior change in HCI research , 2011, CHI.