The Agony of Choice - Analyzing User Preferences regarding Gamification Elements in Learning Management Systems

Gamification is an approach to motivate users of information systems by implementing game elements. Despite its potentials, the use of gamification in practice has also received criticism because most gamification elements are integrated into systems without considering the preferences of the respective users and without considering contextual aspects. Hence, we aim at identifying which elements users of learning management systems prefer. Furthermore, we want to identify their preferences regarding the amount and combination of gamification elements. To identify user preferences, we conducted a discrete choice task among learning management system users following the best-worst scaling method. Our results show that users prefer a bundle of four gamification elements: level, goals, status, and points. This is especially interesting since popular elements, such as a leaderboard or badges, are not most preferred. This highlights the need to consider user preferences and contextual aspects to ensure a successful implementation of gamification elements.

[1]  Gabor Istvan Biro READY, STUDY, SHARE: AN INQUIRY INTO THE DIDACTIC APPROACH OF GAMIFICATION WITH A SPECIAL VIEW TO THE POSSIBLE APPLICATION IN HIGHER EDUCATION , 2013 .

[2]  E. Deci,et al.  Facilitating internalization: the self-determination theory perspective. , 1994, Journal of personality.

[3]  Hans D. Mummendey,et al.  Psychologie der Selbstdarstellung , 1990 .

[4]  Ryan Shaun Joazeiro de Baker,et al.  Modeling how incoming knowledge, persistence, affective states, and in-game progress influence student learning from an educational game , 2015, Comput. Educ..

[5]  Juho Hamari,et al.  Demographic differences in perceived benefits from gamification , 2014, Comput. Hum. Behav..

[6]  Nicola J. Gibbons,et al.  The interactivity effect in multimedia learning , 2007, Comput. Educ..

[7]  Wei-Tsong Wang,et al.  Motivations of employees' knowledge sharing behaviors: A self-determination perspective , 2015, Inf. Organ..

[8]  Z. Demetrovics,et al.  Why do you play? The development of the motives for online gaming questionnaire (MOGQ) , 2011, Behavior research methods.

[9]  J. Louviere,et al.  Some probabilistic models of best, worst, and best–worst choices , 2005 .

[10]  Juho Hamari,et al.  Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification , 2014, 2014 47th Hawaii International Conference on System Sciences.

[11]  J. Lee,et al.  The Best–Worst Scaling Approach: An Alternative to Schwartz's Values Survey , 2008, Journal of personality assessment.

[12]  Katie Davis,et al.  Digital badges in afterschool learning: Documenting the perspectives and experiences of students and educators , 2015, Comput. Educ..

[13]  Gülsün Atanur Baskan,et al.  The Importance of Motivation Theories in Terms Of Education Systems , 2015 .

[14]  Esteban Walter Gonzalez Clua,et al.  Turning Real-World Software Development into a Game , 2011, 2011 Brazilian Symposium on Games and Digital Entertainment.

[15]  Paul J. Krause,et al.  Tiger Nation: Empowering citizen scientists , 2012, 2012 6th IEEE International Conference on Digital Ecosystems and Technologies (DEST).

[16]  Deborah I. Fels,et al.  Gamification in theory and action: A survey , 2015, Int. J. Hum. Comput. Stud..

[17]  Dorothy E. Leidner,et al.  Research Commentary: Technology-Mediated Learning - A Call for Greater Depth and Breadth of Research , 2001, Inf. Syst. Res..

[18]  Monica Aresta,et al.  Peer-supported badge attribution in a collaborative learning platform: The SAPO Campus case , 2015, Comput. Hum. Behav..

[19]  Ivica Boticki,et al.  Usage of a mobile social learning platform with virtual badges in a primary school , 2015, Comput. Educ..

[20]  Shari S. C. Shang,et al.  An Understanding of the Impact of Gamification on Purchase Intentions , 2013, AMCIS.

[21]  Ali Sunyaev,et al.  Cloud Service Certifications: Measuring Consumers' Preferences For Assurances , 2013, ECIS.

[22]  Matthias Söllner,et al.  How to Gamify Information Systems - Adapting Gamification to Individual Preferences , 2016, ECIS.

[23]  D. Bacon A Comparison of Approaches to Importance-Performance Analysis , 2003 .

[24]  Alf Inge Wang,et al.  The wear out effect of a game-based student response system , 2015, Comput. Educ..

[25]  Wei-Cheng Milton Shen,et al.  Gamification of Technology-Mediated Training : Not All Competitions Are the Same Radhika Santhanam , 2016 .

[26]  Jordan J. Louviere,et al.  An introduction to the application of (case 1) best–worst scaling in marketing research , 2013 .

[27]  Alexandre N. Tuch,et al.  Towards understanding the effects of individual gamification elements on intrinsic motivation and performance , 2017, Comput. Hum. Behav..

[28]  Jörg Schmidtke,et al.  Eliciting preferences for priority setting in genetic testing: a pilot study comparing best-worst scaling and discrete-choice experiments , 2013, European Journal of Human Genetics.

[29]  Jordan J. Louviere,et al.  Measuring values using best‐worst scaling: The LOV example , 2007 .

[30]  Chun-Chia Lee,et al.  Designing Attractive Gamification Features for Collaborative Storytelling Websites , 2013, Cyberpsychology Behav. Soc. Netw..

[31]  Tovi Grossman,et al.  GamiCAD: a gamified tutorial system for first time autocad users , 2012, UIST.

[32]  Xianggui Qu,et al.  Multivariate Data Analysis , 2007, Technometrics.

[33]  Ari Korhonen,et al.  How (not) to introduce badges to online exercises , 2014, SIGCSE.

[34]  Eva García,et al.  On the effectiveness of game-like and social approaches in learning: Comparing educational gaming, gamification & social networking , 2016, Comput. Educ..

[35]  T. Peters,et al.  Best--worst scaling: What it can do for health care research and how to do it. , 2007, Journal of health economics.

[36]  Gregory J Madden,et al.  The FIT Game: preliminary evaluation of a gamification approach to increasing fruit and vegetable consumption in school. , 2014, Preventive medicine.

[37]  Jodi Asbell-Clarke,et al.  Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning , 2016, Comput. Hum. Behav..

[38]  Rod Sims,et al.  Promises of Interactivity: Aligning Learner Perceptions and Expectations With Strategies for Flexible and Online Learning , 2003 .

[39]  E. Deci,et al.  Need Satisfaction, Motivation, and Well-Being in the Work Organizations of a Former Eastern Bloc Country: A Cross-Cultural Study of Self-Determination , 2001 .

[40]  Tatsuo Nakajima,et al.  Gamifying intelligent environments , 2011, Ubi-MUI '11.

[41]  Carlos Cruz,et al.  The need to achieve: Players' perceptions and uses of extrinsic meta-game reward systems for video game consoles , 2017, Comput. Hum. Behav..

[42]  James F. Manwell,et al.  Mechanics and Dynamics , 2010 .

[43]  M. Lepper,et al.  Intrinsic and Extrinsic Motivational Orientations in the Classroom: Age Differences and Academic Correlates , 2005 .

[44]  Michael Amberg,et al.  A Descriptive Literature Review and Classification Framework for Gamification in Information Systems , 2015, ECIS.

[45]  Lennart E. Nacke,et al.  From game design elements to gamefulness: defining "gamification" , 2011, MindTrek.

[46]  R. Vallerand,et al.  Development and validation of the Gaming Motivation Scale (GAMS) , 2012 .

[47]  Usef Faghihi,et al.  How Gamification Applies for Educational Purpose Specially with College Algebra , 2014, BICA.

[48]  Marianna Sigala,et al.  The application and impact of gamification funware on trip planning and experiences: the case of TripAdvisor’s funware , 2015, Electron. Mark..

[49]  P. Wouters,et al.  A meta-analysis of the cognitive and motivational effects of serious games , 2013 .

[50]  Lei Shi,et al.  Motivational Gamification Strategies Rooted in Self-Determination Theory for Social Adaptive E-Learning , 2016, ITS.

[51]  Ig Ibert Bittencourt,et al.  A Quantitative Analysis of the Most Relevant Gamification Elements in an Online Learning Environment , 2016, WWW.

[52]  Eduardo Salas,et al.  Toward a Taxonomy Linking Game Attributes to Learning , 2012 .

[53]  Frederik Cornillie,et al.  Concepts and Mechanics for Educational Mini-Games A Human-Centred Conceptual Design Approach involving Adolescent Learners and Domain Experts , 2012 .

[54]  Christian W. Scheiner,et al.  The Backbone of Gamification - a Theoretical Consideration of Play and Game Mechanics , 2013, GI-Jahrestagung.

[55]  Eva Jereb,et al.  The Model for Introduction of Gamification into E-learning in Higher Education , 2015 .

[56]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[57]  J. Louviere,et al.  Determining the Appropriate Response to Evidence of Public Concern: The Case of Food Safety , 1992 .

[58]  Bryan Orme uracy of HB Estimation in MaxDiff Experiments , 2005 .

[59]  Christian Burgers,et al.  How feedback boosts motivation and play in a brain-training game , 2015, Comput. Hum. Behav..

[60]  L. Thurstone,et al.  A low of comparative judgement , 1927 .

[61]  J. Hair Multivariate data analysis : a global perspective , 2010 .

[62]  Tomek Strzalkowski,et al.  We don't need no stinkin' badges: The impact of reward features and feeling rewarded in educational games , 2015, Comput. Hum. Behav..

[63]  J. Arbaugh How Classroom Environment and Student Engagement Affect Learning in Internet-based MBA Courses , 2000 .

[64]  Robert P. Bostrom,et al.  Technology-Mediated Learning: A Comprehensive Theoretical Model , 2009, J. Assoc. Inf. Syst..

[65]  Francisco Luis Gutiérrez Vela,et al.  Analysis and application of gamification , 2012, Interacción.

[66]  Katheryn R. Christy,et al.  Leaderboards in a virtual classroom: A test of stereotype threat and social comparison explanations for women's math performance , 2014, Comput. Educ..

[67]  Andrew J. Stapleton,et al.  Serious Games: Serious Opportunities , 2004 .

[68]  Matthias Söllner,et al.  StaySmart - Individuelles und kompetenzorientiertes e-Learning im Zeitalter des demografischen Wandels , 2016, DeLFI.

[69]  Radhika Santhanam,et al.  Digital Games and Beyond: What Happens When Players Compete , 2013, MIS Q..

[70]  Davinia Hernández Leo,et al.  Group-based mobile learning: Do group size and sharing mobile devices matter? , 2015, Comput. Hum. Behav..

[71]  Christian W. Scheiner The Motivational Fabric of Gamified Idea Competitions: The Evaluation of Game Mechanics from a Longitudinal Perspective , 2015 .

[72]  Khe Foon Hew,et al.  Engaging Asian students through game mechanics: Findings from two experiment studies , 2016, Comput. Educ..

[73]  Gert Breitfuss,et al.  The "ecoGator" app: gamification for enhanced energy efficiency in Europe , 2014, TEEM '14.

[74]  Yigal Attali,et al.  Gamification in assessment: Do points affect test performance? , 2015, Comput. Educ..

[75]  Jesse Fox,et al.  Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance , 2015, Comput. Educ..

[76]  Juho Hamari,et al.  Transforming homo economicus into homo ludens: A field experiment on gamification in a utilitarian peer-to-peer trading service , 2013, Electron. Commer. Res. Appl..

[77]  Hsiao-Ching She,et al.  Learning through playing Virtual Age: Exploring the interactions among student concept learning, gaming performance, in-game behaviors, and the use of in-game characters , 2015, Comput. Educ..

[78]  E. Deci,et al.  The "What" and "Why" of Goal Pursuits: Human Needs and the Self-Determination of Behavior , 2000 .

[79]  Xiaojuan Ma,et al.  Developing a Comprehensive Engagement Framework of Gamification for Reflective Learning , 2016, Conference on Designing Interactive Systems.

[80]  Daniel T. Hickey,et al.  A multilevel analysis of the effects of external rewards on elementary students' motivation, engagement and learning in an educational game , 2014, Comput. Educ..

[81]  Luis de Marcos,et al.  Gamifying learning experiences: Practical implications and outcomes , 2013, Comput. Educ..

[82]  Juho Hamari,et al.  Defining gamification: a service marketing perspective , 2012, MindTrek.

[83]  Yoshinori Adachi,et al.  Development of Web Learning Support System using "My Dictionary" in English Study , 2015, KES.

[84]  Jan Marco Leimeister,et al.  Why different trust relationships matter for information systems users , 2016, Eur. J. Inf. Syst..

[85]  Carlos Delgado Kloos,et al.  Gamification for Engaging Computer Science Students in Learning Activities: A Case Study , 2014, IEEE Transactions on Learning Technologies.

[86]  Bernadette Perry,et al.  Gamifying French Language Learning: a case study examining a quest-based, augmented reality mobile learning-tool , 2015 .

[87]  Miguel Mira da Silva,et al.  iThink: A Game-Based Approach Towards Improving Collaboration and Participation in Requirement Elicitation , 2012, VS-GAMES.

[88]  Darren Wilson,et al.  Gamification Challenges and a Case Study in Online Learning , 2015 .

[89]  Paul E. Green,et al.  Conjoint Analysis in Marketing: New Developments with Implications for Research and Practice , 1990 .

[90]  Moritz von Hoffen,et al.  A Method for Measuring User Preferences in Information Systems Design Choices , 2015, ECIS.

[91]  Traci J. Hess,et al.  Reliability Generalization of Perceived Ease of Use, Perceived Usefulness, and Behavioral Intentions , 2014, MIS Q..

[92]  Henriette Cramer,et al.  Performing a check-in: emerging practices, norms and 'conflicts' in location-sharing using foursquare , 2011, Mobile HCI.

[93]  James M. Boyle,et al.  A systematic literature review of empirical evidence on computer games and serious games , 2012, Comput. Educ..

[94]  Jan Marco Leimeister,et al.  Gamification , 2013, Business & Information Systems Engineering.

[95]  Woonhee Sung,et al.  GREENIFY , 2013 .

[96]  Anna‐Lise Smith,et al.  Getting a clue: creating student detectives and dragon slayers in your library , 2011 .

[97]  W. Stephenson CORRELATING PERSONS INSTEAD OF TESTS , 1935 .

[98]  Sebastian Lins,et al.  Gamifying Information Systems - a synthesis of Gamification mechanics and Dynamics , 2014, ECIS.

[99]  Steven H. Cohen Maximum Difference Scaling: Improved Measures of Importance and Preference for Segmentation , 2003 .

[100]  Rebeca P. Díaz Redondo,et al.  A social gamification framework for a K-6 learning platform , 2013, Comput. Hum. Behav..