Cooperation or competition - When do people contribute more? A field experiment on gamification of crowdsourcing

Abstract Information technology is being increasingly employed to harness under-utilized resources via more effective coordination. This progress has manifested in different developments, for instance, crowdsourcing (e.g. Wikipedia, Amazon Mechanical Turk, and Waze), crowdfunding (e.g. Kickstarter, Indiegogo, and RocketHub) or the sharing economy (e.g. Uber, Airbnb, and Didi Chuxing). Since the sustainability of these IT-enabled forms of resource coordination do not commonly rely merely on direct economic benefits of the participants, but also on other non-monetary, intrinsic gratifications, such systems are increasingly gamified that is, designers use features of games to induce enjoyment and general autotelicy of the activity. However, a key problem in gamification design has been whether it is better to use competition-based or cooperation-based designs. We examine this question through a field experiment in a gamified crowdsourcing system, employing three versions of gamification: competitive, cooperative, and inter-team competitive gamification. We study these gamified conditions’ effects on users’ perceived enjoyment and usefulness of the system as well as on their behaviors (system usage, crowdsourcing participation, engagement with the gamification feature, and willingness to recommend the crowdsourcing application). The results reveal that inter-team competitions are most likely to lead to higher enjoyment and crowdsourcing participation, as well as to a higher willingness to recommending a system. Further, the findings indicate that designers should consider cooperative instead of competitive approaches to increase users’ willingness to recommend crowdsourcing systems. These insights add relevant findings to the ongoing discourse on the roles of different types of competitions in gamification designs and suggest that crowdsourcing system designers and operators should implement gamification with competing teams instead of typically used competitions between individuals.

[1]  Oded Nov,et al.  Analysis of participation in an online photo-sharing community: A multidimensional perspective , 2010 .

[2]  Juho Hamari,et al.  Computer Games and Organization Studies , 2017 .

[3]  Thomas W. Malone,et al.  Heuristics for designing enjoyable user interfaces: Lessons from computer games , 1982, CHI '82.

[4]  Manuel J. Fonseca,et al.  Improving student creativity with gamification and virtual worlds , 2013, Gamification.

[5]  Rimantas Gatautis,et al.  Crowdsourcing Application in Marketing Activities , 2014 .

[6]  Deborah I. Fels,et al.  Gamification in theory and action: A survey , 2015, Int. J. Hum. Comput. Stud..

[7]  Thomas W. Malone,et al.  Toward a Theory of Intrinsically Motivating Instruction , 1981, Cogn. Sci..

[8]  Maria Luiza Machado Campos,et al.  Designing a gamification mechanism to encourage contributions in a crowdsourcing system , 2015, 2015 IEEE 19th International Conference on Computer Supported Cooperative Work in Design (CSCWD).

[9]  Anthony D. Williams,et al.  Macrowikinomics: Rebooting Business and the World , 2010 .

[10]  Kay W. Axhausen,et al.  Effectiveness of the Parking Guidance System in Frankfurt am Main , 1994 .

[11]  Peter Akinsola Okebukola,et al.  Impact of Extended Cooperative and Competitive Relationships on the Performance of Students in Science , 1986 .

[12]  David W. Johnson,et al.  Social interdependence: interrelationships among theory, research, and practice. , 2003, The American psychologist.

[13]  Matthew K. O. Lee,et al.  Exploring online social behavior in crowdsourcing communities: A relationship management perspective , 2014, Comput. Hum. Behav..

[14]  Qinghua Zhu,et al.  Effects of extrinsic and intrinsic motivation on participation in crowdsourcing contest: A perspective of self-determination theory , 2014, Online Inf. Rev..

[15]  Juho Hamari,et al.  Do badges increase user activity? A field experiment on the effects of gamification , 2017, Comput. Hum. Behav..

[16]  Steffen P. Walz,et al.  Gamification and Motivation , 2014 .

[17]  David R. Millen,et al.  Removing gamification from an enterprise SNS , 2012, CSCW.

[18]  Patrick Weber,et al.  OpenStreetMap: User-Generated Street Maps , 2008, IEEE Pervasive Computing.

[19]  Marc Jay,et al.  Using crowdsourcing to support pro-environmental community activism , 2013, CHI.

[20]  R. Nagel,et al.  The Effect of Intergroup Competition on Group Coordination: An Experimental Study , 1999 .

[21]  Juho Hamari,et al.  Player Types: A Meta-synthesis , 2014, Trans. Digit. Games Res. Assoc..

[22]  Christian Kray,et al.  A Gamification Framework for Volunteered Geographic Information , 2015, AGILE Conf..

[23]  Juho Hamari,et al.  A definition for gamification: anchoring gamification in the service marketing literature , 2016, Electronic Markets.

[24]  Juho Hamari,et al.  How to design gamification? A method for engineering gamified software , 2017, Inf. Softw. Technol..

[25]  R. Slavin Research on cooperative learning and achievement: What we know, what we need to know. , 1996 .

[26]  Andrew K. Przybylski,et al.  The Motivational Pull of Video Games: A Self-Determination Theory Approach , 2006 .

[27]  D. Larcker,et al.  STRUCTURAL EQUATION MODELS WITH UNOBSERVABLE VARIABLES AND MEASUREMENT ERROR , 1981 .

[28]  Mark O. Riedl,et al.  Collaboration versus competition: Design and evaluation of mechanics for games with a purpose , 2014, FDG.

[29]  Alexandre N. Tuch,et al.  Do points, levels and leaderboards harm intrinsic motivation?: an empirical analysis of common gamification elements , 2013, Gamification.

[30]  Ping Zhang,et al.  Technical opinionMotivational affordances , 2008 .

[31]  Radhika Santhanam,et al.  Digital Games and Beyond: What Happens When Players Compete , 2013, MIS Q..

[32]  Juho Hamari,et al.  Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification , 2014, 2014 47th Hawaii International Conference on System Sciences.

[33]  A. Pentland,et al.  Collective intelligence , 2006, IEEE Comput. Intell. Mag..

[34]  Richard N. Landers,et al.  Gamification of task performance with leaderboards: A goal setting experiment , 2017, Comput. Hum. Behav..

[35]  Reza Curtmola,et al.  Fostering participaction in smart cities: a geo-social crowdsensing platform , 2013, IEEE Communications Magazine.

[36]  Dietmar Plenz,et al.  powerlaw: A Python Package for Analysis of Heavy-Tailed Distributions , 2013, PloS one.

[37]  Padmini Srinivasan,et al.  Quality through flow and immersion: gamifying crowdsourced relevance assessments , 2012, SIGIR '12.

[38]  Marco Roccetti,et al.  Fighting exclusion: a multimedia mobile app with zombies and maps as a medium for civic engagement and design , 2016, Multimedia Tools and Applications.

[39]  Nick Yee,et al.  Motivations for Play in Online Games , 2006, Cyberpsychology Behav. Soc. Netw..

[40]  D. Shoup The High Cost of Free Parking , 1997 .

[41]  Juho Hamari,et al.  Demographic differences in perceived benefits from gamification , 2014, Comput. Hum. Behav..

[42]  Axel Ockenfels,et al.  Types and patterns: an experimental East-West-German comparison of cooperation and solidarity , 1999 .

[43]  Kai Fischbach,et al.  Internal crowdsourcing: conceptual framework, structured review, and research agenda , 2016, J. Inf. Technol..

[44]  Pratik Mishra,et al.  Incentive design to mould online behavior: a game mechanics perspective , 2014, GamifIR '14.

[45]  Ronnie Schöb,et al.  Alleviating Urban Traffic Congestion , 2005 .

[46]  Juho Hamari,et al.  Transforming homo economicus into homo ludens: A field experiment on gamification in a utilitarian peer-to-peer trading service , 2013, Electron. Commer. Res. Appl..

[47]  Yoshihiko Ueda,et al.  Low-grade central osteosarcoma of the rib , 2005, Skeletal Radiology.

[48]  Wael Soliman,et al.  Understanding Continued Use of Crowdsourcing Systems: An Interpretive Study , 2015, J. Theor. Appl. Electron. Commer. Res..

[49]  Marco Gruteser,et al.  Crowdsensing Maps of On-street Parking Spaces , 2013, 2013 IEEE International Conference on Distributed Computing in Sensor Systems.

[50]  Martin Margreiter,et al.  A Concept for Crowdsourcing of In-Vehicle Data to Improve Urban On-Street Parking , 2015 .

[51]  Kam Star Gamification, interdependence, and the moderating effect of personality on performance , 2015 .

[52]  Margaret Gilbert,et al.  Obligation and Joint Commitment , 1999, Utilitas.

[53]  Vaninha Vieira,et al.  Towards a Framework for Gamification Design on Crowdsourcing Systems: The G.A.M.E. Approach , 2015, 2015 12th International Conference on Information Technology - New Generations.

[54]  Francesco G. B. De Natale,et al.  EventMask: A Game-Based Framework for Event-Saliency Identification in Images , 2015, IEEE Transactions on Multimedia.

[55]  Alon Y. Halevy,et al.  Crowdsourcing systems on the World-Wide Web , 2011, Commun. ACM.

[56]  Jennifer A. Epstein,et al.  Winning is Not Enough: The Effects of Competition and Achievement Orientation on Intrinsic Interest , 1992 .

[57]  Rainer Alt,et al.  Increasing Intranet Usage through Gamification -- Insights from an Experiment in the Banking Industry , 2015, 2015 48th Hawaii International Conference on System Sciences.

[58]  Mads Kock Pedersen,et al.  Exploring the quantum speed limit with computer games , 2015, Nature.

[59]  Pearl Pu,et al.  HealthyTogether: exploring social incentives for mobile fitness applications , 2014, Chinese CHI '14.

[60]  Daniel J. Veit,et al.  More than fun and money. Worker Motivation in Crowdsourcing - A Study on Mechanical Turk , 2011, AMCIS.

[61]  E. A. Locke,et al.  Self-regulation through goal setting , 1991 .

[62]  Robbie T. Nakatsu,et al.  A taxonomy of crowdsourcing based on task complexity , 2014, J. Inf. Sci..

[63]  Steffen P. Walz,et al.  Gamification and Culture , 2014 .

[64]  C. Haythornthwaite,et al.  Motivation for Open Collaboration , 2013 .

[65]  Karim R. Lakhani,et al.  Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects , 2003 .

[66]  Ido Erev,et al.  Constructive Intergroup Competition as a Solution to the Free Rider Problem: A Field Experiment , 1993 .

[67]  Kay W. Axhausen,et al.  Effectiveness of the parking guidance information system in Frankfurt am Main , 1994 .

[68]  Avi Goldfarb,et al.  Some Simple Economics of Crowdfunding , 2013, Innovation Policy and the Economy.

[69]  Don Tapscott,et al.  Wikinomics: How Mass Collaboration Changes Everything , 2006 .

[70]  M. Csíkszentmihályi Flow: The Psychology of Optimal Experience , 1990 .

[71]  T. Garry,et al.  An investigation into gamification as a customer engagement experience environment , 2015 .

[72]  Dirk Schneckenberg,et al.  The Ludic Drive as Innovation Driver: Introduction to the Gamification of Innovation , 2015 .

[73]  Juho Hamari,et al.  "Working out for likes": An empirical study on social influence in exercise gamification , 2015, Comput. Hum. Behav..

[74]  E. A. Locke,et al.  Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. , 2002, The American psychologist.

[75]  Juho Hamari,et al.  Gamification in Crowdsourcing: A Review , 2016, 2016 49th Hawaii International Conference on System Sciences (HICSS).

[76]  Juho Hamari,et al.  International Journal of Information Management Why Do People Use Gamification Services? , 2022 .

[77]  Athanasios Mazarakis,et al.  Interaction and Reflection with Quantified Self and Gamification: an Experimental Study , 2014 .

[78]  Wendy Hall,et al.  Because science is awesome: studying participation in a citizen science game , 2016, WebSci.

[79]  Lav R. Varshney,et al.  Participation in crowd systems , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[80]  Jennifer Preece,et al.  Using gamification to inspire new citizen science volunteers , 2013, Gamification.

[81]  Jian Tang,et al.  Points, stories, worlds, and diegesis: Comparing player experiences in two citizen science games , 2015, Comput. Hum. Behav..

[82]  Joseph S. Valacich,et al.  Enhancing the Motivational Affordance of Information Systems: The Effects of Real-Time Performance Feedback and Goal Setting in Group Collaboration Environments , 2010, Manag. Sci..

[83]  F. A. Perry,et al.  Cooperation contrasted with intra-group and inter-group competition. , 1967, Sociometry.

[84]  Marsha L. Richins Negative Word-of-Mouth by Dissatisfied Consumers: A Pilot Study , 1983 .

[85]  Margaret Meiling Luo,et al.  Web-based information service adoption: A comparison of the motivational model and the uses and gratifications theory , 2011, Decis. Support Syst..

[86]  Fernando González-Ladrón-de-Guevara,et al.  Towards an integrated crowdsourcing definition , 2012, J. Inf. Sci..

[87]  Carole A. Ames,et al.  An examination of children's attributions and achievement-related evaluations in competitive, cooperative, and individualistic reward structures. , 1979 .

[88]  D. Shoup Cruising for Parking , 2006 .

[89]  Michael Scharkow,et al.  Beyond the "core-gamer": Genre preferences and gratifications in computer games , 2015, Comput. Hum. Behav..

[90]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[91]  E. Deci,et al.  When Trying to Win , 1981 .

[92]  J. Bittner,et al.  Motivational effects and age differences of gamification in product advertising , 2014 .

[93]  Edward L. Deci,et al.  Elements of the Competitive Situation that Affect Intrinsic Motivation , 1996 .

[94]  Martin Schader,et al.  Personalized task recommendation in crowdsourcing information systems - Current state of the art , 2014, Decis. Support Syst..

[95]  David Coyle,et al.  Competing or aiming to be average?: normification as a means of engaging digital volunteers , 2014, CSCW.

[96]  Sung S. Kim,et al.  Out of Dedication or Constraint? A Dual Model of Post-Adoption Phenomena and its Empirical Test in the Context of Online Services , 2009, MIS Q..

[97]  Peta Wyeth,et al.  GameFlow: a model for evaluating player enjoyment in games , 2005, CIE.

[98]  Ian P. McCarthy,et al.  How to Work a Crowd: Developing Crowd Capital Through Crowdsourcing , 2014, ArXiv.

[99]  Lee Yik Sheng,et al.  Modelling learning from Ingress (Google's augmented reality social game) , 2013, 2013 IEEE 63rd Annual Conference International Council for Education Media (ICEM).

[100]  M. Stephens,et al.  K-Sample Anderson–Darling Tests , 1987 .

[101]  Adrien Treuille,et al.  Predicting protein structures with a multiplayer online game , 2010, Nature.

[102]  N. Sadat Shami,et al.  Experiments on Motivational Feedback for Crowdsourced Workers , 2013, ICWSM.

[103]  Alexander Maedche,et al.  Gamified crowdsourcing: Conceptualization, literature review, and future agenda , 2017, Int. J. Hum. Comput. Stud..

[104]  Cary J. Roseth,et al.  Promoting early adolescents' achievement and peer relationships: the effects of cooperative, competitive, and individualistic goal structures. , 2008, Psychological bulletin.

[105]  Juho Hamari,et al.  The sharing economy: Why people participate in collaborative consumption , 2016, J. Assoc. Inf. Sci. Technol..

[106]  Elizabeth O. Hayward,et al.  The impact of individual, competitive, and collaborative mathematics game play on learning, performance, and motivation , 2013 .

[107]  Oded Nov,et al.  What motivates Wikipedians? , 2007, CACM.

[108]  Youngmee Kim,et al.  Differentiating autonomy from individualism and independence: a self-determination theory perspective on internalization of cultural orientations and well-being. , 2003, Journal of personality and social psychology.

[109]  Nina Runge,et al.  Tags You Don't Forget: Gamified Tagging of Personal Images , 2015, ICEC.

[110]  Qinghua Zhu,et al.  Evaluation on crowdsourcing research: Current status and future direction , 2012, Information Systems Frontiers.

[111]  Tatsuo Nakajima,et al.  Gamifying intelligent environments , 2011, Ubi-MUI '11.

[112]  Hongxiu Li,et al.  Understanding the effects of gratifications on the continuance intention to use WeChat in China: A perspective on uses and gratifications , 2018, Comput. Hum. Behav..

[113]  David W. Johnson,et al.  Cooperation and Competition: Theory and Research , 1989 .

[114]  Gordon B. Davis,et al.  User Acceptance of Information Technology: Toward a Unified View , 2003, MIS Q..

[115]  Sylvester Arnab,et al.  Competition and collaboration using a social and gamified online learning platform , 2016 .

[116]  Andrew K. Przybylski,et al.  A Motivational Model of Video Game Engagement , 2010 .

[117]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[118]  E. Deci,et al.  The "What" and "Why" of Goal Pursuits: Human Needs and the Self-Determination of Behavior , 2000 .

[119]  Jesús M. González-Barahona,et al.  On the Inequality of Contributions to Wikipedia , 2008, Proceedings of the 41st Annual Hawaii International Conference on System Sciences (HICSS 2008).

[120]  Daniel J. Veit,et al.  Gamification - A Novel Phenomenon or a New Wrapping for Existing Concepts? , 2015, ICIS.

[121]  Sjaak Brinkkemper,et al.  REfine: A gamified platform for participatory requirements engineering , 2015, 2015 IEEE 1st International Workshop on Crowd-Based Requirements Engineering (CrowdRE).

[122]  Yogesh Virkar,et al.  Power-law distributions in binned empirical data , 2012, 1208.3524.

[123]  Fred D. Davis Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[124]  Ping Zhang,et al.  Motivational Affordances : Fundamental Reasons for ICT Design and Use , 2007 .

[125]  Steve Benford,et al.  Designing for reportability: sustainable gamification, public engagement, and promoting environmental debate , 2013, Personal and Ubiquitous Computing.

[126]  Judith M. Harackiewicz,et al.  The Effects of Cooperation and Competition on Intrinsic Motivation and Performance. , 2004 .

[127]  Jodi Asbell-Clarke,et al.  Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning , 2016, Comput. Hum. Behav..

[128]  Wei Peng,et al.  The influence of competition, cooperation, and player relationship in a motor performance centered computer game , 2012, Comput. Hum. Behav..

[129]  Luis von Ahn Human Computation , 2008, ICDE.

[130]  Sebastian Deterding,et al.  The Lens of Intrinsic Skill Atoms: A Method for Gameful Design , 2015, Hum. Comput. Interact..

[131]  E. Deci,et al.  Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. , 2000, The American psychologist.

[132]  Ching-I Teng,et al.  Team participation and online gamer loyalty , 2014, Electron. Commer. Res. Appl..

[133]  John M. Carroll,et al.  What Is the Sharing Economy? , 2019, Hustle and Gig.

[134]  John Prpic,et al.  Next Generation Crowdsourcing for Collective Intelligence , 2016, ArXiv.

[135]  E. Hirschman,et al.  Hedonic Consumption: Emerging Concepts, Methods and Propositions , 1982 .

[136]  Alexander Maedche,et al.  Designing Cooperative Gamification: Conceptualization and Prototypical Implementation , 2017, CSCW.

[137]  Araz Taeihagh,et al.  The Fundamentals of Policy Crowdsourcing , 2015, ArXiv.

[138]  Panagiotis G. Ipeirotis,et al.  Quizz: targeted crowdsourcing with a billion (potential) users , 2014, WWW.

[139]  Radhika Santhanam,et al.  Toward Meaningful Engagement: A Framework for Design and Research of Gamified Information Systems , 2017, MIS Q..

[140]  Hengchang Liu,et al.  SmartRoad , 2015, ACM Trans. Sens. Networks.

[141]  Mark Van Vugt,et al.  The Tribal Instinct Hypothesis , 2009 .

[142]  John C. Turner,et al.  Social comparison and social identity: Some prospects for intergroup behaviour , 1975 .

[143]  Manuel Arriaga,et al.  Distinction and Status Production on User-Generated Content Platforms: Using Bourdieu's Theory of Cultural Production to Understand Social Dynamics in Online Fields , 2014, Inf. Syst. Res..

[144]  Alexander Maedche,et al.  How games induce cooperation? A study on the relationship between game features and we-intentions in an augmented reality game , 2017, Comput. Hum. Behav..

[145]  Henry L. Tosi A Theory of Goal Setting and Task Performance , 1991 .

[146]  Premkumar T. Devanbu,et al.  How social Q&A sites are changing knowledge sharing in open source software communities , 2014, CSCW.

[147]  Laura A. Dabbish,et al.  Designing games with a purpose , 2008, CACM.

[148]  Michael Friedewald,et al.  The Illusion of Security: A fiction scenario of daily life , 2008 .

[149]  Richard M. Ryan,et al.  Glued to Games: How Video Games Draw Us In and Hold Us Spellbound , 2011 .

[150]  Mark E. J. Newman,et al.  Power-Law Distributions in Empirical Data , 2007, SIAM Rev..

[151]  Paul Denny,et al.  The effect of virtual achievements on student engagement , 2013, CHI.