Money, glory and cheap talk: analyzing strategic behavior of contestants in simultaneous crowdsourcing contests on TopCoder.com

Crowdsourcing is a new Web phenomenon, in which a firm takes a function once performed in-house and outsources it to a crowd, usually in the form of an open contest. Designing efficient crowdsourcing mechanisms is not possible without deep understanding of incentives and strategic choices of all participants. This paper presents an empirical analysis of determinants of individual performance in multiple simultaneous crowdsourcing contests using a unique dataset for the world's largest competitive software development portal: TopCoder.com. Special attention is given to studying the effects of the reputation system currently used by TopCoder.com on behavior of contestants. We find that individual specific traits together with the project payment and the number of project requirements are significant predictors of the final project quality. Furthermore, we find significant evidence of strategic behavior of contestants. High rated contestants face tougher competition from their opponents in the competition phase of the contest. In order to soften the competition, they move first in the registration phase of the contest, signing up early for particular projects. Although registration in TopCoder contests is non-binding, it deters entry of opponents in the same contest; our lower bound estimate shows that this strategy generates significant surplus gain to high rated contestants. We conjecture that the reputation + cheap talk mechanism employed by TopCoder has a positive effect on allocative efficiency of simultaneous all-pay contests and should be considered for adoption in other crowdsourcing platforms.

[1]  L. Hansen.  Large Sample Properties of Generalized Method of Moments Estimators , 1982 .

[2]  Gregory K. Leonard,et al.  Competitive Analysis with Differentiated Products , 1994 .

[3]  Panagiotis G. Ipeirotis,et al.  The Dimensions of Reputation in Electronic Markets , 2009 .

[4]  Minos N. Garofalakis,et al.  MashMaker: mashups for the masses , 2007, SIGMOD '07.

[5]  Frank Kleemann,et al.  Un(der)paid innovators: the commercial utilization of consumer work through crowdsourcing , 2008 .

[6]  Samer Faraj,et al.  Why Should I Share? Examining Social Capital and Knowledge Contribution in Electronic Networks of Practice , 2005, MIS Q..

[7]  Fang Wu,et al.  Crowdsourcing, attention and productivity , 2008, J. Inf. Sci..

[8]  Milan Vojnovic,et al.  Crowdsourcing and all-pay auctions , 2009, EC '09.

[9]  Paul Resnick,et al.  The value of reputation on eBay: A controlled experiment , 2002 .

[10]  Daren C. Brabham.  Moving the crowd at iStockphoto: The composition of the crowd and motivations for participation in a crowdsourcing application , 2008, First Monday.

[11]  Chrysanthos Dellarocas,et al.  Simultaneous Auctions of Imperfect Substitute Goods by Sellers of Different Reputations , 2008 .

[12]  Jürgen Bitzer,et al.  Intrinsic motivation in open source software development , 2007 .

[13]  Chrysanthos Dellarocas.  The Digitization of Word-of-mouth: Promise and Challenges of Online Reputation Mechanisms , 2022 .

[14]  Cheng Hsiao,et al.  Formulation and estimation of dynamic models using panel data , 1982 .

[15]  Chrysanthos Dellarocas,et al.  The Digitization of Word-of-Mouth: Promise and Challenges of Online Feedback Mechanisms , 2003, Manag. Sci..

[16]  Mark S. Ackerman,et al.  Crowdsourcing and knowledge sharing: strategic user behavior on taskcn , 2008, EC '08.

[17]  K. Lakhani,et al.  The Principles of Distributed Innovation , 2007, Innovations: Technology, Governance, Globalization.

[18]  Paul Resnick,et al.  Trust among strangers in internet transactions: Empirical analysis of eBay' s reputation system , 2002, The Economics of the Internet and E-commerce.

[19]  J. Morgan,et al.  Cheap Talk , 2005 .