Strategic effort allocation in online innovation tournaments

Abstract Online innovation tournaments, such as those hosted by crowdsourcing platforms (e.g., Kaggle), have been widely adopted by firms to evolve creative solutions to various problems. Solvers compete in these tournaments to earn rewards. In such competitive environments, it is imperative that solvers provide creative solutions with minimum effort. This article explores the factors that influence the solvers’ effort allocation decisions in a dynamic tournament setting. Specifically, comprehensive time variant data of teams that participated in crowdsourcing competitions on Kaggle were analyzed to gain insight into how solvers continually formulate strategies in light of performance feedback obtained through interim ranking. The results suggest that solvers strategically allocate their efforts throughout the contest to dynamically optimize their payoffs through balancing the probability of winning and the cost of expending effort. In particular, solvers tend to increase their efforts toward the end of tournaments or when they get closer to winning positions. Furthermore, our findings indicate that a last-minute surge in effort is more prevalent among high-skill solvers than in those with lower skill levels. In addition to providing insights that may help solvers develop strategies to improve their performance, the study has implications for the design of online crowdsourcing platforms, particularly in terms of incentivizing solvers to put forth their best effort.

[1]  Milan Vojnovic,et al.  Crowdsourcing and all-pay auctions , 2009, EC '09.

[2]  Stephen M. Garcia,et al.  The Psychology of Competition , 2013, Perspectives on psychological science : a journal of the Association for Psychological Science.

[3]  Avishalom Tor,et al.  Ranks and Rivals: A Theory of Competition , 2006, Personality & social psychology bulletin.

[4]  Jesse C. Bockstedt,et al.  Do Participation Strategy and Experience Impact the Likelihood of Winning in Unblind Innovation Contests , 2011 .

[5]  Rebeca P. Díaz Redondo,et al.  A social gamification framework for a K-6 learning platform , 2013, Comput. Hum. Behav..

[6]  S. G. Cohen,et al.  What Makes Teams Work: Group Effectiveness Research from the Shop Floor to the Executive Suite , 1997 .

[7]  Mario Pagliero,et al.  Interim Rank, Risk Taking, and Performance in Dynamic Tournaments , 2012, Journal of Political Economy.

[8]  Kai A. Konrad,et al.  The generalized Stackelberg equilibrium of the all-pay auction with complete information , 2005, SSRN Electronic Journal.

[9]  Yi Xu,et al.  Innovation Contests, Open Innovation, and Multiagent Problem Solving , 2008, Manag. Sci..

[10]  A. Roth,et al.  Last-Minute Bidding and the Rules for Ending Second-Price Auctions: Evidence from eBay and Amazon Auctions on the Internet , 2002 .

[11]  Karl T. Ulrich,et al.  Idea Generation and the Quality of the Best Idea , 2009, Manag. Sci..

[12]  Wei Guo EXPLORING AND MODELING OF BIDDING BEHAVIOR AND STRATEGIES OF ONLINE AUCTIONS , 2013 .

[13]  Marc Thomas Philipp Adam,et al.  The Impact of Computerized Agents on Immediate Emotions, Overall Arousal and Bidding Behavior in Electronic Auctions , 2015, J. Assoc. Inf. Syst..

[14]  Lada A. Adamic,et al.  Crowdsourcing with All-Pay Auctions: A Field Experiment on Taskcn , 2014, Manag. Sci..

[15]  B. Efron Bootstrap Methods: Another Look at the Jackknife , 1979 .

[16]  Huai Zhang,et al.  Tournaments of financial analysts , 2014 .

[17]  Tracy Xiao Liu,et al.  Simultaneous versus sequential all-pay auctions: an experimental study , 2016 .

[18]  Kevin Boudreau,et al.  Performance Responses To Competition Across Skill-Levels In Rank Order Tournaments: Field Evidence and Implications For Tournament Design , 2015 .

[19]  Barry L. Bayus,et al.  Crowdsourcing New Product Ideas over Time: An Analysis of the Dell IdeaStorm Community , 2013, Manag. Sci..

[20]  M. H. Quenouille NOTES ON BIAS IN ESTIMATION , 1956 .

[21]  Nagore Iriberri,et al.  The Importance of Relative Performance Feedback Information: Evidence from a Natural Experiment Using High School Students , 2009 .

[22]  Vincent S. Lai,et al.  Bidding strategies in online single-unit auctions: Their impact and satisfaction , 2013, Inf. Manag..

[23]  Catherine J. Morrison Paul,et al.  Capital-Skill Complementarity, Productivity and Wages: Evidence from Plant-Level Data for a Developing Country , 2008 .

[24]  Neil Savage,et al.  Gaining wisdom from crowds , 2012, Commun. ACM.

[25]  Karim R. Lakhani,et al.  Field Evidence on Individual Behavior & Performance in Rank-Order Tournaments , 2012 .

[26]  Roger Guimerà,et al.  Team Assembly Mechanisms Determine Collaboration Network Structure and Team Performance , 2005, Science.

[27]  Rajiv D. Banker,et al.  Impact of Past Performance and Strategic Bidding on Winner Determination of Open Innovation Contest , 2010 .

[28]  Karim R. Lakhani,et al.  Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis , 2011, Manag. Sci..

[29]  Radhika Santhanam,et al.  Digital Games and Beyond: What Happens When Players Compete , 2013, MIS Q..

[30]  Pablo Casas-Arce,et al.  Relative Performance Compensation, Contests, and Dynamic Incentives , 2009, Manag. Sci..

[31]  A. Zellner,et al.  Further Properties of Efficient Estimators for Seemingly Unrelated Regression Equations , 1962 .

[32]  Florian Ederer,et al.  Feedback and Motivation in Dynamic Tournaments , 2010 .

[33]  Martin Schreier,et al.  The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas? , 2009 .

[34]  Cheryl T. Druehl,et al.  Heterogeneous Submission Behavior and its Implications for Success in Innovation Contests with Public Submissions , 2016 .

[35]  L. Festinger A Theory of Social Comparison Processes , 1954 .

[36]  Andrew Whinston,et al.  Optimal Design of Consumer Contests , 2007 .

[37]  Young-Hoon Park,et al.  Bayesian Estimation of Bid Sequences in Internet Auctions Using a Generalized Record Breaking Model , 2007 .

[38]  A. Ronald Gallant,et al.  Statistical Inference in an Implicit, Nonlinear, Simultaneous Equation Model in the Context of Maximum Likelihood Estimation , 1980 .

[39]  R. Martínez-Espiñeira,et al.  ‘Adopt a Hypothetical Pup’: A Count Data Approach to the Valuation of Wildlife , 2007 .

[40]  S. E. Taylor,et al.  Social comparison activity under threat: downward evaluation and upward contacts. , 1989, Psychological review.

[41]  Param Vir Singh,et al.  Crowdsourcing "blockbuster" Ideas: a Dynamic Structural Model of Ideation , 2011, ICIS.

[42]  Anders U. Poulsen,et al.  Feedback and Incentives: Experimental Evidence , 2008 .

[43]  Karl T. Ulrich,et al.  Idea Generation and the Role of Feedback: Evidence from Field Experiments with Innovation Tournaments , 2017 .

[44]  Ranjani Krishnan,et al.  The Effects of Disseminating Relative Performance Feedback in Tournament and Individual Performance Compensation Plans , 2008 .

[45]  Jie Zhang,et al.  Task Division for Team Success in Crowdsourcing Contests: Resource Allocation and Alignment Effects , 2015, J. Manag. Inf. Syst..

[46]  Nikolay Archak,et al.  Money, glory and cheap talk: analyzing strategic behavior of contestants in simultaneous crowdsourcing contests on TopCoder.com , 2010, WWW '10.