Performance Responses To Competition Across Skill-Levels In Rank Order Tournaments: Field Evidence and Implications For Tournament Design

Tournaments are widely used in the economy to organize production and innovation. We study individual data on 2775 contestants in 755 software algorithm development contests with random assignment. The performance response to added contestants varies nonmonotonically across contestants of different abilities, precisely conforming to theoretical predictions. Most participants respond negatively, whereas the highest-skilled contestants respond positively. In counterfactual simulations, we interpret a number of tournament design policies (number of competitors, prize allocation and structure, number of divisions, open entry) and assess their effectiveness in shaping optimal tournament outcomes for a designer.

[1]  Consumer and Producer Behavior in the Market for Penny Auctions : A Theoretical and Empirical Analysis , 2009 .

[2]  A. Falk,et al.  Individual Risk Attitudes: Measurement, Determinants and Behavioral Consequences , 2009 .

[3]  A. Ullah,et al.  Nonparametric Econometrics , 1999 .

[4]  Roman M. Sheremeta,et al.  A survey of experimental research on contests, all-pay auctions and tournaments , 2012, Experimental Economics.

[5]  Karim R. Lakhani,et al.  Incentives and Problem Uncertainty in Innovation Contests: An Empirical Analysis , 2011, Manag. Sci..

[6]  J. Orszag,et al.  A new look at incentive effects and golf tournaments , 1994 .

[7]  Andrew Schotter,et al.  Asymmetric Tournaments, Equal Opportunity Laws, and Affirmative Action: Some Experimental Results , 1992 .

[8]  Leibniz-Informationszentrum Wirtschaft,et al.  A survey of experimental research on contests, all- pay auctions and tournaments , 2012 .

[9]  Tor Eriksson,et al.  Executive Compensation and Tournament Theory: Empirical Tests on Danish Data , 1999, Journal of Labor Economics.

[10]  Jon Reiersen,et al.  Status and Incentives , 2004 .

[11]  John Cairns,et al.  The Economic Design of Sporting Contests , 2003 .

[12]  M. D. Pratt,et al.  An experimental investigation of efficient rent-seeking , 1989 .

[13]  Edward P. Lazear,et al.  Agency, Earnings Profiles, Productivity, and Hours Restrictions , 1981 .

[14]  Jingfeng Lu,et al.  The optimal multi-stage contest , 2012 .

[15]  Robert Dur,et al.  Tournament Incentives in the Field: Gender Differences in the Workplace , 2009, Journal of Labor Economics.

[16]  Jean-Jacques Laffont,et al.  Structural Analysis of Auction Data , 1996 .

[17]  G. Tullock.  Efficient Rent Seeking , 2001 .

[18]  J. Vickers,et al.  Racing with Uncertainty , 1987 .

[19]  Charles A. Wilson,et al.  AUCTIONS FOR OIL AND GAS LEASES WITH AN INFORMED BIDDER AND A RANDOM RESERVATION PRICE , 1994 .

[20]  R LakhaniKarim,et al.  Incentives and Problem Uncertainty in Innovation Contests , 2011 .

[21]  Qi Li,et al.  Nonparametric Econometrics: Theory and Practice , 2006 .

[22]  Benny Moldovanu,et al.  Contest architecture , 2006, J. Econ. Theory.

[23]  Jordi Blanes i Vidal,et al.  Tournaments Without Prizes: Evidence from Personnel Records , 2009, Manag. Sci..

[24]  Karl T. Ulrich,et al.  Idea Generation and the Role of Feedback: Evidence from Field Experiments with Innovation Tournaments , 2017 .

[25]  Armin Falk,et al.  Performance Pay and Multi-Dimensional Sorting: Productivity, Preferences and Gender , 2006, SSRN Electronic Journal.

[26]  B. Moldovanu,et al.  Contests for Status , 2005, Journal of Political Economy.

[27]  Kai A. Konrad,et al.  Strategy and Dynamics in Contests , 2009 .

[28]  J. Lerner,et al.  Inducement Prizes and Innovation , 2008 .

[29]  B. Moldovanu,et al.  The Optimal Allocation of Prizes in Contests , 2001 .

[30]  Q. Lib,et al.  Nonparametric estimation of regression functions with both categorical and continuous data , 2004 .

[31]  T. Nicholas.  Cheaper patents , 2010 .

[32]  Walter N. Thurman,et al.  Testing the Theory of Tournaments: An Empirical Analysis of Broiler Production , 1994, Journal of Labor Economics.

[33]  R. McAfee,et al.  Auctionin Entry into Tournaments , 1999, Journal of Political Economy.

[34]  J. Laffont,et al.  Structural econometric analysis of descending auctions , 1993 .

[35]  M. Stephens,et al.  K-Sample Anderson–Darling Tests , 1987 .

[36]  Steven Y. Wu,et al.  Mixed Tournaments, Common Shocks, and Disincentives: An Experimental Study , 2006 .

[37]  Yi Xu,et al.  Innovation Contests, Open Innovation, and Multiagent Problem Solving , 2008, Manag. Sci..

[38]  Satendra Kumar,et al.  Awards , 2002, Brain Research.

[39]  Martin Schreier,et al.  The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas? , 2009 .

[40]  Avishalom Tor,et al.  The N-Effect , 2009, Psychological science.

[41]  E. Lazear,et al.  Rank-Order Tournaments as Optimum Labor Contracts , 1979, Journal of Political Economy.

[42]  J. Carpenter,et al.  Tournaments and Office Politics: Evidence from a Real Effort Experiment , 2007, SSRN Electronic Journal.

[43]  Kimberly Mroweic Toter.  New Product Ideas , 1987 .

[44]  Heidi L. Williams,et al.  Incentivizing Innovation: Adding to the Tool Kit , 2010, Innovation Policy and the Economy.

[45]  Curtis R. Taylor.  Digging for golden carrots: an analysis of research tournaments , 1995 .

[46]  Donald W. K. Andrews,et al.  Asymptotics for Semiparametric Econometric Models via Stochastic Equicontinuity , 1994 .

[47]  T. Besley,et al.  Status Incentives , 2008 .

[48]  Michael L. Bognanno,et al.  The Incentive Effects of Tournaments Revisited: Evidence from the European PGA Tour , 1990 .

[49]  Lada A. Adamic,et al.  Crowdsourcing with All-Pay Auctions: A Field Experiment on Taskcn , 2014, Manag. Sci..

[50]  Quang Vuong,et al.  First-Price Sealed-Bid Auctions with Secret Reservation Prices , 1994 .

[51]  Cathy L. Schott,et al.  Experimental Results , 2009 .

[52]  Steven Y. Wu,et al.  Tournaments, Fairness, and Risk , 2006 .

[53]  Brian D. Wright.  The Economics of Invention Incentives: Patents, Prizes, and Research Contracts , 1983 .

[54]  R. Porter.  The Role of Information in U.S. Offshore Oil and Gas Lease Auctions , 1992 .

[55]  J. Laffont,et al.  ECONOMETRICS OF FIRST-PRICE AUCTIONS , 1995 .

[56]  Robert J. Reilly,et al.  Do too many cooks always spoil the stew? An experimental analysis of rent-seeking and the role of a strategic buyer , 1998 .

[57]  Karim R Lakhani,et al.  Using the crowd as an innovation partner. , 2013, Harvard business review.

[58]  Karim R. Lakhani,et al.  Marginality and Problem-Solving Effectiveness in Broadcast Search , 2010, Organ. Sci..

[59]  R. Porter,et al.  An Empirical Study of an Auction with Asymmetric Information , 1988 .

[60]  Michael McKee,et al.  Using Auctions To Reward Tournament Winners: Theory and Experimental Investigations , 2002 .

[61]  Ian L. Gale,et al.  Optimal Design of Research Contests , 2003 .

[62]  Ali Hortaçsu,et al.  Winner's Curse, Reserve Prices and Endogenous Entry: Empirical Insights from Ebay Auctions , 2003 .

[63]  Nagore Iriberri,et al.  The Importance of Relative Performance Feedback Information: Evidence from a Natural Experiment Using High School Students , 2009 .

[64]  Ryuichi Tanaka,et al.  Testing the incentive effects in tournaments with a superstar , 2012 .

[65]  Jennifer Brown,et al.  Quitters Never Win: The (Adverse) Incentive Effects of Competing with Superstars , 2011, Journal of Political Economy.

[66]  Michael Kosfeld,et al.  Getting More Work for Nothing? Symbolic Awards and Worker Performance , 2011, SSRN Electronic Journal.

[67]  Paul R. Milgrom,et al.  Multitask Principal–Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design , 1991 .

[68]  Pablo Casas-Arce,et al.  Relative Performance Compensation, Contests, and Dynamic Incentives , 2009, Manag. Sci..

[69]  Luciano Kay.  The Effect of Inducement Prizes on Innovation: Evidence from the Ansari X Prize and the Northrop Grumman Lunar Lander Challenge , 2011 .

[70]  E. Wagenmakers,et al.  A psychometric analysis of chess expertise. , 2005, The American journal of psychology.

[71]  Jennifer F. Reinganum.  The timing of innovation: Research, development, and diffusion , 1989 .

[72]  Eric Lonstein,et al.  Prize-based contests can provide solutions to computational biology problems , 2013, Nature Biotechnology.

[73]  Andrew Schotter,et al.  Tournaments and Piece Rates: An Experimental Study , 1987, Journal of Political Economy.

[74]  Fiona E. Murray,et al.  Grand Innovation Prizes: A theoretical, normative, and empirical evaluation , 2012 .

[75]  Michael L. Bognanno,et al.  Do Tournaments Have Incentive Effects? , 1988, Journal of Political Economy.

[76]  J. List,et al.  On the Role of Group Size in Tournaments: Theory and Evidence from Lab and Field Experiments , 2014 .