The Effects of Pay-to-Quit Incentives on Crowdworker Task Quality

Companies such as Zappos.com and Amazon.com provide financial incentives for newer employees to quit. The premise is that workers who will accept this offer are misaligned with their company culture, which will therefore negatively affect quality over time. Could this pay-to-quit incentive scheme align workers in online labor markets? We conduct five empirical experiments evaluating different pay-to-quit incentives with crowdworkers and evaluate their effects on mean task accuracy, retention rate, and improvement in mean task accuracy. We find that the number of times a user is prompted for the inducement, the type and frequency of performance feedback given to participants, the type of incentive, as well as the amount offered can help retain high-performing workers but encourage poor-performing workers to quit early. When we combine the best features from our experiments and examine their aggregate effectiveness, mean task accuracy is improved by 28.3%. Last, we also find that certain demographics contribute to the effectiveness of pay-to-quit incentives.

[1]  H. P. Binswanger Attitudes toward risk: Experimental measurement in rural india , 1980 .

[2]  Michael A. Kuhn,et al.  Journal of Economic Behavior & Organization Experimental Methods: Extra-laboratory Experiments-extending the Reach of Experimental Economics , 2022 .

[3]  Manuel Blum,et al.  Verbosity: a game for collecting common-sense facts , 2006, CHI.

[4]  E. A. Locke Toward a theory of task motivation and incentives , 1968 .

[5]  Charles A. Holt,et al.  Risk Aversion and Incentive Effects , 2002 .

[6]  David Gal,et al.  A psychological law of inertia and the illusion of loss aversion , 2005, Judgment and Decision Making.

[7]  Christopher G. Harris You're Hired! An Examination of Crowdsourcing Incentive Models in Human Resource Tasks , 2011 .

[8]  Duncan J. Watts,et al.  Financial incentives and the "performance of crowds" , 2009, HCOMP '09.

[9]  M. Buelens,et al.  Predicting the outcome of a two-party price negotiation: Contribution of reservation price, aspiration price and opening offer , 2002 .

[10]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[11]  Renate Schubert,et al.  Financial Decision-Making: Are Women Really More Risk-Averse? , 1999 .

[12]  David M. Pennock,et al.  1 Billion Pages = 1 Million Dollars? Mining the Web to Play "Who Wants to be a Millionaire?" , 2002, UAI.

[13]  Canice Prendergast The Provision of Incentives in Firms , 1999 .

[14]  Morten I. Lau,et al.  Risk Aversion in Game Shows , 2008 .

[15]  A. P. deVries,et al.  How Crowdsourcable is Your Task , 2011 .

[16]  Roger Hartley,et al.  Who really wants to be a millionaire? Estimates of risk aversion from gameshow data , 2006 .

[17]  Aaron D. Shaw,et al.  Designing incentives for inexpert human raters , 2011, CSCW.

[18]  Mark D. Griffiths,et al.  The role of cognitive bias and skill in fruit machine gambling. , 1994 .

[19]  N. Jianakoplos,et al.  ARE WOMEN MORE RISK AVERSE , 1998 .

[20]  A. Rustichini,et al.  Pay Enough or Don't Pay at All , 2000 .

[21]  Catherine C. Eckel,et al.  Men, Women and Risk Aversion: Experimental Evidence , 2008 .

[22]  Padmini Srinivasan,et al.  Quality through flow and immersion: gamifying crowdsourced relevance assessments , 2012, SIGIR '12.

[23]  Michael B. Walker D. Phil.,et al.  Irrational thinking among slot machine players , 2005, Journal of Gambling Studies.

[24]  E. Miller Risk, Uncertainty, and Divergence of Opinion , 1977 .

[25]  David Gill,et al.  Nuffield Centre for Experimental Social Sciences Discussion Paper Series a Structural Analysis of Disappointment Aversion in a Real Effort Competition a Structural Analysis of Disappointment Aversion in a Real Effort Competition , 2022 .

[26]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[27]  Richard E. Kopelman,et al.  Interpreting the success of Zappos.com, Four Seasons, and Nordstrom: Customer centricity is but one-third of the job , 2012 .

[28]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[29]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[30]  J. March,et al.  Managerial perspectives on risk and risk taking , 1987 .