Monetary Interventions in Crowdsourcing Task Switching

With a large amount of tasks of various types, requesters in crowdsourcing platforms often bundle tasks of different types into a single working session. This creates a task switching setting, where workers need to shift between different cognitive tasks. We design and conduct an experiment on Amazon Mechanical Turk to study how occasionally presented performancecontingent monetary rewards, referred as monetary interventions, affect worker performance in the task switching setting. We use two competing metrics to evaluate worker performance. When monetary interventions are placed on some tasks in a working session, our results show that worker performance on these tasks can be improved in both metrics. Moreover, worker performance on other tasks where monetary interventions are not placed is also affected: workers perform better according to one metric, but worse according to the other metric. This suggests that in addition to providing extrinsic monetary incentives for some tasks, monetary interventions implicitly set performance goals for all tasks. Furthermore, monetary interventions are most effective in improving worker performance when used at switch tasks, tasks that follow a task of a different type, in working sessions with a low task switching frequency.

[1]  Joseph S. Valacich,et al.  The Influence of Task Interruption on Individual Decision Making: An Information Overload Perspective , 1999 .

[2]  S. Monsell,et al.  Costs of a predictible switch between simple cognitive tasks. , 1995 .

[3]  A. Allport,et al.  Task switching and the measurement of “switch costs” , 2000, Psychological research.

[4]  Andrea M Philipp,et al.  Control and interference in task switching--a review. , 2010, Psychological bulletin.

[5]  R. Kliegl,et al.  Task-set switching and long-term memory retrieval. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[6]  G. D. Logan Task Switching , 2022 .

[7]  T. Shallice,et al.  Task Switching : A PDP Model , 2001 .

[8]  Yu-An Sun,et al.  The Effects of Performance-Contingent Financial Incentives in Online Labor Markets , 2013, AAAI.

[9]  Aniket Kittur,et al.  An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets , 2011, ICWSM.

[10]  Robert D. Pritchard,et al.  The influence of goal setting and financial incentives on task performance , 1973 .

[11]  Robert P. Steel,et al.  A meta-analytic study of the effects of goal setting on task performance: 1966-1984 , 1987 .

[12]  Niels Taatgen,et al.  The Multitasking Mind , 2010, Oxford series on cognitive models and architectures.

[13]  Niels Taatgen,et al.  Toward a unified theory of the multitasking continuum: from concurrent performance to task switching, interruption, and resumption , 2009, CHI.

[14]  Eric Horvitz,et al.  Disruption and recovery of computing tasks: field study, analysis, and directions , 2007, CHI.

[15]  A. Treisman,et al.  Attention, Space, and Action: Studies in Cognitive Neuroscience , 2001 .

[16]  Duncan J. Watts,et al.  Financial incentives and the "performance of crowds" , 2009, HCOMP '09.

[17]  J. Ridley Studies of Interference in Serial Verbal Reactions , 2001 .

[18]  Gloria Mark,et al.  The cost of interrupted work: more speed and stress , 2008, CHI.

[19]  Christopher G. Harris You're Hired! An Examination of Crowdsourcing Incentive Models in Human Resource Tasks , 2011 .

[20]  Stephen Monsell,et al.  Residual costs in task switching: Testing the failure-to-engage hypothesis , 2002, Psychonomic bulletin & review.

[21]  E. A. Locke,et al.  Goal setting and task performance: 1969–1980. , 1981 .

[22]  D. Alan Allport,et al.  SHIFTING INTENTIONAL SET - EXPLORING THE DYNAMIC CONTROL OF TASKS , 1994 .

[23]  E. A. Locke,et al.  Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. , 2002, The American psychologist.

[24]  Brian P. Bailey,et al.  On the need for attention-aware systems: Measuring effects of interruption on task performance, error rate, and affective state , 2006, Comput. Hum. Behav..

[25]  G. Wylie,et al.  Task-switching: Positive and negative priming of task-set. , 1999 .