Not Just in it for the Money: A Qualitative Investigation of Workers' Perceived Benefits of Micro-task Crowdsourcing

Micro-task crowd sourcing fosters a labor relation in which large volumes of small, simple tasks are completed at low cost by self-selected online workers. The growth of micro-task crowd sourcing, characterized by apparently low remuneration, begs the question how individual participants perceive the benefits of such micro work. In response we conducted a survey on Amazon's Mechanical Turk, a premier micro-task crowd sourcing platform. The sample included workers in the US and India. Through open-ended questions we inquired about perceived benefits of participants' work. A thematic analysis of responses revealed many benefits: monetary compensation, self-improvement, time management, emotional rewards, and benefits related to the characteristics of micro-tasking. Workers compartmentalized money earned from micro work into different non-fungible mental accounts for different purposes. American and Indian workers differed in non-monetary benefit perceptions. Indian workers valued self-improvement benefits, whereas American workers valued emotional benefits. Our results suggest that workers' recognize a diverse portfolio of benefits through micro work.

[1]  Aniket Kittur,et al.  CrowdForge: crowdsourcing complex work , 2011, UIST.

[2]  Wendell Bell,et al.  The Third Wave. , 1982 .

[3]  K. Kurihara.  Utility Analysis and the Consumption Function: An Interpretation of Cross-Section Data , 2013 .

[4]  Erik Brynjolfsson,et al.  Race against the machine : how the digital revolution is accelerating innovation, driving productivity, and irreversibly transforming employment and the economy , 2011 .

[5]  Adrian Furnham,et al.  The Psychology of Money , 1998 .

[6]  M. Six Silberman,et al.  Turkopticon: interrupting worker invisibility in amazon mechanical turk , 2013, CHI.

[7]  Daren C. Brabham.  Crowdsourcing: A Model for Leveraging Online Communities , 2011 .

[8]  Martin Schader,et al.  Exploring task properties in crowdsourcing - an empirical study on mechanical turk , 2011, ECIS.

[9]  M. Hossain,et al.  Crowdsourcing: Activities, incentives and users' motivations to participate , 2012, 2012 International Conference on Innovation Management and Technology Research.

[10]  Michael Rosemann,et al.  Crowdsourcing Information Systems - A Systems Theory Perspective , 2011 .

[11]  Daniel J. Hruschka,et al.  Reliability in Coding Open-Ended Data: Lessons Learned from HIV Behavioral Research , 2004 .

[12]  Daren C. Brabham.  MOVING THE CROWD AT THREADLESS , 2010 .

[13]  Ellen R. McGrattan,et al.  Money as a medium of exchange in an economy with artificially intelligent agents , 1990 .

[14]  Kwong-Sak Leung,et al.  A Survey of Crowdsourcing Systems , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[15]  廷冕 李,et al.  応用 (Application) について , 1981 .

[16]  T. L. Tang.  The development of a short Money Ethic Scale: Attitudes toward money and pay satisfaction revisited , 1995 .

[17]  Amy E. Mickel,et al.  The meaning of money: An individual-difference perspective , 1999 .

[18]  R. Thaler.  Saving, Fungibility, and Mental Accounts , 1990 .

[19]  Arun Sundararajan,et al.  Optimal Design of Crowdsourcing Contests , 2009, ICIS.

[20]  R. Thaler.  Anomalies: Saving, Fungibility, and Mental Accounts , 2016 .

[21]  Richard E. Boyatzis,et al.  Transforming Qualitative Information: Thematic Analysis and Code Development , 1998 .

[22]  M. Patton.  Qualitative research and evaluation methods , 1980 .

[23]  P. Whitla,et al.  Crowdsourcing and its application in marketing activities , 2009 .

[24]  Eli Blevis,et al.  A survey of crowdsourcing as a means of collaboration and the implications of crowdsourcing for interaction design , 2011, 2011 International Conference on Collaboration Technologies and Systems (CTS).

[25]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[26]  Daniel J. Veit,et al.  More than fun and money. Worker Motivation in Crowdsourcing - A Study on Mechanical Turk , 2011, AMCIS.

[27]  Viviana A. Zelizer,et al.  The Social Meaning of Money: "Special Monies" , 1989, American Journal of Sociology.

[28]  S. Lippman,et al.  THE ECONOMICS OF JOB SEARCH: A SURVEY* , 1976 .

[29]  Anne C. Rouse,et al.  A Preliminary Taxonomy of Crowdsourcing , 2010 .

[30]  J. Richard Landis,et al.  Large sample variance of kappa in the case of different sets of raters. , 1979 .

[31]  Pietro Perona,et al.  Online crowdsourcing: Rating annotators and obtaining cost-effective labels , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[32]  R. Thaler,et al.  THE BEHAVIORAL LIFE‐CYCLE HYPOTHESIS , 1988 .

[33]  Qinghua Zhu,et al.  Exploring the Motivation of Participants in Crowdsourcing Contest , 2012, ICIS.

[34]  Jonathan Corney,et al.  Outsourcing labour to the cloud , 2009 .

[35]  Martin Schader,et al.  Managing the Crowd: Towards a Taxonomy of Crowdsourcing Processes , 2011, AMCIS.

[36]  Stephen J. Hoch,et al.  Spending Time versus Spending Money , 2004 .

[37]  David Clingingsmith,et al.  Mental Accounts, Selective Attention, and the Mutability of Altruism: An Experiment with Online Workers , 2013 .

[38]  R. Thaler.  Toward a positive theory of consumer choice , 1980 .

[39]  Jeffrey Heer,et al.  Crowdsourcing graphical perception: using mechanical turk to assess visualization design , 2010, CHI.

[40]  D. Cicchetti.  Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology. , 1994 .

[41]  Pamela W. Henderson,et al.  Mental accounting and categorization , 1992 .

[42]  Panagiotis G. Ipeirotis.  Demographics of Mechanical Turk , 2010 .

[43]  Benjamin B. Bederson,et al.  Human computation: a survey and taxonomy of a growing field , 2011, CHI.

[44]  M. Weber.  Economy and society : an outline of interpretive sociology , 2008 .

[45]  Richard H. Thaler,et al.  Mental Accounting and Consumer Choice , 1985, Mark. Sci..

[46]  Phuoc Tran-Gia,et al.  Modeling of crowdsourcing platforms and granularity of work organization in Future Internet , 2011, 2011 23rd International Teletraffic Congress (ITC).

[47]  Vladimir Zwass,et al.  Co-Creation: Toward a Taxonomy and an Integrated Research Perspective , 2010, Int. J. Electron. Commer..

[48]  Panagiotis G. Ipeirotis.  Analyzing the Amazon Mechanical Turk marketplace , 2010, XRDS.

[49]  Alon Y. Halevy,et al.  Crowdsourcing systems on the World-Wide Web , 2011, Commun. ACM.

[50]  Phuoc Tran-Gia,et al.  Anatomy of a Crowdsourcing Platform - Using the Example of Microworkers.com , 2011, 2011 Fifth International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing.

[51]  Martin Schader,et al.  Crowdsourcing Information Systems - Definition, Typology, and Design , 2012, ICIS.

[52]  J. Duffy,et al.  Emergence of money as a medium of exchange: An experimental study , 1999 .

[53]  V. Zelizer.  The Social Meaning of Money , 2021 .

[54]  Bill Tomlinson,et al.  Who are the crowdworkers?: shifting demographics in mechanical turk , 2010, CHI Extended Abstracts.

[55]  David Graeber,et al.  The Democracy Project: A History, a Crisis, a Movement , 2013 .

[56]  Chris Callison-Burch,et al.  Crowdsourcing Translation: Professional Quality from Non-Professionals , 2011, ACL.

[57]  Elaine Welsh,et al.  Dealing with Data: Using NVivo in the Qualitative Data Analysis Process , 2002 .

[58]  Daren C. Brabham.  Crowdsourcing as a Model for Problem Solving , 2008 .

[59]  Joop Hartog,et al.  Reservation Wages and Starting Wages , 2011, SSRN Electronic Journal.

[60]  T. L. Tang.  The meaning of money revisited , 1992 .

[61]  Yi Xu,et al.  Innovation Contests, Open Innovation, and Multiagent Problem Solving , 2008, Manag. Sci..

[62]  P. F. Wernimont,et al.  The meaning of money. , 1972 .

[63]  André Gorz,et al.  Paths to paradise : on the liberation from work , 1985 .

[64]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[65]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[66]  Bernd H. Schmitt,et al.  Waiting time and decision making: is time like money? , 1995 .

[67]  Dahui Li,et al.  Task Design, Motivation, and Participation in Crowdsourcing Contests , 2011, Int. J. Electron. Commer..

[68]  Daren C. Brabham.  Moving the crowd at iStockphoto: The composition of the crowd and motivations for participation in a crowdsourcing application , 2008, First Monday.

[69]  Ricardo Baeza-Yates,et al.  Design and Implementation of Relevance Assessments Using Crowdsourcing , 2011, ECIR.