When Does More Money Work? Examining the Role of Perceived Fairness in Pay on the Performance Quality of Crowdworkers

This research adds to the rich discussion on whether increases in payment to crowdworkers lead to increases in performance quality by introducing the concept of perceived fairness in pay (PFP). PFP refers to the belief that one is fairly compensated for their work. We examine whether PFP mediates the impact of payment amount on the performance quality of crowdworkers. We conducted a field experiment with 152 crowdworkers performing a buttonclicking (BC) task and an instructional manipulation check (IMC) task. PFP mediated the impact of payment amount on performance quality in the BC task but not in the IMC task. PFP also mediated the impact of payment amount on satisfaction and task time. Results suggest that PFP can help us better understand the relationship between payment and performance quality in crowdsourcing.

[1]  Derek Ruths,et al.  How One Microtask Affects Another , 2016, CHI.

[2]  Kasper Hornbæk,et al.  Measuring usability: are effectiveness, efficiency, and satisfaction really correlated? , 2000, CHI.

[3]  Lionel P. Robert,et al.  Are you satisfied yet? Shared leadership, trust and individual satisfaction in virtual teams , 2013 .

[4]  Lydia B. Chilton,et al.  The labor economics of paid crowdsourcing , 2010, EC '10.

[5]  M. Sobel.  Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models , 1982 .

[6]  Duncan J. Watts,et al.  Financial incentives and the "performance of crowds" , 2009, HCOMP '09.

[7]  J. H. Davis,et al.  An Integrative Model Of Organizational Trust , 1995 .

[8]  Barton L. Weathington,et al.  The Relationships Between Justice Perceptions, Trust, and Employee Attitudes in a Downsized Organization , 2006, The Journal of psychology.

[9]  R. Folger,et al.  RETALIATION IN THE WORKPLACE: THE ROLES OF DISTRIBUTIVE, PROCEDURAL, AND INTERACTIONAL JUSTICE , 1997 .

[10]  Daniel M. Oppenheimer,et al.  Instructional Manipulation Checks: Detecting Satisficing to Increase Statistical Power , 2009 .

[11]  Yu-An Sun,et al.  The Effects of Performance-Contingent Financial Incentives in Online Labor Markets , 2013, AAAI.

[12]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[13]  J. S. Adams,et al.  Inequity In Social Exchange , 1965 .

[14]  Jerald Greenberg,et al.  Handbook of Organizational Justice , 2005 .

[15]  Aldo Gangemi,et al.  Proceedings of the 24th International Conference on World Wide Web , 2015, WWW.

[16]  Michael S. Bernstein,et al.  The future of crowd work , 2013, CSCW.

[17]  Brent A. Scott,et al.  Trust, trustworthiness, and trust propensity: a meta-analytic test of their unique relationships with risk taking and job performance. , 2007, The Journal of applied psychology.

[18]  Donald E. Conlon,et al.  Justice at the millennium, a decade later: a meta-analytic test of social exchange and affect-based perspectives. , 2013, The Journal of applied psychology.

[19]  Adrian Thornhill,et al.  Organisational justice, trust and the management of change: An exploration , 2003 .

[20]  Wei Liu,et al.  Is it pay levels or pay raises that matter to fairness and turnover , 2005 .

[21]  S. Gilliland.  The Perceived Fairness of Selection Systems: An Organizational Justice Perspective , 1993 .

[22]  J. Colquitt.  On the dimensionality of organizational justice: a construct validation of a measure. , 2001, The Journal of applied psychology.

[23]  Uwe Jirjahn.  Performance Pay and Productivity: A Note on the Moderating Role of a High‐wage Policy , 2016 .

[24]  Aaron D. Shaw,et al.  Designing incentives for inexpert human raters , 2011, CSCW.

[25]  P. Shrout,et al.  Mediation in experimental and nonexperimental studies: new procedures and recommendations. , 2002, Psychological methods.

[26]  D. Kahneman.  Maps of Bounded Rationality: Psychology for Behavioral Economics , 2003 .

[27]  Sirkka L. Jarvenpaa,et al.  Is Anybody Out There? Antecedents of Trust in Global Virtual Teams , 1998, J. Manag. Inf. Syst..

[28]  A. Hayes.  Beyond Baron and Kenny: Statistical Mediation Analysis in the New Millennium , 2009 .

[29]  Alan R. Dennis,et al.  Structuring Time and Task in Electronic Brainstorming , 1999, MIS Q..

[30]  R. Folger,et al.  Effects of Procedural and Distributive Justice on Reactions to Pay Raise Decisions , 1989 .

[31]  R. Folger.  Distributive and procedural justice: Combined impact of voice and improvement on experienced inequity. , 1977 .

[32]  Eric Horvitz,et al.  Volunteering Versus Work for Pay: Incentives and Tradeoffs in Crowdsourcing , 2013, HCOMP.

[33]  Margaret L. Williams,et al.  Organizational Justice and Organizational Citizenship Behavior: A Mediated Multifoci Model † , 2009 .

[34]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[35]  Likoebe M. Maruping,et al.  Idea Generation in Technology-Supported Teams: A Multilevel Motivational Perspective , 2012, 2012 45th Hawaii International Conference on System Sciences.

[36]  Ming Yin,et al.  Predicting Crowd Work Quality under Monetary Interventions , 2016, HCOMP.

[37]  Likoebe M. Maruping,et al.  Mechanisms Underlying Social Loafing in Technology Teams: An Empirical Analysis , 2010, ICIS.

[38]  P. Sweeney,et al.  Research Notes. Distributive and Procedural Justice as Predictors of Satisfaction with Personal and Organizational Outcomes , 1992 .

[39]  Laura A. Dabbish,et al.  The Perverse Effects of Social Transparency on Online Advice Taking , 2015, CSCW.

[40]  L. Witt,et al.  Gender and the relationship between perceived fairness of pay or promotion and job satisfaction. , 1992, The Journal of applied psychology.

[41]  Onne Janssen.  Fairness perceptions as a moderator in the curvilinear relationships between job demands, and job performance and job satisfaction , 2001 .

[42]  R. Thaler,et al.  Labor Supply of New York City Cabdrivers: One Day at a Time , 1997 .

[43]  Aleksandrs Slivkins,et al.  Incentivizing high quality crowdwork , 2015, SECO.

[44]  Jacki O'Neill,et al.  Being a turker , 2014, CSCW.

[45]  Aniket Kittur,et al.  An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets , 2011, ICWSM.

[46]  David G. Rand,et al.  The online laboratory: conducting experiments in a real labor market , 2010, ArXiv.

[47]  Alan R. Dennis,et al.  Individual Swift Trust and Knowledge-Based Trust in Face-to-Face and Virtual Team Members , 2009, J. Manag. Inf. Syst..

[48]  Roger C. Mayer,et al.  Empowerment in veterinary clinics: the role of trust in delegation† , 2016 .

[49]  M. Six Silberman,et al.  Turkopticon: interrupting worker invisibility in amazon mechanical turk , 2013, CHI.

[50]  Amar Cheema,et al.  Data collection in a flat world: the strengths and weaknesses of mechanical turk samples , 2013 .