PRINGL - A domain-specific language for incentive management in crowdsourcing

Novel types of crowdsourcing systems require a wider spectrum of incentives for efficient motivation and management of human workers taking part in complex collaborations. Incentive management techniques used in conventional crowdsourcing platforms are not suitable for more intellectually-challenging tasks. Currently, incentives are custom-developed and managed by each particular platform. This prevents incentive portability and cross-platform comparison. In this paper we present PRINGL - a domain-specific language for programming and managing complex incentive strategies for socio-technical platforms in general. It promotes re-use of proven incentive logic and simplifies modeling, adjustment and enactment of complex incentives for socio-technical systems. We demonstrate its applicability and expressiveness on a set of realistic use-cases and discuss its properties.

[1]  R. Preston McAfee,et al.  The wisdom of smaller, smarter crowds , 2014, EC.

[2]  Øystein Haugen,et al.  Evaluating Domain-Specific Modelling Solutions , 2010, ER Workshops.

[3]  Schahram Dustdar,et al.  Provisioning Quality-Aware Social Compute Units in the Cloud , 2013, ICSOC.

[4]  Luciano Baresi,et al.  Tutorial Introduction to Graph Transformation: A Software Engineering Perspective , 2002, ICGT.

[5]  Luciano Baresi,et al.  Tutorial Introduction to Graph Transformation: A Software Engineering Perspective , 2002, ICGT.

[6]  Abraham Bernstein,et al.  CrowdLang: A Programming Language for the Systematic Exploration of Human Computation Systems , 2012, SocInfo.

[7]  Lydia B. Chilton,et al.  Exploring iterative and parallel human computation processes , 2010, HCOMP '10.

[8]  Schahram Dustdar,et al.  Virtualizing Communication for Hybrid and Diversity-Aware Collective Adaptive Systems , 2014, ICSOC Workshops.

[9]  G. Milkovich,et al.  Relationships Among Risk, Incentive Pay, and Organizational Performance , 1998 .

[10]  Wai-Tat Fu,et al.  What Will Others Choose? How a Majority Vote Reward Scheme Can Improve Human Computation in a Spatial Location Identification Task , 2013, HCOMP.

[11]  David C. Parkes,et al.  Dwelling on the Negative: Incentivizing Effort in Peer Prediction , 2013, HCOMP.

[12]  Schahram Dustdar,et al.  Incentives and rewarding in social computing , 2013, CACM.

[13]  Alon Y. Halevy,et al.  Crowdsourcing systems on the World-Wide Web , 2011, Commun. ACM.

[14]  G. Milkovich,et al.  The Relationship Between Risk, Incentive Pay, and Organizational Performance , 1997 .

[15]  Eytan Adar.  Why I Hate Mechanical Turk Research (and Workshops) , 2011 .

[16]  Marco Zamarian,et al.  Analyzing Crowd Labor and Designing Incentives for Humans in the Loop , 2012, IEEE Internet Computing.

[17]  Sarvapali D. Ramchurn,et al.  Collabmap: crowdsourcing maps for emergency planning , 2013, WebSci.

[18]  Tobias Hoßfeld,et al.  Analyzing costs and accuracy of validation mechanisms for crowdsourcing platforms , 2013, Math. Comput. Model..

[19]  Schahram Dustdar,et al.  Managing Incentives in Social Computing Systems with PRINGL , 2014, WISE.

[20]  Edward J. Lusk,et al.  Country-Compatible Incentive Design , 2009 .

[21]  Wai-Tat Fu,et al.  Don't hide in the crowd!: increasing social transparency between peer workers improves crowdsourcing outcomes , 2013, CHI.

[22]  Ya'akov Gal,et al.  A study of computational and human strategies in revelation games , 2014, Autonomous Agents and Multi-Agent Systems.

[23]  Andrew McGregor,et al.  AutoMan: a platform for integrating human-based and digital computation , 2012, OOPSLA '12.

[24]  Dennis D. Fehrenbacher.  Design of Incentive Systems , 2013 .

[25]  Schahram Dustdar,et al.  Simulation-Based Modeling and Evaluation of Incentive Schemes in Crowdsourcing Environments , 2013, OTM Conferences.

[26]  Michael S. Bernstein,et al.  The future of crowd work , 2013, CSCW.

[27]  Schahram Dustdar,et al.  On the Elasticity of Social Compute Units , 2014, CAiSE.

[28]  Schahram Dustdar,et al.  Programming Incentives in Information Systems , 2013, CAiSE.

[29]  J. Laffont,et al.  The Theory of Incentives: The Principal-Agent Model , 2001 .

[30]  Daniele Miorandi,et al.  "Programming" Social Collective Intelligence , 2014, IEEE Technology and Society Magazine.

[31]  Rex B. Kline,et al.  Usability measurement and metrics: A consolidated model , 2006, Software Quality Journal.

[32]  Mahmood Hosseini,et al.  The four pillars of crowdsourcing: A reference model , 2014, 2014 IEEE Eighth International Conference on Research Challenges in Information Science (RCIS).

[33]  Duncan J. Watts,et al.  Financial incentives and the "performance of crowds" , 2009, HCOMP '09.

[34]  Alexis Battle,et al.  The jabberwocky programming environment for structured social computing , 2011, UIST.

[35]  Sebastian Buchwald,et al.  GrGen.NET , 2010, International Journal on Software Tools for Technology Transfer.

[36]  Bruno Ribeiro,et al.  The socio-monetary incentives of online social network malware campaigns , 2014, Conference on Online Social Networks.