Mechanism Design for Crowdsourcing Markets with Heterogeneous Tasks

Designing optimal pricing policies and mechanisms for allocating tasks to workers is central to online crowdsourcing markets. In this paper, we consider the following realistic setting of online crowdsourcing markets -- we are given a set of heterogeneous tasks requiring certain skills; each worker has certain expertise and interests which define the set of tasks she is interested in and willing to do. Given this bipartite graph between workers and tasks, we design our mechanism \truthuniform which does the allocation of tasks to workers, while ensuring budget feasibility, incentive-compatibility and achieves near-optimal utility. We further extend our results by exploiting a link with online Adwords allocation problem and present a randomized mechanism \truthfractional with improved approximation guarantees. Apart from strong theoretical guarantees, we carry out extensive experimentation using simulations as well as on a realistic case study of Wikipedia translation project with Mechanical Turk workers. Our results demonstrate the practical applicability of our mechanisms for realistic crowdsourcing markets on the web.

[1]  Aranyak Mehta,et al.  Online budgeted matching in random input models with applications to Adwords , 2008, SODA '08.

[2]  Robert D. Kleinberg,et al.  Learning on a budget: posted price mechanisms for online procurement , 2012, EC '12.

[3]  Aleksandrs Slivkins,et al.  Bandits with Knapsacks , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[4]  Nikhil R. Devanur,et al.  Randomized Primal-Dual analysis of RANKING for Online BiPartite Matching , 2013, SODA.

[5]  K. Taira Proof of Theorem 1.3 , 2004 .

[6]  Chien-Ju Ho,et al.  Online Task Assignment in Crowdsourcing Markets , 2012, AAAI.

[7]  Gagan Goel,et al.  Online Vertex-Weighted Bipartite Matching and Single-bid Budgeted Allocations , 2010, SODA.

[8]  Andreas Krause,et al.  Truthful incentives in crowdsourcing tasks using regret minimization mechanisms , 2013, WWW.

[9]  Gianluca Demartini,et al.  Pick-a-crowd: tell me what you like, and i'll tell you what to do , 2013, CIDR.

[10]  Yaron Singer,et al.  How to win friends and influence people, truthfully: influence maximization mechanisms for social networks , 2012, WSDM '12.

[11]  Andreas Krause,et al.  Incentives for Privacy Tradeoff in Community Sensing , 2013, HCOMP.

[12]  Archie C. Chapman,et al.  ε-first policies for budget-limited multi-armed bandits , 2010, AAAI 2010.

[13]  Lydia B. Chilton,et al.  The labor economics of paid crowdsourcing , 2010, EC '10.

[14]  Ning Chen,et al.  On the approximability of budget feasible mechanisms , 2010, SODA '11.

[15]  Thomas P. Hayes,et al.  The adwords problem: online keyword matching with budgeted bidders under random permutations , 2009, EC '09.

[16]  E. W. Beth,et al.  Proof by Contradiction , 1970 .

[17]  Ning Chen,et al.  Budget feasible mechanism design: from prior-free to bayesian , 2012, STOC '12.

[18]  Richard M. Karp,et al.  An optimal algorithm for on-line bipartite matching , 1990, STOC '90.

[19]  Nicholas R. Jennings,et al.  Efficient Crowdsourcing of Unknown Experts using Multi-Armed Bandits , 2012, ECAI.

[20]  Archie C. Chapman,et al.  Knapsack Based Optimal Policies for Budget-Limited Multi-Armed Bandits , 2012, AAAI.

[21]  Yaron Singer,et al.  Budget Feasible Mechanisms , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[22]  Yaron Singer,et al.  Pricing mechanisms for crowdsourcing markets , 2013, WWW.

[23]  Archie C. Chapman,et al.  Epsilon-First Policies for Budget-Limited Multi-Armed Bandits , 2010, AAAI.

[24]  J. Horton Algorithmic Wage Negotiations: Applications to Paid Crowdsourcing , 2010 .

[25]  Aranyak Mehta,et al.  AdWords and Generalized On-line Matching , 2005, FOCS.