Context-Aware Reliable Crowdsourcing in Social Networks

There are two problems in the traditional crowdsourcing systems for handling complex tasks. First, decomposing complex tasks into a set of micro-subtasks requires the decomposition capability of the requesters; thus, some requesters may abandon using crowdsourcing to accomplish a large number of complex tasks since they cannot bear such heavy burden by themselves. Second, tasks are often assigned redundantly to multiple workers to achieve reliable results, but reliability may not be ensured when there are many malicious workers in the crowd. Currently, it is observed that the workers are often connected through social networks, a feature that can significantly facilitate task allocation and task execution in crowdsourcing. Therefore, this paper investigates crowdsourcing in social networks and presents a novel context-aware reliable crowdsourcing approach. In our presented approach, the two problems in traditional crowdsourcing are addressed as follows: 1) the complex tasks can be performed through autonomous coordination between the assigned worker and his contextual workers in the social network; thus, the requesters can be exempt from a heavy computing load for decomposing complex tasks into subtasks and combing the partial results of subtasks, thereby enabling more requesters to accomplish a large number of complex tasks through crowdsourcing, and 2) the reliability of a worker is determined not only by the reputation of the worker himself but also by the reputations of the contextual workers in the social network; thus, the unreliability of transient or malicious workers can be effectively addressed. The presented approach addresses two types of social networks including simplex and multiplex networks. Based on theoretical analyses and experiments on a real-world dataset, we find that the presented approach can achieve significantly higher task allocation and execution efficiency than the previous benchmark task allocation approaches; moreover, the presented contextual reputation mechanism can achieve relatively higher reliability when there are many malicious workers in the crowd.

[1]  Luca de Alfaro,et al.  Reliable Aggregation of Boolean Crowdsourced Tasks , 2015, HCOMP.

[2]  Minjie Zhang,et al.  A belief propagation-based method for task allocation in open and dynamic cloud environments , 2017, Knowl. Based Syst..

[3]  Nicola Conci,et al.  Context-aware mobile crowdsourcing , 2012, UbiComp '12.

[4]  Jiming Liu,et al.  Network-Based Modeling for Characterizing Human Collective Behaviors During Extreme Events , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[5]  Sarvapali D. Ramchurn,et al.  BudgetFix: budget limited crowdsourcing for interdependent task allocation with quality guarantees , 2014, AAMAS.

[6]  Nuno Silva,et al.  A survey of task-oriented crowdsourcing , 2015, Artificial Intelligence Review.

[7]  Victor C. M. Leung,et al.  Multidimensional context-aware social network architecture for mobile crowdsensing , 2014, IEEE Communications Magazine.

[8]  Anirban Dasgupta,et al.  Crowdsourced judgement elicitation with endogenous proficiency , 2013, WWW.

[9]  Jiming Chen,et al.  Toward optimal allocation of location dependent tasks in crowdsensing , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[10]  Stéphane Bressan,et al.  An efficient and truthful pricing mechanism for team formation in crowdsourcing markets , 2015, 2015 IEEE International Conference on Communications (ICC).

[11]  Sepehr Assadi,et al.  Online Assignment of Heterogeneous Tasks in Crowdsourcing Markets , 2015, HCOMP.

[12]  Yifeng Zeng,et al.  Structured Memetic Automation for Online Human-Like Social Behavior Learning , 2017, IEEE Transactions on Evolutionary Computation.

[13]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[14]  Aijun An,et al.  Discovering top-k teams of experts with/without a leader in social networks , 2011, CIKM '11.

[15]  Yunpeng Li,et al.  Reliable Task Allocation with Load Balancing in Multiplex Networks , 2015, TAAS.

[16]  Martin Vetterli,et al.  Locating the Source of Diffusion in Large-Scale Networks , 2012, Physical review letters.

[17]  Mihaela van der Schaar,et al.  Reputation-based incentive protocols in crowdsourcing applications , 2011, 2012 Proceedings IEEE INFOCOM.

[18]  Wen Hu,et al.  Ear-Phone: A context-aware noise mapping using smart phones , 2013, Pervasive Mob. Comput..

[19]  Athanasios V. Vasilakos,et al.  A Survey of Self-Organization Mechanisms in Multiagent Systems , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[20]  Huiji Gao,et al.  Harnessing the Crowdsourcing Power of Social Media for Disaster Relief , 2011, IEEE Intelligent Systems.

[21]  Alessandro Bozzon,et al.  Choosing the right crowd: expert finding in social networks , 2013, EDBT '13.

[22]  Lydia B. Chilton,et al.  The labor economics of paid crowdsourcing , 2010, EC '10.

[23]  Devavrat Shah,et al.  Budget-Optimal Task Allocation for Reliable Crowdsourcing Systems , 2011, Oper. Res..

[24]  Xuemin Shen,et al.  SACRM: Social Aware Crowdsourcing with Reputation Management in mobile sensing , 2014, Comput. Commun..

[25]  Christian Timmerer,et al.  Survey of web-based crowdsourcing frameworks for subjective quality assessment , 2014, 2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP).

[26]  Aijun An,et al.  Efficient Bi-objective Team Formation in Social Networks , 2012, ECML/PKDD.

[27]  Peng Dai,et al.  POMDP-based control of workflows for crowdsourcing , 2013, Artif. Intell..

[28]  Wanyuan Wang,et al.  Task Allocation for Undependable Multiagent Systems in Social Networks , 2013, IEEE Transactions on Parallel and Distributed Systems.

[29]  Mary L. Gray,et al.  The Crowd is a Collaborative Network , 2016, CSCW.

[30]  Alireza Sahami Shirazi,et al.  Location-based crowdsourcing: extending crowdsourcing to the real world , 2010, NordiCHI.

[31]  Jon Chamberlain,et al.  Groupsourcing: Distributed Problem Solving Using Social Networks , 2014, HCOMP.

[32]  Daniele Quercia,et al.  StakeSource: harnessing the power of crowdsourcing and social networks in stakeholder analysis , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[33]  Ming Yin,et al.  The Communication Network Within the Crowd , 2016, WWW.

[34]  Jong Kim,et al.  CrowdTarget: Target-based Detection of Crowdturfing in Online Social Networks , 2015, CCS.

[35]  Longbing Cao,et al.  Formalization and Verification of Group Behavior Interactions , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[36]  Nicholas R. Jennings,et al.  Trust-based fusion of untrustworthy information in crowdsourcing applications , 2013, AAMAS.

[37]  Anthony T. Chronopoulos,et al.  Cost minimization in utility computing systems , 2014, Concurr. Comput. Pract. Exp..

[38]  Agostino Poggi,et al.  Social Media for Online Collaboration in Firms and Organizations , 2016, Int. J. Inf. Syst. Model. Des..

[39]  Theodoros Lappas,et al.  Finding a team of experts in social networks , 2009, KDD.