Distributed ACO Based Reputation Management in Crowdsourcing

Crowdsourcing is an economical and efficient tool that hires human labour to execute tasks which are difficult to solve otherwise. Verification of the quality of the workers is a major problem in Crowd sourcing. We need to judge the performance of the workers based on their history of service and it is difficult to do so without hiring other workers. In this paper, we propose an Ant Colony Optimization (ACO) based reputation management system that can differentiate between good and bad workers. Using experimental evaluation, we show that, the algorithm works fine on the real scenario and efficiently differentiate workers with higher reputations.

[1]  Eric Horvitz,et al.  Incentives for truthful reporting in crowdsourcing , 2012, AAMAS.

[2]  Krishna P. Gummadi,et al.  An analysis of social network-based Sybil defenses , 2010, SIGCOMM '10.

[3]  Anirban Dasgupta,et al.  Aggregating crowdsourced binary ratings , 2013, WWW.

[4]  Lakshminarayanan Subramanian,et al.  Reputation-based Worker Filtering in Crowdsourcing , 2014, NIPS.

[5]  Subhasis Thakur,et al.  Expertise Discovery in Decentralised Online Social Networks , 2017, ASONAM.

[6]  Chun How Tan,et al.  Trust, but verify: predicting contribution quality for knowledge base construction and curation , 2014, WSDM.

[7]  Yan Wang,et al.  Crowd Trust: A Context-Aware Trust Model for Worker Selection in Crowdsourcing Environments , 2015, 2015 IEEE International Conference on Web Services.

[8]  Tom Minka,et al.  How To Grade a Test Without Knowing the Answers - A Bayesian Graphical Model for Adaptive Crowdsourcing and Aptitude Testing , 2012, ICML.

[9]  Tarek Awwad,et al.  Context-aware worker selection for efficient quality control in crowdsourcing. (Selection contextuelle de travailleurs pour un contrôle de qualité efficace dans les systèmes de production participative) , 2018 .

[10]  Shijun Liu,et al.  Distributed ACO based on a crowdsourcing model for multiobjective problem , 2017, 2017 IEEE 21st International Conference on Computer Supported Cooperative Work in Design (CSCWD).

[11]  Aleksandrs Slivkins,et al.  Adaptive contract design for crowdsourcing markets: bandit algorithms for repeated principal-agent problems , 2014, J. Artif. Intell. Res..

[12]  Jurij Silc,et al.  Ant Colonies and the Mesh-Partitioning Problem , 2005, Handbook of Bioinspired Algorithms and Applications.

[13]  Alex Kass,et al.  Trustworthiness in Enterprise Crowdsourcing: A Taxonomy & Evidence from Data , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[14]  Zhiqi Shen,et al.  Bringing reputation-awareness into crowdsourcing , 2013, 2013 9th International Conference on Information, Communications & Signal Processing.

[15]  Chunyan Miao,et al.  An Incentive Mechanism to Elicit Truthful Opinions for Crowdsourced Multiple Choice Consensus Tasks , 2015, 2015 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT).

[16]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[17]  Antonio Fernández Anta,et al.  Algorithmic Mechanisms for Reliable Crowdsourcing Computation under Collusion , 2015, PloS one.

[18]  Hongxia Yang,et al.  Adversarial Detection with Model Interpretation , 2018, KDD.

[19]  Yair Movshovitz-Attias,et al.  Analysis of the reputation system and user contributions on a question answering website: StackOverflow , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).