Bayesian Vote Weighting in Crowdsourcing Systems

In social collaborative crowdsourcing platforms, the votes which people give on the content generated by others is a very important component of the system which seeks to find the best content through collaborative action. In a crowdsourced innovation platform, people vote on innovations/ideas generated by others which enables the system to synthesize the view of the crowd about an idea. However, in many such systems gaming or vote spamming as it is commonly known is prevalent. In this paper we present a Bayesian mechanism for weighting the actual vote given by a user to compute an effective vote which incorporates the voters history of voting and also what the crowd is thinking about the value of the innovation. The model results into some interesting insights about social voting systems and new avenues for gamification.

[1]  Thomas Hofmann,et al.  Lies and propaganda: detecting spam users in collaborative filtering , 2007, IUI '07.

[2]  Robert P. W. Duin,et al.  Limits on the majority vote accuracy in classifier fusion , 2003, Pattern Analysis & Applications.

[3]  Gilad Mishne,et al.  YR-2007-005 FINDING HIGH-QUALITY CONTENT IN SOCIAL MEDIA WITH AN APPLICATION TO COMMUNITY-BASED QUESTION ANSWERING , 2007 .

[4]  W. Bruce Croft,et al.  A framework to predict the quality of answers with non-textual features , 2006, SIGIR.

[5]  Qi Su,et al.  Internet-scale collection of human-reviewed data , 2007, WWW '07.

[6]  Eugene Agichtein,et al.  A few bad votes too many?: towards robust ranking in social media , 2008, AIRWeb '08.

[7]  Gilad Mishne,et al.  Finding high-quality content in social media , 2008, WSDM '08.

[8]  R. Catrambone,et al.  Proceedings of the 32nd Annual Conference of the Cognitive Science Society , 2010 .

[9]  M. Lee,et al.  Wisdom of the Crowds in Minimum Spanning Tree Problems , 2010 .

[10]  David M. Pennock,et al.  An Empirical Comparison of Algorithms for Aggregating Expert Predictions , 2006, UAI.

[11]  Fan Chung Graham,et al.  Internet and Network Economics, Third International Workshop, WINE 2007, San Diego, CA, USA, December 12-14, 2007, Proceedings , 2007, WINE.

[12]  James Surowiecki The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations Doubleday Books. , 2004 .

[13]  Morgan Sonderegger,et al.  Proceedings of the 35th Annual Conference of the Cognitive Science Society , 2013 .

[14]  Filip Radlinski,et al.  Minimally Invasive Randomization for Collecting Unbiased Preferences from Clickthrough Logs , 2006, AAAI 2006.

[15]  Michael D. Lee,et al.  Conjunction and Disjunction Fallacies in Prediction Markets , 2009 .

[16]  Brandon M. Turner,et al.  A Wisdom of the Crowd Approach to Forecasting , 2011 .

[17]  Nicole Immorlica,et al.  Click Fraud Resistant Methods for Learning Click-Through Rates , 2005, WINE.