Two Sides of Collective Decision Making - Votes from Crowd and Knowledge from Experts

This paper deals with the role of experts and crowds in solving important societal issues. The authors argue that both experts and crowds are important stakeholders in collective decision making which should jointly participate in the decision-making process to improve it. Usually studied in different research areas, there have been a few models that integrate crowds and experts in a joint model. The authors give an overview of the advantages and disadvantages of crowd and expert decision making and highlight possibilities to connect these two worlds. They position the research in the area of Computational Social Choice (COMSOC) and crowd voting, emerging fields that bring great potential for collective decision making. COMSOC focuses on improving social welfare and the quality of products and services through the inclusion of community or clients into the decision-making process. Despite these altruistic goals, there are several shortcomings that call for the engagement of experts in voting procedures. The authors propose a simple participatory model for weighting and selection of voters and votes through the integration of expert rankings into crowd voting systems.

[1]  I. Rahwan,et al.  Verification in Referral-Based Crowdsourcing , 2012, PloS one.

[2]  Boualem Benatallah,et al.  Quality Control in Crowdsourcing , 2018, ACM Comput. Surv..

[3]  Francesca Rossi,et al.  Multi-agent soft constraint aggregation via sequential voting: theoretical and experimental results , 2018, Autonomous Agents and Multi-Agent Systems.

[4]  P. Faliszewski,et al.  Control and Bribery in Voting , 2016, Handbook of Computational Social Choice.

[5]  Michael S. Bernstein,et al.  Boomerang: Rebounding the Consequences of Reputation Feedback on Crowdsourcing Platforms , 2016, UIST.

[6]  Svetlana Obraztsova,et al.  Strategic Voting with Incomplete Information , 2016, IJCAI.

[7]  Nihar B. Shah,et al.  No Oops, You Won't Do It Again: Mechanisms for Self-correction in Crowdsourcing , 2016, ICML.

[8]  Piotr Faliszewski,et al.  Complexity of manipulation, bribery, and campaign management in Bucklin and fallback voting , 2013, Autonomous Agents and Multi-Agent Systems.

[9]  Ian P. McCarthy,et al.  How to Work a Crowd: Developing Crowd Capital Through Crowdsourcing , 2014, ArXiv.

[10]  Nicholas R. Miller.  Reflections on Arrow’s theorem and voting rules , 2019 .

[11]  Yi Liu,et al.  Literature review on collective intelligence: a crowd science perspective , 2018 .

[12]  Lydia B. Chilton,et al.  MicroTalk: Using Argumentation to Improve Crowdsourcing Accuracy , 2016, HCOMP.

[13]  Benjamin Livshits,et al.  Saving Money While Polling with InterPoll Using Power Analysis , 2014, HCOMP.

[14]  Tanja Aitamurto,et al.  Crowdsourcing for Participatory Democracies: Efficient Elicitation of Social Choice Functions , 2014, HCOMP.

[15]  Ulrich Endriss,et al.  Group Manipulation in Judgment Aggregation , 2016, AAMAS.

[16]  Stephen Chen,et al.  Efficient elicitation of software configurations using crowd preferences and domain knowledge , 2018, Automated Software Engineering.

[17]  P. Jean-Jacques Herings,et al.  The Condorcet paradox revisited , 2010, Social Choice and Welfare.

[18]  Karl Aberer,et al.  Minimizing Efforts in Validating Crowd Answers , 2015, SIGMOD Conference.

[19]  Ulrich Endriss,et al.  Aggregating Dependency Graphs into Voting Agendas in Multi-Issue Elections , 2011, IJCAI.

[20]  Katja Knecht,et al.  Citizen Engagement through Design Space Exploration Integrating citizen knowledge and expert design in computational urban planning , 2019, Blucher Design Proceedings.

[21]  Alex Bennet,et al.  The Decision-Making Process in a Complex Situation , 2008 .

[22]  Edith Elkind,et al.  Computational Social Choice: The First Ten Years and Beyond , 2019, Computing and Software Science.

[23]  Zorica Dodevska,et al.  Computational Social Choice and challenges оf voting in multi-agent systems , 2019, Tehnika.

[24]  Y. Narahari,et al.  Frugal bribery in voting , 2017, Theor. Comput. Sci..

[25]  Ulrich Endriss,et al.  Social Choice Theory as a Foundation for Multiagent Systems , 2014, MATES.

[26]  Tanja Aitamurto,et al.  Unmasking the crowd: participants’ motivation factors, expectations, and profile in a crowdsourced law reform , 2017 .

[27]  De Liu,et al.  Pure and hybrid crowds in crowdfunding markets , 2016 .

[28]  Matthias Hirth,et al.  Using Attention Testing to Select Crowdsourced Workers and Research Participants , 2019, Social Science Computer Review.

[29]  Vincent Conitzer,et al.  Introduction to Computational Social Choice , 2016, Handbook of Computational Social Choice.

[30]  Piotr Faliszewski,et al.  The complexity of fully proportional representation for single-crossing electorates , 2015, Theor. Comput. Sci..

[31]  Yann Chevaleyre,et al.  A Short Introduction to Computational Social Choice , 2007, SOFSEM.

[32]  Ashish Goel,et al.  Knapsack Voting for Participatory Budgeting , 2019, ACM Trans. Economics and Comput..

[33]  Ulrich Endriss.  Computational Social Choice: Prospects and Challenges , 2011, FET.

[34]  Tera D. Letzring,et al.  Exploring beyond simple demographic variables: Differences between traditional laboratory samples and crowdsourced online samples on the Big Five personality traits , 2017, Personality and Individual Differences.

[35]  Ksenija Mandic,et al.  Modeling Interactions Among Criteria in MCDM Methods: A Review , 2015, ICDSST.

[36]  Tanja Aitamurto,et al.  Five Design Principles for Crowdsourced Policymaking: Assessing the Case of Crowdsourced Off-Road Traffic Law in Finland , 2015 .

[37]  De Liu,et al.  Effect of Crowd Voting on Participation in Crowdsourcing Contests , 2020, J. Manag. Inf. Syst..

[38]  Saman Haratizadeh,et al.  Reliable graph-based collaborative ranking , 2018, Inf. Sci..

[39]  A. Ghezzi,et al.  Crowdsourcing: A Review and Suggestions for Future Research , 2018 .

[40]  Piotr Faliszewski,et al.  Committee Scoring Rules: Axiomatic Classification and Hierarchy , 2016, IJCAI.

[41]  J. Nadal,et al.  Manifesto of computational social science , 2012 .

[42]  Michael S. Bernstein,et al.  A Glimpse Far into the Future: Understanding Long-term Crowd Worker Quality , 2016, CSCW.

[43]  Isma Farah Siddiqui,et al.  To enhance Effectiveness of Crowdsource Software Testing by applying Personality Types , 2019, ICSIE.

[44]  Saman Haratizadeh,et al.  Graph-based collaborative ranking , 2016, Expert Syst. Appl..

[45]  M. Haan,et al.  Expert Judgment Versus Public Opinion – Evidence from the Eurovision Song Contest , 2005 .

[46]  F. O. Isinkaye,et al.  Recommendation systems: Principles, methods and evaluation , 2015 .

[47]  Francesca Rossi.  Preferences, Constraints, Uncertainty, and Multi-Agent Scenarios , 2008, ISAIM.

[48]  José María Moreno-Jiménez,et al.  E-democracy and knowledge: A multicriteria framework for the new democratic era , 2003 .