Challenges in Collective Intelligence: A Survey

Collective Intelligence (CI) has been gaining significant attention as an effective method for decision-making and forecasting. Prediction Markets (PMs), as a subset of CI, aim to aggregate participants’ diverse opinions and knowledge to produce more accurate predictions than any individual could make alone. The unique market-based mechanism of PMs incentivizes participants to reveal their information truthfully, leading to a collectively superior prediction. However, CI and PMs have challenges, including manipulation, fallacies, and group polarization. This paper provides an overview of the challenges facing CI and PMs as tools for collective knowledge aggregation and examines the role of machine learning (ML) models as tools for amplification and hybridization in the future development of CI. Furthermore, the importance of continued research in this field is emphasized.

[1]  A. Lo,et al.  The wisdom of crowds versus the madness of mobs: An evolutionary model of bias, polarization, and other challenges to collective intelligence , 2022, SSRN Electronic Journal.

[2]  Mario Brčič,et al.  Prismal view of ethics , 2022, Philosophies.

[3]  C. Mildenberger.  Testimonial Injustice and Prediction Markets , 2022, Social Epistemology.

[4]  B. Scotney,et al.  Predictive Market Making via Machine Learning , 2022, Operations Research Forum.

[5]  Ganesh Iyer,et al.  Strategic Polarization in Group Interactions , 2021, Journal of Marketing Research.

[6]  Debdeep Mukhopadhyay,et al.  A survey on adversarial attacks and defences , 2021, CAAI Trans. Intell. Technol..

[7]  Fotios Petropoulos,et al.  Forecasting: theory and practice , 2020, International Journal of Forecasting.

[8]  Jan Stallaert,et al.  Anomalies in Probability Estimates for Event Forecasting on Prediction Markets , 2020 .

[9]  Mario Brcic,et al.  AI safety: state of the field through quantitative lens , 2020, 2020 43rd International Convention on Information, Communication and Electronic Technology (MIPRO).

[10]  Qika Lin,et al.  Collective Intelligence: A Taxonomy and Survey , 2019, IEEE Access.

[11]  M. Fricker.  Testimonial Injustice , 2019, Contemporary Epistemology.

[12]  Eugene Santos,et al.  A Failure of Collective Intelligence , 2018, 2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI).

[13]  Dario Amodei,et al.  Supervising strong learners by amplifying weak experts , 2018, ArXiv.

[14]  G. Mulgan.  Artificial intelligence and collective intelligence: the emergence of a new field , 2018, AI & SOCIETY.

[15]  Yi Liu,et al.  Literature review on collective intelligence: a crowd science perspective , 2018 .

[16]  Jan Marco Leimeister,et al.  Finding the Unicorn: Predicting Early Stage Startup Success Through a Hybrid Intelligence Method , 2017, ICIS.

[17]  Ngoc Thanh Nguyen,et al.  Prediction markets as a vital part of collective intelligence , 2017, 2017 IEEE International Conference on INnovations in Intelligent SysTems and Applications (INISTA).

[18]  Patrick Buckley,et al.  The effect of malicious manipulations on prediction market accuracy , 2015, Information Systems Frontiers.

[19]  Philip E. Tetlock,et al.  Distilling the Wisdom of Crowds: Prediction Markets versus Prediction Polls , 2017 .

[20]  Randall M. Livingstone,et al.  Models for Understanding Collective Intelligence on Wikipedia , 2016 .

[21]  Sanmay Das,et al.  Trading on a Rigged Game: Outcome Manipulation in Prediction Markets , 2016, IJCAI.

[22]  Hiroki Sayama,et al.  Collective decision making, leadership, and collective intelligence: Tests with agent-based simulations and a Field study , 2016 .

[23]  Kurt Matzler,et al.  Leadership and the wisdom of crowds: how to tap into the collective intelligence of an organization , 2016 .

[24]  Michael S. Bernstein,et al.  Handbook of Collective Intelligence , 2015 .

[25]  Alimohammad Shahri,et al.  Wisdom of the Crowd within enterprises: Practices and challenges , 2015, Comput. Networks.

[26]  Prithviraj Dasgupta,et al.  Automated Pricing in a Multiagent Prediction Market Using a Partially Observable Stochastic Game , 2015, ACM Trans. Intell. Syst. Technol..

[27]  Prithviraj Dasgupta,et al.  A multi-agent prediction market based on partially observable stochastic game , 2011, ICEC '11.

[28]  Prithviraj Dasgupta,et al.  A multi‐agent system for analyzing the effect of information on prediction markets , 2011, Int. J. Intell. Syst..

[29]  David M. Pennock,et al.  Designing Markets for Prediction , 2010, AI Mag..

[30]  Tuomas Sandholm,et al.  When do markets with simple agents fail? , 2010, AAMAS.

[31]  Vincent Conitzer,et al.  Prediction Markets, Mechanism Design, and Cooperative Game Theory , 2009, UAI.

[32]  D. MacKenzie,et al.  The use of knowledge about society , 2008 .

[33]  A. Tiwana,et al.  Does peripheral knowledge complement control? An empirical test in technology outsourcing alliances , 2007 .

[34]  Charles R. Plott,et al.  Information Aggregation Mechanisms: Concept, Design and Implementation for a Sales Forecasting Problem , 2002 .

[35]  L. Breiman.  Random Forests , 2001, Encyclopedia of Machine Learning and Data Mining.

[36]  C. Sunstein.  The Law of Group Polarization , 1999, How Change Happens.

[37]  Yoram Singer,et al.  An Efficient Boosting Algorithm for Combining Preferences by , 2013 .

[38]  Paul Myers,et al.  Knowledge Management and Organizational Design , 1996 .

[39]  P. Schrimpf,et al.  Dynamic Programming , 2011 .

[40]  James Surowiecki.  The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations Doubleday Books. , 2004 .

[41]  B. Malkiel.  Journal of Economic Perspectives—Volume 17, Number 1—Winter 2003—Pages 59 – 82 The Ef � cient Market Hypothesis and Its Critics , 2022 .