Effect of Crowd Composition on the Wisdom of Artificial Crowds Metaheuristic

This paper investigates the impact that task difficulty and crowd composition have on the success of the Wisdom of Artificial Crowds metaheuristic. The metaheuristic, which is inspired by the wisdom of crowds phenomenon, combines the intelligence from a group of optimization searches to form a new solution. Unfortunately, the aggregate formed by the metaheuristic is not always better than the best individual solution within the crowd, and little is known about the variables which maximize the metaheuristic’s success. Our study offers new insights into the influential factors of artificial crowds and the collective intelligence of multiple optimization searches performed on the same problem. The results show that favoring the opinions of experts (i.e., the better searches) improves the chances of the metaheuristic succeeding by more than 15% when compared to the traditional means of equal weighting. Furthermore, weighting expertise was found to require smaller crowd sizes for the metaheuristic to reach its peak chances of success. Finally, crowd size was discovered to be a critical factor, especially as problem complexity grows or average crowd expertise declines. However, crowd size matters only up to a point, after which the probability of success plateaus.

[1]  Roman V. Yampolskiy,et al.  Wisdom of Artificial Crowds—A Metaheuristic Algorithm for Optimization , 2012 .

[2]  Roman V. Yampolskiy,et al.  GA with Wisdom of Artificial Crowds for Solving Mastermind Satisfiability Problem , 2011, Int. J. Intell. Games Simul..

[3]  Roman Yampolskiy,et al.  Shortest Total Path Length Spanning Tree via Wisdom of Artificial Crowds Algorithm , 2017, MAICS.

[4]  Daniel M. Romero,et al.  Crowd Size, Diversity and Performance , 2015, CHI.

[5]  Ryan Hughes,et al.  Solving Sudoku Puzzles with Wisdom of Artificial Crowds , 2012, Int. J. Intell. Games Simul..

[6]  Matthew Welsh,et al.  Expertise and the Wisdom of Crowds: Whose Judgments to Trust and When , 2012, CogSci.

[7]  Christopher J. Lowrance,et al.  Evolution of a Metaheuristic for Aggregating Wisdom from Artificial Crowds , 2015, EPIA.

[8]  Aniket Kittur,et al.  Harnessing the wisdom of crowds in wikipedia: quality through coordination , 2008, CSCW.

[9]  Mika Mäntylä,et al.  More testers - The effect of crowd size and time restriction in software testing , 2013, Inf. Softw. Technol..

[10]  Ayoung Suh,et al.  The Wisdom of Crowds: Impact of Collective Size and Expertise Transfer on Collective Performance , 2014, 2014 47th Hawaii International Conference on System Sciences.

[11]  Roman V. Yampolskiy,et al.  Wisdom of artificial crowds feature selection in untargeted metabolomics: An application to the development of a blood-based diagnostic test for thrombotic myocardial infarction , 2017, bioRxiv.

[12]  Roman V. Yampolskiy,et al.  Genetic algorithm and Wisdom of Artificial Crowds algorithm applied to Light up , 2011, 2011 16th International Conference on Computer Games (CGAMES).

[13]  Roman V. Yampolskiy,et al.  Using a GA and Wisdom of Artificial Crowds to solve solitaire battleship puzzles , 2012, 2012 17th International Conference on Computer Games (CGAMES).

[14]  Heiko A. von der Gracht,et al.  Testing weighting approaches for forecasting in a Group Wisdom Support System environment , 2016 .

[15]  Roman V. Yampolskiy,et al.  Wisdom of artificial crowds algorithm for solving NP-hard problems , 2011, Int. J. Bio Inspired Comput..

[16]  Ashok N. Srivastava,et al.  Data Mining: Concepts, Models, Methods, and Algorithms , 2005, J. Comput. Inf. Sci. Eng..

[17]  Tyler Moore,et al.  Evaluating the Wisdom of Crowds in Assessing Phishing Websites , 2008, Financial Cryptography.

[18]  Marko Velic,et al.  Wisdom of crowds algorithm for stock market predictions , 2013, Proceedings of the ITI 2013 35th International Conference on Information Technology Interfaces.

[19]  John Riedl,et al.  The effects of diversity on group productivity and member withdrawal in online volunteer groups , 2010, CHI.

[20]  Chris Preist,et al.  Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems , 2015 .

[21]  熊谷 ユリヤ,et al.  James Surowiecki, 『The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations』, Random House, 5,2004, $24.95 , 2005 .