Enhancing crowd wisdom using measures of diversity computed from social media data

"Wisdom of Crowds" (WoC) refers to a form of collective intelligence in which the aggregate judgment of a group of individuals is, in most instances, superior to that of any one group member. For a crowd to be wise, its members must possess diverse knowledge and viewpoints. Such diversity leads to uncorrelated judgment errors that cancel out in aggregate. Yet despite the fact that diversity is known to be an essential ingredient in WoC, little research aims to measure and exploit diversity in human social systems for the purpose of maximizing crowd intelligence. Here we quantify the diversity of a group of individuals through semantic analysis of their social media (Twitter) communications. Focusing on the domain of fantasy sports, we show that virtual crowds of fantasy team owners selected based on the diversity of their tweet content can outperform both non-diverse and randomly sampled crowds. Our results suggest a new approach for intelligent crowd assembly in which measures of diversity extracted from online social media communications can guide the selection of crowd members. These results have implications for numerous domains that utilize aggregated judgments - from consumer reviews, to econometrics, to geopolitical forecasting and intelligence analysis.

[1]  R. Preston McAfee,et al.  The wisdom of smaller, smarter crowds , 2014, EC.

[2]  H. Van Dyke Parunak,et al.  Estimating Diversity among Forecaster Models , 2012, AAAI Fall Symposium: Machine Aggregation of Human Judgment.

[3]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[4]  Hang Li,et al.  Convolutional Neural Network Architectures for Matching Natural Language Sentences , 2014, NIPS.

[5]  Thomas Demeester,et al.  Learning Semantic Similarity for Very Short Texts , 2015, 2015 IEEE International Conference on Data Mining Workshop (ICDMW).

[6]  E Nevo,et al.  Evolution of genome–phenome diversity under environmental stress , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Lu Hong,et al.  Groups of diverse problem solvers can outperform groups of high-ability problem solvers. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Philip Tetlock,et al.  The psychology of intelligence analysis: drivers of prediction accuracy in world politics. , 2015, Journal of experimental psychology. Applied.

[9]  Ludmila I. Kuncheva,et al.  Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy , 2003, Machine Learning.

[10]  S. Pavoine,et al.  Measuring diversity from dissimilarities with Rao's quadratic entropy: are any dissimilarities suitable? , 2005, Theoretical population biology.

[11]  Richard P. Larrick,et al.  The social psychology of the wisdom of crowds. , 2012 .

[12]  A. Pentland,et al.  Computational Social Science , 2009, Science.

[13]  Jason Weston,et al.  #TagSpace: Semantic Embeddings from Hashtags , 2014, EMNLP.

[14]  Wesley De Neve,et al.  Alleviating manual feature engineering for part-of-speech tagging of Twitter microposts using distributed word representations , 2014, NIPS 2014.

[15]  Johanna D. Moore,et al.  Twitter Sentiment Analysis: The Good the Bad and the OMG! , 2011, ICWSM.

[16]  D. Helbing,et al.  How social influence can undermine the wisdom of crowd effect , 2011, Proceedings of the National Academy of Sciences.

[17]  David V. Budescu,et al.  When is a crowd wise? , 2014, ArXiv.

[18]  A. Pentland,et al.  Life in the network: The coming age of computational social science: Science , 2009 .

[19]  Amit P. Sheth,et al.  Word Embeddings to Enhance Twitter Gang Member Profile Identification , 2016, ArXiv.

[20]  Henrik Olsson,et al.  A Comparison of Small Crowd Selection Methods , 2015, CogSci.

[21]  A. Solow,et al.  Measuring biological diversity , 2006, Environmental and Ecological Statistics.

[22]  Katherine W. Phillips,et al.  Social Category Diversity Promotes Premeeting Elaboration: The Role of Relationship Focus , 2012, Organ. Sci..

[23]  A. Stirling.  A general framework for analysing diversity in science, technology and society , 2007, Journal of The Royal Society Interface.

[24]  Michael Vitale,et al.  The Wisdom of Crowds , 2015, Cell.