Crowds on Wall Street: Extracting Value from Collaborative Investing Platforms

In crowdsourced systems, it is often difficult to separate the highly capable "experts" from the average worker. In this paper, we study the problem of evaluating and identifying experts in the context of SeekingAlpha and StockTwits, two crowdsourced investment services that are encroaching on a space dominated for decades by large investment banks. We seek to understand the quality and impact of content on collaborative investment platforms, by empirically analyzing complete datasets of SeekingAlpha articles (9 years) and StockTwits messages (4 years). We develop sentiment analysis tools and correlate contributed content to the historical performance of relevant stocks. While SeekingAlpha articles and StockTwits messages provide minimal correlation to stock performance in aggregate, a subset of experts contribute more valuable (predictive) content. We show that these authors can be easily identified by user interactions, and investments using their analysis significantly outperform broader markets. Finally, we conduct a user survey that sheds light on users views of SeekingAlpha content and stock manipulation.

[1]  Paulo Cortez,et al.  On the Predictability of Stock Market Behavior Using StockTwits Sentiment and Posting Volume , 2013, EPIA.

[2]  Wai Lam,et al.  Stock prediction: Integrating text mining approach using real-time news , 2003, 2003 IEEE International Conference on Computational Intelligence for Financial Engineering, 2003. Proceedings..

[3]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[4]  Tim Loughran,et al.  When is a Liability not a Liability? Textual Analysis, Dictionaries, and 10-Ks , 2010 .

[5]  H. Stanley,et al.  Quantifying Trading Behavior in Financial Markets Using Google Trends , 2013, Scientific Reports.

[6]  Eric D. Brown.  Will Twitter Make You a Better Investor? A Look at Sentiment, User Reputation and Their Effect on the Stock Market , 2012 .

[7]  K. Pearson.  Contributions to the Mathematical Theory of Evolution. II. Skew Variation in Homogeneous Material , 1895 .

[8]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[9]  Yu-An Sun,et al.  When majority voting fails: Comparing quality assurance methods for noisy human computation environment , 2012, ArXiv.

[10]  Gang Wang,et al.  Serf and turf: crowdturfing for fun and profit , 2011, WWW.

[11]  Jeffrey Nichols,et al.  Analyzing the quality of information solicited from targeted strangers on social media , 2013, CSCW '13.

[12]  Gang Wang,et al.  Social Turing Tests: Crowdsourcing Sybil Detection , 2012, NDSS.

[13]  Brian P. Bailey,et al.  Voyant: generating structured feedback on visual designs using a crowd of non-experts , 2014, CSCW.

[14]  Fabrício Benevenuto,et al.  Comparing and combining sentiment analysis methods , 2013, COSN '13.

[15]  Andrea Esuli,et al.  PageRanking WordNet Synsets: An Application to Opinion Mining , 2007, ACL.

[16]  Junlan Feng,et al.  Robust Sentiment Detection on Twitter from Biased and Noisy Data , 2010, COLING.

[17]  Ronen Feldman,et al.  Identifying and Following Expert Investors in Stock Microblogs , 2011, EMNLP.

[18]  Munmun De Choudhury,et al.  Can blog communication dynamics be correlated with stock market activity? , 2008, Hypertext.

[19]  Sameena Shah,et al.  Winning by Following the Winners: Mining the Behaviour of Stock Market Experts in Social Media , 2014, SBP.

[20]  Aniket Kittur,et al.  Collaborative problem solving: a study of MathOverflow , 2014, CSCW.

[21]  Jahna Otterbacher,et al.  'Helpfulness' in online communities: a measure of message quality , 2009, CHI.

[22]  Gang Wang,et al.  Wisdom in the social crowd: an analysis of quora , 2013, WWW.

[23]  Krishna P. Gummadi,et al.  Measurement and analysis of online social networks , 2007, IMC '07.

[24]  Hsinchun Chen,et al.  Textual analysis of stock market prediction using breaking financial news: The AZFin text system , 2009, TOIS.

[25]  Olivia Sheng,et al.  Investigating Predictive Power of Stock Micro Blog Sentiment in Forecasting Future Stock Price Directional Movement , 2011, ICIS.

[26]  H. Eugene Stanley,et al.  Quantifying Wikipedia Usage Patterns Before Stock Market Moves , 2013, Scientific Reports.

[27]  Michael S. Bernstein,et al.  The future of crowd work , 2013, CSCW.

[28]  Michael S. Bernstein,et al.  Ensemble: exploring complementary strengths of leaders and crowds in creative collaboration , 2014, CSCW.

[29]  Yigitcan Karabulut.  Can Facebook Predict Stock Market Activity? , 2013 .

[30]  Eric Gilbert,et al.  Widespread Worry and the Stock Market , 2010, ICWSM.

[31]  Isabell M. Welpe,et al.  Tweets and Trades: The Information Content of Stock Microblogs , 2010 .

[32]  Steven Skiena,et al.  Large-Scale Sentiment Analysis for News and Blogs (system demonstration) , 2007, ICWSM.

[33]  Sameena Shah,et al.  Stock Prediction Using Event-Based Sentiment Analysis , 2013, 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT).

[34]  Johan Bollen,et al.  Twitter mood predicts the stock market , 2010, J. Comput. Sci..

[35]  Lada A. Adamic,et al.  Knowledge sharing and yahoo answers: everyone knows something , 2008, WWW.

[36]  Gang Wang,et al.  Man vs. Machine: Practical Adversarial Detection of Malicious Crowdsourcing Workers , 2014, USENIX Security Symposium.

[37]  Wai-Tat Fu,et al.  Understanding experts' and novices' expertise judgment of twitter users , 2012, CHI.

[38]  K. Pearson.  Contributions to the Mathematical Theory of Evolution , 1894 .

[39]  Isabell M. Welpe,et al.  Tweets and Trades: The Information Content of Stock Microblogs , 2010 .

[40]  Ben Y. Zhao,et al.  User interactions in social networks and their implications , 2009, EuroSys '09.

[41]  Clifton Forlines,et al.  Crowdsourcing the future: predictions made with a social network , 2014, CHI.

[42]  Ido Guy,et al.  The perception of others: inferring reputation from social media in the enterprise , 2014, CSCW.

[43]  S. Pokharel.  Wisdom of Crowds: The Value of Stock Opinions Transmitted through Social Media , 2014 .

[44]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[45]  Sheizaf Rafaeli,et al.  Predictors of answer quality in online Q&A sites , 2008, CHI.

[46]  T. Rao,et al.  Analyzing Stock Market Movements Using Twitter Sentiment Analysis , 2012, ASONAM 2012.

[47]  Robert E. Verrecchia,et al.  Constraints on short-selling and asset price adjustment to private information , 1987 .