Popularity and Quality in Social News Aggregators: A Study of Reddit and Hacker News

In this paper we seek to understand the relationship between the online popularity of an article and its intrinsic quality. Prior experimental work suggests that the relationship between quality and popularity can be very distorted due to factors like social influence bias and inequality in visibility. We conduct a study of popularity on two different social news aggregators, Reddit and Hacker News. We define quality as the relative number of votes an article would have received if each article was shown, in a bias-free way, to an equal number of users. We propose a simple poisson regression method to estimate this quality metric from time-series voting data. We validate our methods on data from Reddit and Hacker News, as well the experimental data from prior work. This method works well even though the collected data is subject to common social media biases. Using these estimates, we find that popularity on Reddit and Hacker News is a stronger reflection of intrinsic quality than expected.

[1]  Matthew J. Salganik,et al.  Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market , 2006, Science.

[2]  Nick Craswell,et al.  An experimental comparison of click position-bias models , 2008, WSDM '08.

[3]  Matthew J. Salganik,et al.  Leading the Herd Astray: An Experimental Study of Self-fulfilling Prophecies in an Artificial Cultural Market , 2008, Social psychology quarterly.

[4]  Jure Leskovec,et al.  What's in a Name? Understanding the Interplay between Titles, Content, and Communities in Social Media , 2013, ICWSM.

[5]  Chun Liu,et al.  Social Influence Bias : A Randomized Experiment , 2014 .

[6]  Jure Leskovec,et al.  Can cascades be predicted? , 2014, WWW.

[7]  Tad Hogg,et al.  Effects of Social Influence in Peer Online Recommendation , 2014, ArXiv.

[8]  Jussara M. Almeida,et al.  Using early view patterns to predict the popularity of youtube videos , 2013, WSDM.

[9]  Kristina Lerman,et al.  The Simple Rules of Social Contagion , 2013, Scientific Reports.

[10]  Pascal Van Hentenryck,et al.  Measuring and Optimizing Cultural Markets , 2014, ArXiv.

[11]  Kristina Lerman,et al.  Disentangling the Effects of Social Signals , 2014, Hum. Comput..

[12]  Tad Hogg,et al.  Using a model of social dynamics to predict popularity of news , 2010, WWW '10.

[13]  Bernardo A. Huberman,et al.  The Pulse of News in Social Media: Forecasting Popularity , 2012, ICWSM.

[14]  Daniel G. Goldstein,et al.  The structure of online diffusion networks , 2012, EC '12.

[15]  Alex Leavitt,et al.  Upvoting hurricane Sandy: event-based news production processes on a social news site , 2014, CHI.

[16]  SzaboGabor,et al.  Predicting the popularity of online content , 2010 .

[17]  Matthew Richardson,et al.  Predicting clicks: estimating the click-through rate for new ads , 2007, WWW '07.

[18]  Ye Chen,et al.  Position-normalized click prediction in search advertising , 2012, KDD.

[19]  Paul Resnick,et al.  Slash(dot) and burn: distributed moderation in a large online conversation space , 2004, CHI.

[20]  Benjamin Piwowarski,et al.  A user browsing model to predict search engine click data from past observations. , 2008, SIGIR '08.

[21]  Lada A. Adamic,et al.  The role of social networks in information diffusion , 2012, WWW.

[22]  Berkant Barla Cambazoglu,et al.  On the Feasibility of Predicting News Popularity at Cold Start , 2014, SocInfo.

[23]  Bernardo A. Huberman,et al.  Predicting the popularity of online content , 2008, Commun. ACM.

[24]  Justin Cheng,et al.  Rumor Cascades , 2014, ICWSM.

[25]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[26]  Kristina Lerman,et al.  Leveraging Position Bias to Improve Peer Recommendation , 2014, PloS one.

[27]  Galen Pickard,et al.  Quantifying Social Influence in an Online Cultural Market , 2012, PloS one.

[28]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[29]  Eric Gilbert,et al.  Widespread underprovision on Reddit , 2013, CSCW.

[30]  Tad Hogg,et al.  Stochastic Models of User-Contributory Web Sites , 2009, ICWSM.

[31]  Duncan J. Watts,et al.  Everyone's an influencer: quantifying influence on twitter , 2011, WSDM '11.