Human content filtering in Twitter: The influence of metadata

Social micro-blogging systems such as Twitter are designed for rapid and informal communication from a large potential number of participants. Due to the volume of content received, human users must typically skim their timeline of received content and exercise judgement in selecting items for consumption, necessitating a selection process based on heuristics and content meta-data. This selection process is not well understood, yet is important due to its potential use in content management systems.In this research we have conducted an open online experiment in which participants are shown quantitative and qualitative meta-data describing two pieces of Twitter content. Without revealing the text of the tweet, participants are asked to make a selection. We observe the decisions made from 239 surveys and discover insights into human behaviour on decision making for content selection. We find that for qualitative meta-data consumption decisions are driven by online friendship and for quantitative meta-data the largest numerical value presented influences choice. Overall, the 'number of retweets' is found to be the most influential quantitative meta-data, while displaying multiple cues about an author's identity provides the strongest qualitative meta-data. When both quantitative and qualitative meta-data is presented, it is the qualitative meta-data (friendship information) that drives selection. The results are consistent with application of the Recognition heuristic, which postulates that when faced with constrained decision-making, humans will tend to exercise judgement based on cues representing familiarity. These findings are useful for future interface design for content filtering and recommendation systems. Author-HighlightsWe have examined which metadata cues are used when deciding to consume content.The online experiment used Twitter content and users as its subject.Users prefer content from someone with whom they already have a relationship.Users prefer content judged by others to have value (based on number of retweets).Clear metadata signals can affect decision making in content consumption.

[1]  Brendan T. O'Connor,et al.  Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks , 2008, EMNLP.

[2]  Gilad Mishne,et al.  Finding high-quality content in social media , 2008, WSDM '08.

[3]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[4]  Kimberly A. Barchard,et al.  Practical advice for conducting ethical online experiments and questionnaires for United States psychologists , 2008, Behavior research methods.

[5]  Carla Simone,et al.  Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work Companion , 2012, CSCW 2012.

[6]  Will Webberley,et al.  Retweeting: A study of message-forwarding in twitter , 2011, 2011 Workshop on Mobile and Online Social Networks.

[7]  Junghoo Cho,et al.  Topical semantics of twitter links , 2011, WSDM '11.

[8]  Timothy W. Finin,et al.  Why we twitter: understanding microblogging usage and communities , 2007, WebKDD/SNA-KDD '07.

[9]  Jianjun Yu,et al.  Mining User Interest and Its Evolution for Recommendation on the Micro-blogging System , 2013, WAIM.

[10]  Mary Beth Rosson,et al.  How and why people Twitter: the role that micro-blogging plays in informal communication at work , 2009, GROUP.

[11]  Ulf-Dietrick Reips,et al.  Internet-Based Psychological Experimenting , 2002 .

[12]  Scott Counts,et al.  Tweeting is believing?: understanding microblog credibility perceptions , 2012, CSCW.

[13]  R. Ratcliff,et al.  Similarity information versus relational information: Differences in the time course of retrieval , 1989, Cognitive Psychology.

[14]  Harry Shum,et al.  An Empirical Study on Learning to Rank of Tweets , 2010, COLING.

[15]  R. Hertwig,et al.  How forgetting aids heuristic inference. , 2005, Psychological review.

[16]  Ulf-Dietrich Reips.  Internet-Based Psychological Experimenting , 2002 .

[17]  Ulf-Dietrich Reips.  Standards for Internet-based experimenting. , 2002, Experimental psychology.

[18]  Jenny Chen,et al.  Opportunities for Crowdsourcing Research on Amazon Mechanical Turk , 2011 .

[19]  Qi Gao,et al.  Analyzing user modeling on twitter for personalized news recommendations , 2011, UMAP'11.

[20]  Barry Smyth,et al.  Using twitter to recommend real-time topical news , 2009, RecSys '09.

[21]  Gerd Gigerenzer,et al.  Models of ecological rationality: the recognition heuristic. , 2002, Psychological review.

[22]  Gerd Gigerenzer,et al.  Heuristic decision making. , 2011, Annual review of psychology.

[23]  Andrea Gaggioli,et al.  Working the Crowd , 2008, Science.

[24]  Amit P. Sheth,et al.  Personalized Filtering of the Twitter Stream , 2011, SPIM.

[25]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[26]  Sreenivas Gollapudi,et al.  Ranking mechanisms in twitter-like forums , 2010, WSDM '10.

[27]  U. Hoffrage,et al.  Fast, frugal, and fit: Simple heuristics for paired comparison , 2002 .

[28]  Martine De Cock,et al.  Ranking Approaches for Microblog Search , 2010, 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[29]  Qi He,et al.  TwitterRank: finding topic-sensitive influential twitterers , 2010, WSDM '10.

[30]  Mario Cataldi,et al.  Emerging topic detection on Twitter based on temporal and social terms evaluation , 2010, MDMKDD '10.

[31]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[32]  Hiroyuki Kitagawa,et al.  TURank: Twitter User Ranking Based on User-Tweet Graph Analysis , 2010, WISE.

[33]  Daniele Quercia,et al.  Our Twitter Profiles, Our Selves: Predicting Personality with Twitter , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[34]  Peter C. Fishburn,et al.  LEXICOGRAPHIC ORDERS, UTILITIES AND DECISION RULES: A SURVEY , 1974 .

[35]  John Hannon,et al.  Recommending twitter users to follow using content and collaborative filtering approaches , 2010, RecSys '10.

[36]  Phuoc Tran-Gia,et al.  Modeling of crowdsourcing platforms and granularity of work organization in Future Internet , 2011, 2011 23rd International Teletraffic Congress (ITC).

[37]  L. R. Goldberg.  The structure of phenotypic personality traits. , 1993, The American psychologist.

[38]  Stuart M. Allen,et al.  Better the Tweeter You Know: Social Signals on Twitter , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[39]  Alek Felstiner.  Working the Crowd: Employment and Labor Law in the Crowdsourcing Industry , 2011 .

[40]  Ben Carterette,et al.  An Analysis of Assessor Behavior in Crowdsourced Preference Judgments , 2010 .