The Impact of Twitter Features on Credibility Ratings - An Explorative Examination Combining Psychological Measurements and Feature Based Selection Methods

In a post-truth age determined by Social Media channels providing large amounts of information of questionable credibility while at the same time people increasingly tend to rely on online information, the ability to detect whether content is believable is developing into an important challenge. Most of the work in that field suggested automated approaches to perform binary classification to determine information veracity. Recipients ́ perspectives and multidimensional psychological credibility measurements have rarely been considered. To fill this gap and gain more insights into the impact of a tweet ́s features on perceived credibility, we conducted a survey asking participants (N=2626) to rate the credibility of crisis-related tweets. The resulting 24.823 ratings were used for an explorative feature selection analysis revealing that mostly meta-related features like the number of followers of the author, the count of tweets produced and the ratio of tweet number and days since account creation affect credibility judgments.

[1]  Mike Schmierbach,et al.  A Little Bird Told Me, So I Didn't Believe It: Twitter, Credibility, and Issue Perceptions , 2012 .

[2]  Caroline M. Eastman,et al.  Automatic evaluation of credibility on the Web , 2006, ASIST.

[3]  Sven G. Bilen,et al.  Increasing the veracity of event detection on social media networks through user trust modeling , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[4]  Xiuzhen Zhang,et al.  User Perception of Information Credibility of News on Twitter , 2014, ECIR.

[5]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[6]  Annie Lang,et al.  The limited capacity model of mediated message processing , 2000 .

[7]  Albert C. Gunther,et al.  Broad Reach or Biased Source? Decomposing the Hostile Media Effect , 2006 .

[8]  Wonchan Choi,et al.  Web credibility assessment: Conceptualization, operationalization, variability, and models , 2015, J. Assoc. Inf. Sci. Technol..

[9]  Heng Ji,et al.  Tweet, but verify: epistemic study of information verification on Twitter , 2013, Social Network Analysis and Mining.

[10]  David Elsweiler,et al.  Manipulating the Perception of Credibility in Refugee Related Social Media Posts , 2017, CHIIR.

[11]  Subbarao Kambhampati,et al.  RAProp: ranking tweets by exploiting the tweet/user/web ecosystem and inter-tweet agreement , 2013, AAAI.

[12]  Albert C. Gunther,et al.  Mapping Boundaries of the Hostile Media Effect , 2004 .

[13]  Scott Counts,et al.  Tweeting is believing?: understanding microblog credibility perceptions , 2012, CSCW.

[14]  Fiona Kayleen Bryer,et al.  Use of Data Collapsing Strategies to Identify Latent Variables in CHP Questionnaire Data , 2005 .

[15]  Miriam J. Metzger,et al.  Social and Heuristic Approaches to Credibility Evaluation Online , 2010 .

[16]  Stefan Stieglitz,et al.  Social media analytics - Challenges in topic discovery, data collection, and data preparation , 2018, Int. J. Inf. Manag..

[17]  S. Sundar The MAIN Model : A Heuristic Approach to Understanding Technology Effects on Credibility , 2007 .

[18]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[19]  S. Chaiken The heuristic model of persuasion. , 1987 .

[20]  Asher Levi,et al.  The Social Aspect of Voting for Useful Reviews , 2014, SBP.

[21]  Arkaitz Zubiaga,et al.  PHEME dataset of rumours and non-rumours , 2016 .

[22]  Stefan Stieglitz,et al.  Communication Roles in Public Events - A Case Study on Twitter Communication , 2014, IS&O.

[23]  S. Shyam Sundar,et al.  Authority vs. peer: how interface cues influence users , 2009, CHI Extended Abstracts.

[24]  Barbara Poblete,et al.  Twitter under crisis: can we trust what we RT? , 2010, SOMA '10.

[25]  B. AlDoaies,et al.  Exploring and Evaluating the Impact of the Veracity of Big Data Sources , 2017 .

[26]  M. Gentzkow,et al.  Social Media and Fake News in the 2016 Election , 2017 .

[27]  Hoi K. Suen,et al.  The Role of MTurk in Education Research: Advantages, Issues, and Future Directions , 2017 .

[28]  Arkaitz Zubiaga,et al.  SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours , 2017, *SEMEVAL.

[29]  Duncan J. Watts,et al.  Everyone's an influencer: quantifying influence on twitter , 2011, WSDM '11.

[30]  Liang Zhao,et al.  A topic-focused trust model for Twitter , 2016, Comput. Commun..

[31]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[32]  John T. Cacioppo,et al.  The Elaboration Likelihood Model of Persuasion , 1986, Advances in Experimental Social Psychology.

[33]  Barbara Poblete,et al.  Predicting information credibility in time-sensitive social media , 2013, Internet Res..

[34]  Eun-Ju Lee,et al.  Are They Talking to Me? Cognitive and Affective Effects of Interactivity in Politicians' Twitter Communication , 2012, Cyberpsychology Behav. Soc. Netw..

[35]  Panagiotis G. Ipeirotis,et al.  Get another label? improving data quality and data mining using multiple, noisy labelers , 2008, KDD.

[36]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[37]  C. I. Hovland,et al.  The Influence of Source Credibility on Communication Effectiveness , 1951 .

[38]  Eun-Ju Lee,et al.  Not So Imaginary Interpersonal Contact With Public Figures on Social Network Sites , 2013, Commun. Res..

[39]  Huan Liu,et al.  Chi2: feature selection and discretization of numeric attributes , 1995, Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence.

[40]  Xiuzhen Zhang,et al.  On the credibility perception of news on Twitter: Readers, topics and features , 2017, Comput. Hum. Behav..

[41]  Arkaitz Zubiaga,et al.  Detection and Resolution of Rumours in Social Media , 2017, ACM Comput. Surv..

[42]  J. Mccroskey,et al.  Goodwill: A reexamination of the construct and its measurement , 1999 .

[43]  W. Beamish Consensus about Program Quality: An Australian Study in Early Childhood Special Education , 2008 .

[44]  Arkaitz Zubiaga,et al.  Stance Classification in Out-of-Domain Rumours: A Case Study Around Mental Health Disorders , 2017, SocInfo.

[45]  Toma Bedolla,et al.  Credibility of Sources and the Veracity of Content , 2012 .

[46]  Miriam J. Metzger,et al.  Psychological Approaches to Credibility Assessment Online , 2015 .

[47]  Nicole C. Krämer,et al.  A question of credibility – Effects of source cues and recommendations on information selection on news sites and blogs , 2014 .

[48]  Suhang Wang,et al.  Fake News Detection on Social Media: A Data Mining Perspective , 2017, SKDD.

[49]  G. Gigerenzer,et al.  Intuitive and Deliberate Judgments Are Based on Common Principles This Article Has Been Corrected. See Last Page , 2022 .

[50]  S. Sundar,et al.  Measuring Message Credibility , 2016 .

[51]  Marco Mellia,et al.  Crowdsourced network measurements: Benefits and best practices , 2015, Comput. Networks.

[52]  Mark Dredze,et al.  Annotating Named Entities in Twitter Data with Crowdsourcing , 2010, Mturk@HLT-NAACL.

[53]  Barbara Poblete,et al.  Information credibility on twitter , 2011, WWW.

[54]  Karl Aberer,et al.  Web Credibility: Features Exploration and Credibility Prediction , 2013, ECIR.

[55]  L. Bode Political News in the News Feed: Learning Politics from Social Media , 2016 .