What does Twitter Measure?: Influence of Diverse User Groups in Altmetrics

The most important goal for digital libraries is to ensure high quality search experience for all kinds of users. To attain this goal, it is necessary to have as much relevant metadata as possible at hand to assess the quality of publications. Recently, a new group of metrics appeared, that has the potential to raise the quality of publication metadata to the next level -- the altmetrics. These metrics try to reflect the impact of publications within the social web. However, currently it is still unclear if and how altmetrics should be used to assess the quality of a publication and how altmetrics are related to classical bibliographical metrics (like e.g. citations). To gain more insights about what kind of concepts are reflected by altmetrics, we conducted an in-depth analysis on a real world dataset crawled from the Public Library of Science (PLOS). Especially, we analyzed if the common approach to regard the users in the social web as one homogeneous group is sensible or if users need to be divided into diverse groups in order to receive meaningful results.

[1]  Blaise Cronin,et al.  Metrics à la mode , 2013, J. Assoc. Inf. Sci. Technol..

[2]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[3]  Blaise Cronin Editor-in-Chief,et al.  Metrics à la mode , 2013 .

[4]  Ian H. Witten,et al.  An open-source toolkit for mining Wikipedia , 2013, Artif. Intell..

[5]  Mike Thelwall,et al.  Scholars on soap boxes: Science communication and dissemination in TED videos , 2013, J. Assoc. Inf. Sci. Technol..

[6]  Rodrigo Costas,et al.  Users, narcissism and control – tracking the impact of scholarly publications in the 21st century , 2012 .

[7]  Mike Taylor,et al.  Exploring the Boundaries: How Altmetrics Can Expand Our Vision of Scholarly Communication and Social Impact , 2013 .

[8]  Johan Bollen,et al.  How the Scientific Community Reacts to Newly Submitted Preprints: Article Downloads, Twitter Mentions, and Citations , 2012, PloS one.

[9]  C. Neylon,et al.  Article-Level Metrics and the Evolution of Scientific Impact , 2009, PLoS biology.

[10]  Dario Taraborelli,et al.  Soft peer review. Social software and distributed scientific evaluation , 2008, COOP.

[11]  Cassidy R. Sugimoto,et al.  Do Altmetrics Work? Twitter and Ten Other Social Web Services , 2013, PloS one.

[12]  Vincent Larivière,et al.  Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature , 2013, J. Assoc. Inf. Sci. Technol..

[13]  Paul Groth,et al.  Studying Scientific Discourse on the Web Using Bibliometrics: A Chemistry Blogging Case Study , 2010 .

[14]  Priem Jason,et al.  Prevalence and use of Twitter among scholars , 2012 .

[15]  Rodrigo Costas,et al.  Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective , 2014, J. Assoc. Inf. Sci. Technol..

[16]  M. Fenner What Can Article-Level Metrics Do for You? , 2013, PLoS biology.

[17]  Mike Thelwall,et al.  Validating online reference managers for scholarly impact measurement , 2011, Scientometrics.

[18]  Patrick Siehndel,et al.  TwikiMe! - User Profiles That Make Sense , 2012, International Semantic Web Conference.

[19]  Krishna P. Gummadi,et al.  Cognos: crowdsourcing search for topic experts in microblogs , 2012, SIGIR '12.

[20]  David Carmel,et al.  Mining expertise and interests from social media , 2013, WWW.

[21]  F. Menczer,et al.  Scholarometer: A Social Framework for Analyzing Impact across Disciplines , 2012, PloS one.

[22]  Emily S. Darling,et al.  The role of twitter in the life cycle of a scientific publication , 2013, ArXiv.

[23]  M. Gerstein,et al.  The Spread of Scientific Information: Insights from the Web Usage Statistics in PLoS Article-Level Metrics , 2011, PloS one.

[24]  Stevan Harnad,et al.  Earlier Web Usage Statistics as Predictors of Later Citation Impact , 2005, J. Assoc. Inf. Sci. Technol..

[25]  Bradley M. Hemminger,et al.  Scientometrics 2.0: New metrics of scholarly impact on the social Web , 2010, First Monday.

[26]  Tim Brody,et al.  Earlier Web usage statistics as predictors of later citation impact: Research Articles , 2006 .

[27]  A. Pritchard,et al.  Statistical bibliography or bibliometrics , 1969 .

[28]  William Gunn,et al.  Social Signals Reflect Academic Impact: What it Means When a Scholar Adds a Paper to Mendeley , 2013 .

[29]  Judit Bar-Ilan,et al.  Beyond citations: Scholars' visibility on the social Web , 2012, ArXiv.

[30]  Katrin Weller,et al.  Citation Analysis in Twitter: Approaches for Defining and Measuring Information Flows within Tweets during Scientific Conferences , 2011, #MSM.

[31]  Robin Chin Roemer,et al.  Institutional Altmetrics and Academic Libraries , 2013 .

[32]  Heather A. Piwowar,et al.  Altmetrics: Value all research products , 2013, Nature.

[33]  Jean Liu Data Curator Five challenges in altmetrics: A toolmaker's perspective , 2013 .

[34]  Jean Liu,et al.  Five challenges in altmetrics: A toolmaker's perspective , 2013 .

[35]  Gunther Eysenbach Correction: Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact , 2012, Journal of Medical Internet Research.

[36]  Gunther Eysenbach,et al.  Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact , 2011, Journal of medical Internet research.

[37]  Jason Priem,et al.  How and why scholars cite on Twitter , 2010, ASIST.