Which people use which scientific papers? An evaluation of data from F1000 and Mendeley

The increased interest in an impact measurement of research on other areas of the society than research has led in scientometrics to an investigation of altmetrics. Particular attention is paid here to a targeted broad impact measurement: The aim is to discover the impact which a particular publication set has on specific user groups (outside research) by using altmetrics. This study used the Mendeley application programming interface (API) to download the Mendeley counts (broken down by different user types of publications in Mendeley) for a comprehensive F1000Prime data set. F1000Prime is a post-publication peer review system for papers from the biomedical area. As the F1000 papers are provided with tags from experts in this area (Faculty members) which can characterise a paper more exactly (such as “good for teaching” or “new finding”), the interest of different user groups in specifically tagged papers could be investigated. This study's evaluation of the variously tagged F1000 papers provided interesting insights into the use of research papers by different user groups. The most interesting tag for altmetrics research is “good for teaching”. This applies to papers which are well written and provide an overview of a topic. Papers with this tag can be expected to arouse interest among people who are hardly or not at all involved in research. The results of the regression models in this study do in fact show that lecturers, researchers at a non-academic institution, and others (such as librarians) have a special interest in this kind of papers. In the case of a key article in a field, or a particularly well written article that provides a good overview of a topic, then it will tend to be better received by people which are not particularly related to academic research.

[1]  Mike Thelwall,et al.  Validating online reference managers for scholarly impact measurement , 2011, Scientometrics.

[2]  Mike Thelwall,et al.  Not all international collaboration is beneficial: The Mendeley readership and citation impact of biochemical research collaboration , 2016, J. Assoc. Inf. Sci. Technol..

[3]  Lutz Bornmann,et al.  Usefulness of altmetrics for measuring the broader impact of research: A case study using data from PLOS and F1000Prime , 2014, Aslib J. Inf. Manag..

[4]  Frauke Kreuter,et al.  Data Analysis Using Stata , 2005 .

[5]  Cameron Neylon,et al.  Rethinking Impact: Applying Altmetrics to Southern African Research , 2014 .

[6]  Katrin Weller,et al.  Citations in Web 2.0 , 2012 .

[7]  Mike Thelwall,et al.  Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics , 2014, J. Assoc. Inf. Sci. Technol..

[8]  Lutz Bornmann,et al.  How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects , 2013, J. Informetrics.

[9]  Mike Thelwall,et al.  Can Mendeley bookmarks reflect readership? A survey of user motivations , 2016, J. Assoc. Inf. Sci. Technol..

[10]  Rodrigo Costas,et al.  Assessing the Impact of Publications Saved by Mendeley Users: Is There Any Different Pattern Among Users? , 2014 .

[11]  Vincent Larivière,et al.  Tweets vs. Mendeley readers: How do these two social media metrics differ? , 2014, it Inf. Technol..

[12]  Lutz Bornmann,et al.  Interrater reliability and convergent validity of F1000Prime peer review , 2014, J. Assoc. Inf. Sci. Technol..

[13]  Gabriel Kreiman,et al.  Nine Criteria for a Measure of Scientific Output , 2011, Front. Comput. Neurosci..

[14]  Rodrigo Costas,et al.  Users, narcissism and control – tracking the impact of scholarly publications in the 21st century , 2012 .

[15]  W. Glänzel Seven Myths in Bibliometrics About facts and fiction in quantitative science studies , 2008 .

[16]  Judit Bar-Ilan,et al.  Coverage and adoption of altmetrics sources in the bibliometric community , 2014, Scientometrics.

[17]  Vincent Larivière,et al.  A multidimensional analysis of Aslib proceedings - using everything but the impact factor , 2014, Aslib J. Inf. Manag..

[18]  Lutz Bornmann,et al.  What do citation counts measure? A review of studies on citing behavior , 2008, J. Documentation.

[19]  Cassidy R. Sugimoto,et al.  Bibliographic References in Web 2.0 , 2014 .

[20]  Lutz Bornmann,et al.  The substantive and practical significance of citation impact differences between institutions: Guidelines for the analysis of percentiles using effect sizes and confidence intervals , 2014, ArXiv.

[21]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .

[22]  Lutz Bornmann,et al.  What is societal impact of research and how can it be assessed? a literature survey , 2013, J. Assoc. Inf. Sci. Technol..

[23]  Rodrigo Costas,et al.  Interpreting "altmetrics": viewing acts on social media through the lens of citation and social theories , 2015, ArXiv.

[24]  L. Bornmann,et al.  F1000Prime: an analysis of discipline-specific reader data from Mendeley , 2015 .

[25]  Sarah Barbrow,et al.  A Look at Altmetrics and Its Growing Significance to Research Libraries , 2013 .

[26]  Mike Thelwall,et al.  Are scholarly articles disproportionately read in their own country? An analysis of mendeley readers , 2015, J. Assoc. Inf. Sci. Technol..

[27]  Lutz Bornmann Letter to the Editor: On the conceptualisation and theorisation of the impact caused by publications , 2015, Scientometrics.

[28]  Rodrigo Costas,et al.  How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications , 2014, Scientometrics.

[29]  Lutz Bornmann,et al.  Measuring the societal impact of research , 2012, EMBO reports.

[30]  Lutz Bornmann,et al.  Validity of altmetrics data for measuring societal impact: A study using data from Altmetric and F1000Prime , 2014, J. Informetrics.

[31]  Lutz Bornmann Usefulness of altmetrics for measuring the broader impact of research: A case study using data from PLOS (altmetrics) and F1000Prime (paper tags) , 2014 .

[32]  Lutz Bornmann,et al.  Alternative metrics in scientometrics: a meta-analysis of research into three altmetrics , 2015, Scientometrics.

[33]  Vincent Larivière,et al.  Who reads research articles? An altmetrics analysis of Mendeley user categories , 2015, J. Assoc. Inf. Sci. Technol..

[34]  Richard Williams,et al.  Using the Margins Command to Estimate and Interpret Adjusted Predictions and Marginal Effects , 2012 .