The citation advantage of promoted articles in a cross‐publisher distribution platform: A 12‐month randomized controlled trial

There is currently a paucity of evidence‐based strategies that have been shown to increase citations of peer‐reviewed articles following their publication. We conducted a 12‐month randomized controlled trial to examine whether the promotion of article links in an online cross‐publisher distribution platform (TrendMD) affects citations. In all, 3,200 articles published in 64 peer‐reviewed journals across eight subject areas were block randomized at the subject level to either the TrendMD group (n = 1,600) or the control group (n = 1,600) of the study. Our primary outcome compares the mean citations of articles randomized to TrendMD versus control after 12 months. Articles randomized to TrendMD showed a 50% increase in mean citations relative to control at 12 months. The difference in mean citations at 12 months for articles randomized to TrendMD versus control was 5.06, 95% confidence interval [2.87, 7.25], was statistically significant (p < .001) and found in three of eight subject areas. At 6 months following publication, articles randomized to TrendMD showed a smaller, yet statistically significant (p = .005), 21% increase in mean citations, relative to control. To our knowledge, this is the first randomized controlled trial to demonstrate how an intervention can be used to increase citations of peer‐reviewed articles after they have been published.

[1]  Arun Ravindran,et al.  Online distribution channel increases article usage on Mendeley: a randomized controlled trial , 2017, Scientometrics.

[2]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[3]  Casey S Greene,et al.  Sci-Hub provides access to nearly all scholarly literature , 2018, eLife.

[4]  Rajeev Kumar,et al.  Sample size calculation , 2012, Indian journal of ophthalmology.

[5]  Rick Anderson Is rational discussion of open access possible , 2014 .

[6]  Gunther Eysenbach,et al.  TrendMD: Reaching Larger, More Targeted Audiences by Distributing Scholarly Content Online , 2015 .

[7]  Philip M. Davis,et al.  The impact of free access to the scientific literature: a review of recent research. , 2011, Journal of the Medical Library Association : JMLA.

[8]  S. Lawrence Free online availability substantially increases a paper's impact , 2001, Nature.

[9]  Mike Thelwall,et al.  Could scientists use Altmetric.com scores to predict longer term citation counts? , 2018, J. Informetrics.

[10]  Éric Archambault,et al.  Research impact of paywalled versus open access papers , 2016 .

[11]  Antonio Abbate,et al.  Letter by Dixon et al Regarding Article, "Watching Television and Risk of Mortality From Pulmonary Embolism Among Japanese Men and Women: The JACC Study (Japan Collaborative Cohort)". , 2016, Circulation.

[12]  Stephen S. Murray,et al.  The effect of use and access on citations , 2005, Inf. Process. Manag..

[13]  Gunther Eysenbach,et al.  TrendMD: Helping scholarly content providers reach larger and more targeted audiences , 2016, Learn. Publ..

[14]  Joseph Loscalzo,et al.  Randomized Controlled Trial of Social Media: Effect of Increased Intensity of the Intervention , 2016, Journal of the American Heart Association.

[15]  Frank Gaillard,et al.  Letter by Dixon et al regarding article, "A randomized trial of social media from Circulation". , 2015, Circulation.

[16]  Mike Thelwall,et al.  Early Mendeley readers correlate with later citation counts , 2018, Scientometrics.

[17]  João Biehl,et al.  Life of the mind: The interface of psychopharmaceuticals, domestic economies, and social abandonment , 2004 .

[18]  Brent Thoma,et al.  Impact of a Physician-Led Social Media Sharing Program on a Medical Journal's Web Traffic. , 2018, Journal of the American College of Radiology : JACR.

[19]  Bo-Christer Björk,et al.  Delayed open access: An overlooked high-impact category of openly available scientific literature , 2013, J. Assoc. Inf. Sci. Technol..

[20]  Zhe Feng,et al.  A general introduction to adjustment for multiple comparisons. , 2017, Journal of thoracic disease.

[21]  Charles Oppenheim,et al.  The citation advantage of open-access articles , 2008, J. Assoc. Inf. Sci. Technol..

[22]  Mike Thelwall,et al.  Are Mendeley reader counts useful impact indicators in all fields? , 2017, Scientometrics.

[23]  Koler-PovhTeja,et al.  Impact of open access on citation of scholarly publications in the field of civil engineering , 2014 .

[24]  Susanne Mikki Scholarly publications beyond pay-walls: increased citation advantage for open publishing , 2017, Scientometrics.

[25]  C Matthew Hawkins,et al.  Social Media and Peer-Reviewed Medical Journal Readership: A Randomized Prospective Controlled Trial. , 2017, Journal of the American College of Radiology : JACR.

[26]  Htet Htet Aung,et al.  Analysing researchers’ outreach efforts and the association with publication metrics: A case study of Kudos , 2017, PloS one.

[27]  Xianwen Wang,et al.  The open access advantage considering citation, article usage and social media attention , 2015, Scientometrics.

[28]  Christopher M. Snyder,et al.  Identifying the Effect of Open Access on Citations Using a Panel of Science Journals , 2013 .

[29]  H. White A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity , 1980 .

[30]  The science that’s never been cited , 2018, Nature.

[31]  Carol Tenopir,et al.  Seeking, Reading, and Use of Scholarly Articles: An International Study of Perceptions and Behavior of Researchers , 2019, Publ..

[32]  Richard Van Noorden Scientists may be reaching a peak in reading habits , 2014 .

[33]  Mike Thelwall,et al.  Do Mendeley reader counts reflect the scholarly impact of conference papers? An investigation of computer science and engineering , 2017, Scientometrics.

[34]  Michelle Lin,et al.  Social media in the emergency medicine residency curriculum: social media responses to the residents' perspective article. , 2015, Annals of emergency medicine.

[35]  Philip M. Davis,et al.  Open access, readership, citations: a randomized controlled trial of scientific journal publishing , 2011, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[36]  Joseph Loscalzo,et al.  A Randomized Trial of Social Media From Circulation , 2015, Circulation.

[37]  Mike Thelwall,et al.  Mendeley readership altmetrics for medical articles: An analysis of 45 fields , 2016, J. Assoc. Inf. Sci. Technol..

[38]  Teja Koler-Povh,et al.  Impact of open access on citation of scholarly publications in the field of civil engineering , 2013, Scientometrics.

[39]  M. Wacha,et al.  The State of OA: A large-scale analysis of the prevalence and impact of Open Access articles , 2017 .

[40]  M. Thelwall,et al.  F 1000 , Mendeley and Traditional Bibliometric Indicators , 2012 .

[41]  M. D. de Boer,et al.  Testing for baseline differences in randomized controlled trials: an unhealthy research behavior that is hard to eradicate , 2015, International Journal of Behavioral Nutrition and Physical Activity.

[42]  Jim Ottaviani The Post-Embargo Open Access Citation Advantage: It Exists (Probably), It’s Modest (Usually), and the Rich Get Richer (of Course) , 2016, PloS one.

[43]  Philip M. Davis,et al.  Open access publishing, article downloads, and citations: randomised controlled trial , 2008, BMJ : British Medical Journal.

[44]  B. Björk,et al.  Open access versus subscription journals: a comparison of scientific impact , 2012, BMC Medicine.

[45]  Jafar Mehrad,et al.  Path analysis of the relationship between visibility and citation: the mediating roles of save, discussion, and recommendation metrics , 2016, Scientometrics.

[46]  Yuri Niyazov,et al.  Open Access Meets Discoverability: Citations to Articles Posted to Academia.edu , 2016, PloS one.

[47]  A. Sockloff,et al.  Statistical power analysis for the behavioral sciences: (revised edition), by Jacob Cohen. New York: Academic Press, 1977, xv + 474 pp., $24.50. , 1978 .

[48]  Mike Thelwall,et al.  Mendeley readership counts: An investigation of temporal and disciplinary differences , 2016, J. Assoc. Inf. Sci. Technol..

[49]  G. Franck Open access , 2012, Cell cycle.