Citation behavior: A large‐scale test of the persuasion by name‐dropping hypothesis

Citation frequencies are commonly interpreted as measures of quality or impact. Yet, the true nature of citations and their proper interpretation have been the center of a long, but still unresolved discussion in Bibliometrics. A comparison of 67,578 pairs of studies on the same healthcare topic, with the same publication age (1–15 years) reveals that when one of the studies is being selected for citation, it has on average received about three times as many citations as the other study. However, the average citation‐gap between selected or deselected studies narrows slightly over time, which fits poorly with the name‐dropping interpretation and better with the quality and impact‐interpretation. The results demonstrate that authors in the field of Healthcare tend to cite highly cited documents when they have a choice. This is more likely caused by differences related to quality than differences related to status of the publications cited.

[1]  S. Goodman,et al.  A Systematic Examination of the Citation of Prior Research in Reports of Randomized, Controlled Trials , 2011, Annals of Internal Medicine.

[2]  Harlan M Krumholz,et al.  Articles in Peer-Reviewed Cardiovascular Journals from 1997-2007 : Analysis of 5-Year Citation Rates , 2015 .

[3]  J. P. Flynn,et al.  Twenty-year perspective of randomized controlled trials for surgery of chronic nonspecific low back pain: citation bias and tangential knowledge. , 2013, The spine journal : official journal of the North American Spine Society.

[4]  M. Tramèr,et al.  Ability of a meta-analysis to prevent redundant research: systematic review of studies on pain from propofol injection , 2014, BMJ : British Medical Journal.

[5]  M. Harisinghani,et al.  Metrics for Original Research Articles in the AJR: From First Submission to Final Publication. , 2015, AJR. American journal of roentgenology.

[6]  Francis Narin,et al.  To believe or not to believe , 1987, Scientometrics.

[7]  K. Robinson,et al.  Biased and inadequate citation of prior research in reports of cardiovascular trials is a continuing source of waste in research. , 2016, Journal of clinical epidemiology.

[8]  F. Mosteller,et al.  A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. , 1992, JAMA.

[9]  Walter Ricciardi,et al.  Evaluation of the Endorsement of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Statement on the Quality of Published Systematic Review and Meta-Analyses , 2013, PloS one.

[10]  L. Hooft,et al.  Reporting Quality of Systematic Reviews and Meta-Analyses of Otorhinolaryngologic Articles Based on the PRISMA Statement , 2015, PloS one.

[11]  Alexander J Sutton,et al.  Empirical assessment suggests that existing evidence could be used more fully in designing randomized controlled trials. , 2010, Journal of clinical epidemiology.

[12]  G. Gilbert.  Referencing as Persuasion , 1977 .

[13]  J. Geddes,et al.  Recent meta-analyses neglect previous systematic reviews and meta-analyses about the same topic: a systematic examination , 2015, BMC Medicine.

[14]  Carol Tenopir,et al.  The role of trust and authority in the citation behaviour of researchers , 2015, Inf. Res..

[15]  I Chalmers,et al.  Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? , 1998, JAMA.

[16]  Henry G. Small.  The significance of bibliographic references , 2005, Scientometrics.

[17]  C. Hughes.  Systematic Reviews to Support Evidence‐based Medicine: How to Review and Apply Findings of Healthcare Research. , 2005 .

[18]  Cheri K Walker,et al.  A Descriptive and Historical Review of Bibliometrics with Applications to Medical Sciences , 2015, Pharmacotherapy.

[19]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[20]  Michael H. MacRoberts,et al.  Problems of citation analysis , 1992, Scientometrics.

[21]  Terrence A. Brooks,et al.  Evidence of complex citer motivations , 1986, J. Am. Soc. Inf. Sci..

[22]  Lisa Hartling,et al.  Quality of reporting in systematic reviews of adverse events: systematic review , 2014, BMJ : British Medical Journal.

[23]  Melvin Weinatoek.  Citation Indexes , .

[24]  Harriet Zuckerman,et al.  Citation analysis and the complex problem of intellectual influence , 1987, Scientometrics.

[25]  I. Roberts,et al.  Exploring redundant research into the effect of tranexamic acid on surgical bleeding: further analysis of a systematic review of randomised controlled trials , 2015, BMJ Open.

[26]  Anthony F. J. van Raan,et al.  Citation theory and the Ortega hypothesis , 1987, Scientometrics.

[27]  H. Williams,et al.  Cochrane Skin Group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology , 2006, The British journal of dermatology.

[28]  Andy R Weale,et al.  The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor , 2004, BMC medical research methodology.

[29]  R. Gómez-Huelgas.  [Bariatric surgery versus conventional medical therapy for type 2 diabetes]. , 2012, Revista clinica espanola.

[30]  Brian Hutton,et al.  Randomized controlled trials of aprotinin in cardiac surgery: could clinical equipoise have stopped the bleeding? , 2005, Clinical trials.

[31]  N. Simunovic,et al.  Poor citation of prior evidence in hip fracture trials. , 2011, The Journal of bone and joint surgery. American volume.

[32]  J. Colquitt,et al.  Surgery for weight loss in adults. , 2014, The Cochrane database of systematic reviews.

[33]  G. Patil,et al.  Rejoinder , 2004, Environmental and Ecological Statistics.

[34]  Catherine Sherrington,et al.  Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy. , 2009, Journal of clinical epidemiology.

[35]  Angela Boland,et al.  Doing a Systematic Review. A Student's Guide. Third Edition , 2023, Revue d'Épidémiologie et de Santé Publique.

[36]  J. A. Stewart,et al.  Achievement and Ascriptive Processes in the Recognition of Scientific Articles , 1983 .

[37]  Howard D. White,et al.  Reward, persuasion, and the Sokal Hoax: A study in citation identities , 2004, Scientometrics.

[38]  Howard D. White,et al.  Authors as citers over time , 2001, J. Assoc. Inf. Sci. Technol..

[39]  Mike Clarke,et al.  Accumulating Research: A Systematic Account of How Cumulative Meta-Analyses Would Have Provided Knowledge, Improved Health, Reduced Harm and Saved Resources , 2014, PloS one.

[40]  Guy Tsafnat,et al.  Citation networks of related trials are often disconnected: implications for bidirectional citation searches. , 2014, Journal of clinical epidemiology.

[41]  P. Shekelle,et al.  Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement , 2015, Systematic Reviews.

[42]  Plergiorgio Strata,et al.  Citation analysis , 1995, Nature.

[43]  Michael B. Usher,et al.  Science in action , 1993, Nature.

[44]  Michael F. Winter,et al.  : The Footnote: A Curious History , 1999 .

[45]  Stephen Cole,et al.  Professional Standing and the Reception of Scientific Discoveries , 1970, American Journal of Sociology.

[46]  J. Grimshaw,et al.  When and how to update systematic reviews. , 2008, The Cochrane database of systematic reviews.

[47]  Stephen Cole.  Merton’s Contribution to the Sociology of Science , 2004 .

[48]  Kirstin Borgerson.  Redundant, Secretive, and Isolated: When Are Clinical Trials Scientifically Valid? , 2014, Kennedy Institute of Ethics journal.

[49]  Kène Henkens,et al.  What Makes a Scientific article Influential , 2000 .

[50]  Hendrik P. van Dalen,et al.  What makes a scientific article influential? The case of demographers , 2001, Scientometrics.

[51]  P. Zhang,et al.  Reporting Quality of Systematic Reviews/Meta-Analyses of Acupuncture , 2014, PloS one.

[52]  Terrence A. Brooks,et al.  Private acts and public objects: An investigation of citer motivations , 1985, J. Am. Soc. Inf. Sci..

[53]  R. Merton.  The Matthew Effect in Science , 1968, Science.

[54]  Michael H. MacRoberts,et al.  Quantitative Measures of Communication in Science: A Study of the Formal Level , 1986 .