Citation networks of related trials are often disconnected: implications for bidirectional citation searches.

BACKGROUND AND OBJECTIVES Reports of randomized controlled trials (RCTs) should set findings within the context of previous research. The resulting network of citations would also provide an alternative search method for clinicians, researchers, and systematic reviewers seeking to base decisions on all available evidence. We sought to determine the connectedness of citation networks of RCTs by examining direct (referenced trials) and indirect (through references of referenced trials, etc) citation of trials to one another. METHODS Meta-analyses were used to create citation networks of RCTs addressing the same clinical questions. The primary measure was the proportion of networks where following citation links between RCTs identifies the complete set of RCTs, forming a single connected citation group. Other measures included the number of disconnected groups (islands) within each network, the number of citations in the network relative to the maximum possible, and the maximum number of links in the path between two connected trials (a measure of indirectness of citations). RESULTS We included 259 meta-analyses with a total of 2,413 and a median of seven RCTs each. For 46% (118 of 259) of networks, the RCTs formed a single connected citation group-one island. For the other 54% of networks, where at least one RCT group was not cited by others, 39% had two citation islands and 4% (10 of 257) had 10 or more islands. On average, the citation networks had 38% of the possible citations to other trials (if each trial had cited all earlier trials). The number of citation islands and the maximum number of citation links increased with increasing numbers of trials in the network. CONCLUSION Available evidence to answer a clinical question may be identified by using network citations created with a small initial corpus of eligible trials. However, the number of islands means that citation networks cannot be relied on for evidence retrieval.

[1]  K Ann McKibbon,et al.  The capture-mark-recapture technique can be used as a stopping rule when searching in systematic reviews. , 2009, Journal of clinical epidemiology.

[2]  P. Gøtzsche.  Reference bias in reports of drug trials. , 1987, British medical journal.

[3]  R. Wears,et al.  Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. , 2002, JAMA.

[4]  Leonard Leibovici,et al.  β lactam monotherapy versus β lactam-aminoglycoside combination therapy for sepsis in immunocompetent patients: systematic review and meta-analysis of randomised trials , 2004, BMJ : British Medical Journal.

[5]  C. Gluud,et al.  Citation bias of hepato-biliary randomized clinical trials. , 2002, Journal of clinical epidemiology.

[6]  Richard Gray,et al.  Monoamine oxidase type B inhibitors in early Parkinson's disease: meta-analysis of 17 randomised trials involving 3525 patients , 2004, BMJ : British Medical Journal.

[7]  Steven A Greenberg,et al.  How citation distortions create unfounded authority: analysis of a citation network , 2009, BMJ : British Medical Journal.

[8]  Sally Hopewell,et al.  Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting , 2010, The Lancet.

[9]  P. Deltenre,et al.  Evaluation of amantadine in chronic hepatitis C: a meta-analysis. , 2004, Journal of hepatology.

[10]  T. Greenhalgh,et al.  Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources , 2005, BMJ : British Medical Journal.

[11]  U. Ravnskov,et al.  Cholesterol lowering trials in coronary heart disease: frequency of citation and outcome. , 1992, BMJ.

[12]  H. Stelfox,et al.  Capture-mark-recapture as a tool for estimating the number of articles available for systematic reviews in critical care medicine. , 2013, Journal of critical care.

[13]  T Treasure,et al.  Clinical reports of pulmonary metastasectomy for colorectal cancer: a citation network analysis , 2011, British Journal of Cancer.

[14]  Gordon Parker,et al.  Can the Highly Cited Psychiatric Paper be Predicted Early? , 2009, The Australian and New Zealand journal of psychiatry.

[15]  A. Kulkarni,et al.  Characteristics Associated with Citation Rate of the Medical Literature , 2007, PloS one.

[16]  M. Swiontkowski,et al.  Factors associated with citation rates in the orthopedic literature. , 2007, Canadian journal of surgery. Journal canadien de chirurgie.

[17]  D. de Zeeuw,et al.  The effect of metformin on blood pressure, plasma cholesterol and triglycerides in type 2 diabetes mellitus: a systematic review. , 2002, Journal of internal medicine.

[18]  L. Leibovici,et al.  Beta lactam monotherapy versus beta lactam-aminoglycoside combination therapy for sepsis in immunocompetent patients: systematic review and meta-analysis of randomised trials. , 2004, BMJ.

[19]  R. Gansevoort,et al.  The effect of metformin on blood pressure, plasma cholesterol and triglycerides in type 2 diabetes mellitus: a systematic review , 2004 .

[20]  M. Sampson,et al.  Checking reference lists to find additional studies for systematic reviews. , 2011, The Cochrane database of systematic reviews.

[21]  A. Kazanjian,et al.  Evidence-based practice: extending the search to find material for the systematic review. , 2001, Bulletin of the Medical Library Association.

[22]  S. Goodman,et al.  A Systematic Examination of the Citation of Prior Research in Reports of Randomized, Controlled Trials , 2011, Annals of Internal Medicine.

[23]  Martin Schumacher,et al.  The relationship between quality of research and citation frequency , 2006, BMC medical research methodology.

[24]  A. Kazanjian,et al.  BEYOND MEDLINE , 2003, International Journal of Technology Assessment in Health Care.

[25]  K. A. McKibbon,et al.  Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study , 2008, BMJ : British Medical Journal.

[26]  F M Campbell,et al.  National bias: a comparison of citation practices by health professionals. , 1990, Bulletin of the Medical Library Association.

[27]  Enrico Coiera,et al.  The automation of systematic reviews , 2013, BMJ.