Systematic literature searching in policy relevant, inter-disciplinary reviews: an example from culture and sport.

Within the systematic review process, the searching phase is critical to the final synthesis product, its use and value. Yet, relatively little is known about the utility of different search strategies for reviews of complex, inter-disciplinary evidence. This article used a recently completed programme of work on cultural and sporting engagement to conduct an empirical evaluation of a comprehensive search strategy. Ten different types of search source were evaluated, according to three dimensions: (i) effectiveness in identifying relevant studies; (ii) efficiency in identifying studies; and (iii) adding value by locating studies that were not identified by any other sources. The study found that general bibliographic databases and specialist databases ranked the highest on all three dimensions. Overall, websites and journals were the next most valuable types of source. For reviewers, these findings highlight that general and specialist databases should remain a core component of the comprehensive search strategy, supplemented with other types of sources that can efficiently identify unique or grey literature. For policy makers and other research commissioners, this study highlights the value of methodological analysis for improving the understanding of, and practice in, policy relevant, inter-disciplinary systematic reviews. Copyright © 2011 John Wiley & Sons, Ltd.

[1]  M. Nasser,et al.  A comparison of handsearching versus embase searching of the archives of Iranian medicine to identify reports of randomized controlled trials. , 2006, Archives of Iranian medicine.

[2]  J. Orme,et al.  The Impact of Participation in Performing Arts on Adolescent Health and Behaviour , 2008, Journal of health psychology.

[3]  I. Walker Hall,et al.  Medical Library Association , 1910 .

[4]  J. Schuster,et al.  Participation Studies and Cross-National Comparison: Proliferation, Prudence, and Possibility , 2007 .

[5]  Su Golder,et al.  Systematic searches for the effectiveness of respite care. , 2008, Journal of the Medical Library Association : JMLA.

[6]  Jodie Doyle,et al.  It's in your hands: the value of handsearching in conducting systematic reviews of public health interventions. , 2005, Journal of public health.

[7]  Ray Pawson,et al.  Evidence-based policy: a realist perspective. , 2006 .

[8]  David Ogilvie,et al.  Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? , 2005, Journal of Epidemiology and Community Health.

[9]  J. Sterne,et al.  How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. , 2003, Health technology assessment.

[10]  Janice Tripney,et al.  Updating the CASE database: analysis of original approach , 2011 .

[11]  Mark Newman,et al.  Understanding the impact of engagement in culture and sport: a systematic review of the learning impacts for young people , 2010 .

[12]  S. Golder,et al.  The effectiveness and cost-effectiveness of prophylactic removal of wisdom teeth. , 2000, Health technology assessment.

[13]  T. Greenhalgh,et al.  Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources , 2005, BMJ : British Medical Journal.

[14]  M. Dixon-Woods,et al.  Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups , 2006 .

[15]  Pamela Royle,et al.  LITERATURE SEARCHING FOR RANDOMIZED CONTROLLED TRIALS USED IN COCHRANE REVIEWS: RAPID VERSUS EXHAUSTIVE SEARCHES , 2003, International Journal of Technology Assessment in Health Care.

[16]  B. Ireland Health Development Agency , 2005 .

[17]  Kate Flemming,et al.  Electronic searching to locate qualitative research: evaluation of three strategies. , 2007, Journal of advanced nursing.

[18]  Tina Miller,et al.  Finding qualitative research: an evaluation of search strategies , 2004, BMC medical research methodology.

[19]  Steven Allender,et al.  Understanding participation in sport and physical activity among children and adults: a review of qualitative studies. , 2006, Health education research.

[20]  Kate Ashton,et al.  Effectiveness of Search Strategies for Qualitative Research About Barriers and Facilitators of Program Delivery , 2011, Evaluation & the health professions.

[21]  Kay Dickersin,et al.  Problems and Limitations in Conducting Systematic Reviews , 2008 .

[22]  Naomi Priest,et al.  Policy interventions implemented through sporting organisations for promoting healthy behaviour change. , 2008, The Cochrane database of systematic reviews.

[23]  S. Galloway Theory-based evaluation and the social impact of the arts , 2009 .

[24]  F. Coalter,et al.  The role of sport in regenerating deprived urban areas. , 2000 .

[25]  Alex J. Sutton,et al.  Publication and related biases: a review , 2000 .

[26]  Michael Donnelly,et al.  Systematically Retrieving Research: A Case Study Evaluating Seven Databases , 2007 .

[27]  F. Coalter THE ROLE OF SPORT IN REGENERATING DEPRIVED AREAS , 2000 .

[28]  A. Booth,et al.  Literature searching for social science systematic reviews: consideration of a range of search techniques. , 2009, Health information and libraries journal.

[29]  Z. Fedorowicz,et al.  A comparison of handsearching versus EMBASE searching of the Annals of Saudi Medicine to identify reports of randomized controlled trials , 2006, Annals of Saudi medicine.

[30]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[31]  Ranjan Das,et al.  Biomedical Research Methodology , 2011 .

[32]  L. Say,et al.  Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality , 2005, BMC medical research methodology.

[33]  J Chilcott,et al.  The role of modelling in prioritising and planning clinical trials. , 2003, Health technology assessment.