Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy.

OBJECTIVE To describe the quality and methods of systematic reviews of physiotherapy interventions, compare Cochrane and non-Cochrane reviews, and establish the interrater reliability of the Overview Quality Assessment Questionnaire (OQAQ) quality assessment tool. STUDY DESIGN AND SETTING A survey of 200 published systematic reviews was done. Two independent raters assessed the search strategy, assessment of trial quality, outcomes, pooling, conclusions, and overall quality (OQAQ). The study was carried out in the University research center. RESULTS In these reviews, the five most common databases searched were MEDLINE, EMBASE, Cochrane Library, CINAHL, and Cochrane Review Group Registers. The Cochrane allocation concealment system and Jadad Scale were most frequently used to assess trial quality. Cochrane reviews searched more databases and were more likely to assess trial quality, report dichotomous outcomes for individual trials, and conduct a meta-analysis than non-Cochrane reviews. Non-Cochrane reviews were more likely to conclude that there was a beneficial effect of treatment. Cochrane reviews were of higher quality than non-Cochrane reviews. There has been an increase in the quality of systematic reviews over time. The OQAQ has fair to good interrater reliability. CONCLUSION The quality of systematic reviews in physiotherapy is improving, and the use of Cochrane Collaboration procedures appears to improve the methods and quality.

[1]  R. Deyo.  Conservative therapy for low back pain. Distinguishing useful from useless therapy. , 1983, JAMA.

[2]  E. Antman,et al.  ACC/AHA 2002 guideline update for exercise testing: summary article. A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Update the 1997 Exercise Testing Guidelines). , 2002, Journal of the American College of Cardiology.

[3]  D. Grossman,et al.  Effectiveness of health promotion programs to increase motor vehicle occupant restraint use among young children. , 1999, American journal of preventive medicine.

[4]  D. Moher,et al.  Assessing the quality of reports of systematic reviews in pediatric complementary and alternative medicine , 2002, BMC pediatrics.

[5]  Critical appraisal of continuing medical education in the rheumatic diseases for primary care physicians. , 1995, Arthritis and rheumatism.

[6]  M. Dorgan,et al.  Evaluating the quality of systematic reviews in the emergency medicine literature. , 2001, Annals of emergency medicine.

[7]  C. Mulrow,et al.  St John's wort for depression—an overview and meta-analysis of randomised clinical trials , 1996, BMJ.

[8]  M. Hovell.  The experimental evidence for weight-loss treatment of essential hypertension: a critical review. , 1982, American journal of public health.

[9]  Peter C Gøtzsche,et al.  Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review , 2006, BMJ : British Medical Journal.

[10]  J. Kleijnen,et al.  Acupuncture and chronic pain: a criteria-based meta-analysis. , 1990, Journal of clinical epidemiology.

[11]  I. Olkin,et al.  Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement , 1999, The Lancet.

[12]  J. Sterne,et al.  How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. , 2003, Health technology assessment.

[13]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[14]  L M Bouter,et al.  Method guidelines for systematic reviews in the Cochrane Collaboration Back Review Group for Spinal Disorders. , 1997, Spine.

[15]  S. Thacker,et al.  The prevention of shin splints in sports: a systematic review of literature. , 2002, Medicine and science in sports and exercise.

[16]  Douglas G Altman,et al.  Systematic reviews in health care: Assessing the quality of controlled clinical trials. , 2001, BMJ.

[17]  A D Oxman,et al.  Systematic Reviews: Checklists for review articles , 1994, BMJ.

[18]  Trisha Greenhalgh,et al.  Evidence Based Health Care Workbook , 1999 .

[19]  Anthony Delaney,et al.  A systematic evaluation of the quality of meta-analyses in the critical care literature , 2005, Critical care.

[20]  G. Guyatt,et al.  Users' guides to the medical literature. , 1993, JAMA.

[21]  Catherine Sherrington,et al.  Reliability of the PEDro scale for rating quality of randomized controlled trials. , 2003, Physical therapy.

[22]  J. Fleiss.  The design and analysis of clinical experiments , 1987 .

[23]  R. Brian Haynes,et al.  Compliance in Health Care , 1979 .

[24]  A R Jadad,et al.  Assessing the quality of reports of randomized clinical trials: is blinding necessary? , 1996, Controlled clinical trials.

[25]  G H Guyatt,et al.  Respiratory muscle training in chronic airflow limitation: a meta-analysis. , 1992, The American review of respiratory disease.

[26]  R. J. Hayes,et al.  Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. , 1995, JAMA.

[27]  P. Tugwell,et al.  Scope for improvement in the quality of reporting of systematic reviews. From the Cochrane Musculoskeletal Group. , 2006, The Journal of rheumatology.

[28]  H. Vet,et al.  The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. , 1998, Journal of clinical epidemiology.

[29]  Joy C MacDermid,et al.  An introduction to evidence-based practice for hand therapists. , 2004, Journal of hand therapy : official journal of the American Society of Hand Therapists.

[30]  G. Guyatt,et al.  Validation of an index of the quality of review articles. , 1991, Journal of clinical epidemiology.

[31]  M. Hazzard.  Klaus Linde, Gilbert Ramirez, Cynthia D. Mulrow, Andrej Pauls, Wolfgang Weidenhammer, and Dieter Melchart. (1996). St. John's wort for depression—An overview and meta-analysis of randomized clinical trials. British Medical Journal, 313, 3 August 1996, 253-8 , 1997 .

[32]  Gordon H. Guyatt,et al.  How to Use an Article About Therapy or Prevention , 1995 .

[33]  M Egger,et al.  Grey literature in meta-analyses of randomized trials of health care interventions. , 2007, The Cochrane database of systematic reviews.

[34]  T. Chalmers,et al.  Meta-analyses of randomized controlled trials. , 1987, The New England journal of medicine.

[35]  A. Vermeer,et al.  Qualitative analysis of therapeutic motor intervention programmes for children with cerebral palsy: an update. , 2002, Developmental medicine and child neurology.

[36]  B. Manns,et al.  The quality of reports of critical care meta-analyses in the Cochrane Database of Systematic Reviews: An independent appraisal* , 2007, Critical care medicine.

[37]  A R Jadad,et al.  Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. , 1998, JAMA.

[38]  J. Michálek,et al.  High incidence of Epstein-Barr virus, cytomegalovirus and human herpesvirus 6 infections in children with cancer , 2002, BMC pediatrics.

[39]  Gordon H. Guyatt,et al.  Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. , 1993, JAMA.

[40]  S. Mazzuca,et al.  Does patient education in chronic disease have therapeutic value? , 1982, Journal of chronic diseases.

[41]  G H Guyatt,et al.  USERS' GUIDES TO THE MEDICAL LITERATURE. II: HOW TO USE AN ARTICLE ABOUT THERAPY OR PREVENTION A. ARE THE RESULTS OF THE STUDY VALID ? , 1993 .

[42]  Susan Eastwood,et al.  Mejora de la calidad de los informes de los metaanálisis de los ensayos clínicos controlados: el acuerdo QUOROM , 2000 .

[43]  Catherine Sherrington,et al.  Evidence for physiotherapy practice: a survey of the Physiotherapy Evidence Database (PEDro). , 2002, The Australian journal of physiotherapy.

[44]  P. Tugwell,et al.  A Comparison of the Quality of Cochrane Reviews and Systematic Reviews Published in Paper-Based Journals , 2002, Evaluation & the health professions.

[45]  Youping Li,et al.  The reporting quality of meta-analyses improves: a random sampling study. , 2008, Journal of clinical epidemiology.

[46]  T C Chalmers,et al.  A method for assessing the quality of a randomized control trial. , 1981, Controlled clinical trials.

[47]  J. David Cassidy,et al.  A Systematic Review of the Prognosis of Acute Whiplash and a New Conceptual Framework to Synthesize the Literature , 2001, Spine.