Systematic reviews in health care: Assessing the quality of controlled clinical trials.

This is the first in a series of four articles The quality of controlled trials is of obvious relevance to systematic reviews. If the “raw material” is flawed then the conclusions of systematic reviews cannot be trusted. Many reviewers formally assess the quality of primary trials by following the recommendations of the Cochrane Collaboration and other experts. 1 2 However, the methodology for both the assessment of quality and its incorporation into systematic reviews and meta-analysis are a matter of ongoing debate.3-5 In this article we discuss the concept of study quality and the methods used to assess quality. #### Components of internal and external validity of controlled clinical trials Internal validity —extent to which systematic error (bias) is minimised in clinical trials Quality is a multidimensional concept, which could relate to the design, conduct, and analysis of a trial, its clinical relevance, or quality of reporting.6 The validity of the findings generated by a study clearly is an important dimension of quality. In the 1950s the social scientist Campbell proposed a useful distinction between internal and external validity (see box below). 7 8 Internal validity implies that the differences observed between groups of patients allocated to different …

[1]  Douglas G. Altman,et al.  Systematic Reviews in Health Care: Meta-Analysis in Context: Second Edition , 2008 .

[2]  Jonathan A C Sterne,et al.  Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. , 2001, BMJ.

[3]  D. Moher,et al.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. , 2001, Annals of internal medicine.

[4]  D. Moher,et al.  Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. , 2001, JAMA.

[5]  G. Guyatt,et al.  Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. , 2001, JAMA.

[6]  M Egger,et al.  Value of flow diagrams in reports of randomized controlled trials. , 2001, JAMA.

[7]  J A Espinosa,et al.  Reducing errors made by emergency physicians in interpreting radiographs: longitudinal study , 2000, BMJ : British Medical Journal.

[8]  M. Egger,et al.  The hazards of scoring the quality of clinical trials for meta-analysis. , 1999, JAMA.

[9]  S. Hollis,et al.  What is meant by intention to treat analysis? Survey of published randomised controlled trials , 1999, BMJ.

[10]  W B Jonas,et al.  Impact of study quality on outcome in placebo-controlled trials of homeopathy. , 1999, Journal of clinical epidemiology.

[11]  Douglas G Altman,et al.  Treatment allocation in controlled trials: why randomise? , 1999, BMJ.

[12]  J. Wyatt,et al.  The association between seniority of Accident and Emergency doctor and outcome following trauma. , 1999, Injury.

[13]  Karla Kerlikowske,et al.  Antihypertensive drugs in very old people: a subgroup meta-analysis of randomised controlled trials , 1999, The Lancet.

[14]  D. Cook,et al.  Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. , 1999, Health technology assessment.

[15]  D. Cook,et al.  Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? , 1998, The Lancet.

[16]  S. Meryn.  Improving doctor-patient communication , 1998, BMJ.

[17]  Enrico W. Coiera,et al.  Communication behaviours in a hospital setting: an observational study , 1998, BMJ.

[18]  G Lewis,et al.  Putting trials on trial--the costs and consequences of small trials in depression: a systematic review of methodology. , 1997, Journal of epidemiology and community health.

[19]  J. Gijn.  Slip-ups in diagnosis of subarachnoid haemorrhage , 1997, The Lancet.

[20]  Kenneth F Schulz,et al.  Randomised trials, human nature, and reporting guidelines , 1996, The Lancet.

[21]  K. Schulz,et al.  Subverting randomization in controlled trials. , 1995, JAMA.

[22]  A R Jadad,et al.  Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. , 1995, Controlled clinical trials.

[23]  R. J. Hayes,et al.  Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. , 1995, JAMA.

[24]  A R Feinstein,et al.  Meta-analysis: statistical alchemy for the 21st century. , 1995, Journal of clinical epidemiology.

[25]  D. Cook,et al.  Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on Meta-Analysis. , 1995, Journal of clinical epidemiology.

[26]  S. Greenland.  Quality Scores Are Useless and Potentially Misleading: Reply to “Re: A Critical Look at Some Popular Analytic Methods” , 1994 .

[27]  C. Fry,et al.  Science of urinary incontinence Report of a Meeting of Physicians and Scientists, University College London , 1994, The Lancet.

[28]  Fibrinolytic Therapy Trialists' Collaborative Group.  Indications for fibrinolytic therapy in suspected acute myocardial infarction: collaborative overview of early mortality and major morbidity results from all randomised trials of more than 1000 patients , 1994, The Lancet.

[29]  J. Noseworthy,et al.  The impact of blinding on the results of a randomized, placebo‐controlled multiple sclerosis clinical trial , 1994, Neurology.

[30]  Johan Herlitz,et al.  Indications for fibrinolytic therapy in suspected acute myocardial infarction : collaborative overview of early mortality and major morbidity results from all randomised trials of more than 1000 patients , 1994 .

[31]  S G Thompson,et al.  Controversies in meta-analysis: the case of the trials of serum cholesterol reduction , 1993, Statistical methods in medical research.

[32]  C D Naylor,et al.  Incorporating variations in the quality of individual randomized trials into meta-analysis. , 1992, Journal of clinical epidemiology.

[33]  Douglas G. Altman,et al.  RANDOMISATION : ESSENTIAL FOR REDUCING BIAS , 1991 .

[34]  M. Keirse.  Amniotomy Or Oxytocin For Induction Of Labor: Re‐Analysis Of A Randomized Controlled Trial , 1988, Acta obstetricia et gynecologica Scandinavica.

[35]  A Liberati,et al.  A quality assessment of randomized control trials of primary treatment of breast cancer. , 1986, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[36]  M. Wittrock.  Handbook of research on teaching , 1986 .

[37]  D. DeMets,et al.  The randomized clinical trial: bias in analysis. , 1981, Circulation.

[38]  T C Chalmers,et al.  A method for assessing the quality of a randomized control trial. , 1981, Controlled clinical trials.

[39]  D. Sackett,et al.  Controversy in counting and attributing events in clinical trials. , 1979, The New England journal of medicine.

[40]  E A Murphy,et al.  The Logic of Medicine , 1976, The American journal of medicine.

[41]  S. Rössner,et al.  Coronary Drug Project Research Group. , 1978, Atherosclerosis.

[42]  Alan Sillitoe,et al.  Raw material , 1972 .

[43]  H. Lincoff.  Guilty until proved innocent. , 1967, Archives of ophthalmology.

[44]  D. Campbell.  Factors relevant to the validity of experiments in social settings. , 1957, Psychological bulletin.