"Meta-analysis: statistical alchemy for the 21st century": discussion. A plea for a more balanced view of meta-analysis and systematic overviews of the effect of health care interventions.

The paper discusses some of the most common criticisms to meta-analysis presented by Professor Feinstein in this Conference. As many of the points raised in his contributions are not new, a critique to them is presented in the context of the type of contribution given by systematic reviews (meta-analysis) to the analysis of the effects of health care interventions. After discussing some terminological issues, the paper challenges Feinsteins' arguments indicating that meta-analysis is inherently faulted on four grounds: (a) reproducibility, (b) precision, (c) suitable extrapolation, (d) fair comparison. Each point is discussed providing examples drawn from the published literature with a view to indicate that--despite their current limitations--systematic reviews are a necessary step to synthesize information, orient clinical research and help produce practice guidelines.

[1]  R Jaeschke,et al.  Selective Decontamination of the Digestive Tract: An Overview , 1994 .

[2]  M. Pike,et al.  Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. analysis and examples. , 1977, British Journal of Cancer.

[3]  C. Viscoli,et al.  Problems in the conduct and analysis of randomized clinical trials. Are we getting the right answers to the wrong questions? , 1992, Archives of Internal Medicine.

[4]  H. A. Quigley,et al.  Open-angle glaucoma. , 1993, The New England journal of medicine.

[5]  J D Emerson,et al.  An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. , 1990, Controlled clinical trials.

[6]  T C Chalmers,et al.  A method for assessing the quality of a randomized control trial. , 1981, Controlled clinical trials.

[7]  C D Naylor,et al.  Incorporating variations in the quality of individual randomized trials into meta-analysis. , 1992, Journal of clinical epidemiology.

[8]  L. Brazzi,et al.  Meta-analysis of randomised controlled trials of selective decontamination of the digestive tract. Selective Decontamination of the Digestive Tract Trialists' Collaborative Group. , 1993, British medical journal.

[9]  F. Mosteller,et al.  Reporting standards and research strategies for controlled trials , 1980 .

[10]  F. Mosteller,et al.  A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. , 1992, JAMA.

[11]  P. Sismondi,et al.  Is the published literature a reliable guide for deciding between alternative treatments for patients with early cervical cancer? , 1989, International journal of radiation oncology, biology, physics.

[12]  T. Chalmers,et al.  Meta-analyses of randomized controlled trials. , 1987, The New England journal of medicine.

[13]  V. Torri,et al.  Randomized Clinical Trials on Medical Treatment of Glaucoma: Are They Appropriate to Guide Clinical Practice? , 1994 .

[14]  F. Parazzini,et al.  Critical review of the quality and development of randomized clinical trials (RCTs) and their influence on the treatment of advanced epithelial ovarian cancer. , 1990, Annals of oncology : official journal of the European Society for Medical Oncology.

[15]  C. Mulrow.  The medical review article: state of the science. , 1987, Annals of internal medicine.

[16]  Mike Clarke,et al.  Systemic treatment of early breast cancer by hormonal, cytotoxic, or immune therapy: 133 randomised trials involving 31 000 recurrences and 24 000 deaths among 75 000 women , 1992 .

[17]  P. Armitage,et al.  Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. , 1976, British Journal of Cancer.

[18]  A. D. Schryver,et al.  Systemic treatment of early breast-cancer by hormonal, cytotoxic, or immune therapy: 133 randomized trials involving 31000 recurrences and 24000 deaths among 75000 women: 1 , 1992 .

[19]  V. Torri,et al.  Randomized clinical trials on medical treatment of glaucoma. Are they appropriate to guide clinical practice? , 1992, Archives of ophthalmology.

[20]  D L Sackett,et al.  An assessment of clinically useful measures of the consequences of treatment. , 1988, The New England journal of medicine.

[21]  S. Pocock,et al.  Statistical problems in the reporting of clinical trials. A survey of three medical journals. , 1987, The New England journal of medicine.

[22]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[23]  A R Feinstein,et al.  Meta-analysis: statistical alchemy for the 21st century. , 1995, Journal of clinical epidemiology.