A Brief History of Research Synthesis

Science is supposed to be cumulative, but scientists only rarely cumulate evidence scientifically. This means that users of research evidence have to cope with a plethora of reports of individual studies with no systematic attempt made to present new results in the context of similar studies. Although the need to synthesize research evidence has been recognized for well over two centuries, explicit methods for this form of research were not developed until the 20th century. The development of methods to reduce statistical imprecision using quantitative synthesis (meta-analysis) preceded the development of methods to reduce biases, the latter only beginning to receive proper attention during the last quarter of the 20th century. In this article, the authors identify some of the trends and highlights in this history, to which researchers in the physical, natural, and social sciences have all contributed, and speculate briefly about the “future history” of research synthesis.

[1]  Mike Clarke,et al.  Discussion sections in reports of controlled trials published in general medical journals. , 2002, JAMA.

[2]  Douglas G. Altman,et al.  Systematic Reviews in Health Care: Meta-Analysis in Context: Second Edition , 2008 .

[3]  Reto Steiner Is clinical research still too haphazard? , 2001, The Lancet.

[4]  B. Bausell,et al.  REVIEWING THE REVIEWS , 2001, International Journal of Technology Assessment in Health Care.

[5]  M. Limburg,et al.  Nimodipine in Animal Model Experiments of Focal Cerebral Ischemia: A Systematic Review , 2001, Stroke.

[6]  L. Brunt The advent of the sample survey in the social sciences , 2001 .

[7]  Steff Lewis,et al.  Forest plots: trying to see the wood and the trees , 2001, BMJ : British Medical Journal.

[8]  Mark Petticrew,et al.  Systematic reviews from astronomy to zoology: myths and misconceptions , 2001, BMJ : British Medical Journal.

[9]  Douglas G. Altman,et al.  Systematic Reviews in Health Care , 2001 .

[10]  G. Smith,et al.  1 Rationale, potentials, and promise of systematic reviews , 2001 .

[11]  T. Horder The organizer concept and modern embryology: Anglo-American perspectives. , 2001, The International journal of developmental biology.

[12]  I. Roberts,et al.  Human albumin solution for resuscitation and volume expansion in critically ill patients. The Albumin Reviewers. , 2000, The Cochrane database of systematic reviews.

[13]  D. McDowell Foreword , 1999 .

[14]  I Chalmers,et al.  Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? , 1998, JAMA.

[15]  J. Hampton The End of Medical History? , 1998, Journal of the Royal College of Physicians of London.

[16]  K. Dickersin,et al.  The Cochrane Collaboration: evaluation of health care and services using systematic reviews of the results of randomized controlled trials. , 1998, Clinical obstetrics and gynecology.

[17]  W. Winkelstein The first use of meta-analysis? , 1998, American journal of epidemiology.

[18]  G. Smith,et al.  Meta-analysis Spurious precision? Meta-analysis of observational studies , 1998, BMJ.

[19]  M. Hunt How Science Takes Stock: The Story of Meta-Analysis , 1997 .

[20]  I. Chalmers,et al.  Non-random reflections on health services research : on the 25th anniversary of Archie Cochrane's Effectiveness and efficiency , 1997 .

[21]  A. Böhmer,et al.  Macrolides and community-acquired pneumonia: is quorum sensing the key? , 2010, Critical care.

[22]  Adrian F. M. Smith Mad cows and ecstasy : Chance and choice in an evidence-based society , 1996 .

[23]  J. Last,et al.  Making the Dictionary of Epidemiology. , 1996, International journal of epidemiology.

[24]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[25]  M. Peckham The Implementation , 1995, Journal of the Royal College of Physicians of London.

[26]  E. Chelimsky Politics, Policy and Research Synthesis , 1995 .

[27]  M. J. R. Healy Frank Yates, 1902-1994 : the work of a statistician , 1995 .

[28]  A R Feinstein,et al.  Meta-analysis: statistical alchemy for the 21st century. , 1995, Journal of clinical epidemiology.

[29]  S Shapiro,et al.  Meta-analysis/Shmeta-analysis. , 1994, American journal of epidemiology.

[30]  Mark W. Lipsey,et al.  The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-analysis. , 1993, The American psychologist.

[31]  I. Chalmers The Cochrane Collaboration: Preparing, Maintaining, and Disseminating Systematic Reviews of the Effects of Health Care , 1993, Annals of the New York Academy of Sciences.

[32]  F Mosteller The prospect of data-based medicine in the light of ECPC. , 1993, The Milbank quarterly.

[33]  Gwenn W. Gröndal,et al.  Meta-analytic procedures for social research , 1993 .

[34]  R. Bausell After the Meta-Analytic Revolution , 1993 .

[35]  J K Harcourt,et al.  Effectiveness and efficiency. , 1992, Australian dental journal.

[36]  F. Mosteller,et al.  A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. , 1992, JAMA.

[37]  John C. Sinclair,et al.  Effective care of the newborn infant , 1992 .

[38]  D. Alberts,et al.  Chemotherapy in advanced ovarian cancer: an overview of randomised clinical trials. Advanced Ovarian Cancer Trialists Group. , 1991, BMJ.

[39]  D. Paintin Effective Care in Pregnancy and Childbirth , 1990, British journal of obstetrics and gynaecology.

[40]  R. Peto,et al.  Effects of adjuvant tamoxifen and of cytotoxic therapy on mortality in early breast cancer. An overview of 61 randomized trials among 28,896 women. , 1989, The New England journal of medicine.

[41]  G H Guyatt,et al.  Guidelines for reading literature reviews. , 1988, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[42]  Jack Cuzick,et al.  Effects of adjuvant tamoxifen and of cytotoxic therapy on mortality in early breast cancer. An overview of 61 randomized trials among 28,896 women , 1988 .

[43]  K A L'Abbé,et al.  Meta-analysis in clinical research. , 1987, Annals of internal medicine.

[44]  Larry V. Hedges,et al.  How hard is hard science, how soft is soft science? The empirical cumulativeness of research. , 1987 .

[45]  R Peto,et al.  Why do we need systematic overviews of randomized trials? , 1987, Statistics in medicine.

[46]  C. Mulrow The medical review article: state of the science. , 1987, Annals of internal medicine.

[47]  T. Chalmers,et al.  Meta-analyses of randomized controlled trials. , 1987, The New England journal of medicine.

[48]  L Mutch,et al.  The Oxford Database of Perinatal Trials: developing a register of published reports of controlled trials. , 1986, Controlled clinical trials.

[49]  Stephen M. Stigler,et al.  The History of Statistics: The Measurement of Uncertainty before 1900 by Stephen M. Stigler (review) , 1986, Technology and Culture.

[50]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[51]  R. Peto,et al.  Beta blockade during and after myocardial infarction: an overview of the randomized trials. , 1985, Progress in cardiovascular diseases.

[52]  David B. Pillemer,et al.  Conceptual Issues in Research Synthesis , 1984 .

[53]  Larry V. Hedges,et al.  Estimation of Effect Size under Nonrandom Sampling: The Effects of Censoring Studies Yielding Statistically Insignificant Mean Differences , 1984 .

[54]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[55]  H M Schoolman,et al.  Anatomy, physiology and pathology of biomedical information. , 1982, The Western journal of medicine.

[56]  Harris Cooper,et al.  Scientific Guidelines for Conducting Integrative Research Reviews , 1982 .

[57]  K. Warren,et al.  Coping With the Biomedical Literature , 1981 .

[58]  D. R. Lide Critical data for critical needs. , 1981, Science.

[59]  P. Glasziou Aspirin after myocardial infarction , 1989, Lancet.

[60]  Gregg B. Jackson,et al.  Methods for Integrative Reviews , 1980 .

[61]  G. Andrews,et al.  Meta-analysis of the effects of stuttering treatment. , 1980, The Journal of speech and hearing disorders.

[62]  F. Schmidt,et al.  Differential validity of employment tests by race: A comprehensive review and analysis. , 1979 .

[63]  R. Rosenthal The file drawer problem and tolerance for null results , 1979 .

[64]  Gene V. Glass,et al.  Meta-Analysis of Research on Class Size and Achievement , 1979 .

[65]  G. Smith,et al.  Medicines for the year 2000 , 1979 .

[66]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[67]  Donald B. Rubin,et al.  Interpersonal expectancy effects: the first 345 studies , 1978, Behavioral and Brain Sciences.

[68]  H. Eysenck An exercise in mega-silliness. , 1978 .

[69]  J. Tukey,et al.  Variations of Box Plots , 1978 .

[70]  T C Chalmers,et al.  Evidence favoring the use of anticoagulants in the hospital phase of acute myocardial infarction. , 1977, The New England journal of medicine.

[71]  M. L. Smith,et al.  Meta-analysis of psychotherapy outcome studies. , 1977, The American psychologist.

[72]  G. Glass Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[73]  J. Macintyre,et al.  POSTOPERATIVE RADIOTHERAPY AND BREAST CANCER , 1976, The Lancet.

[74]  J. Stjernswärd,et al.  POSTOPERATIVE RADIOTHERAPY AND BREAST CANCER , 1976, The Lancet.

[75]  E Vayda,et al.  A systematic review of the literature on evaluative studies of tonsillectomy and adenoidectomy. , 1976, Pediatrics.

[76]  William R. Shadish,et al.  Evaluation studies : review annual , 1976 .

[77]  Y. S. Touloukian REFERENCE DATA ON THERMOPHYSICS , 1975 .

[78]  A. H. Rosenfeld,et al.  The Particle Data Group: Growth and Operations-Eighteen Years of Particle Physics , 1975 .

[79]  G. Davis " MENSTRUAL TOXIN" AND HUMAN FERTILITY , 1974 .

[80]  Jr. D. Lide Status Report on Critical Compilation of Physical Chemical Data , 1973 .

[81]  J. Chao,et al.  Critically evaluated tables of thermodynamic data. , 1973 .

[82]  R. Light,et al.  Accumulating Evidence: Procedures for Resolving Contradictions among Different Research Studies. , 1971 .

[83]  Kenneth A. Feldman,et al.  Using the Work of Others: Some Observations on Reviewing and Integrating , 1971 .

[84]  B. Taylor,et al.  Determination of eh, Using Macroscopic Quantum Phase Coherence in Superconductors: Implications for Quantum Electrodynamics and the Fundamental Physical Constants , 1969 .

[85]  B. Taylor,et al.  DETERMINATION OF e/h, USING MACROSCOPIC QUANTUM PHASE COHERENCE IN SUPERCONDUCTORS. I. EXPERIMENT. , 1969 .

[86]  C. Herring Distil or drown: the need for reviews , 1968 .

[87]  Conyers Herring Distill or drown: The need for reviews , 1968 .

[88]  B. Glass The Critical State of the Critical Review Article , 1964, The Quarterly Review of Biology.

[89]  R. A. Chase,et al.  Bibliography: delayed auditory feedback. , 1959, Journal of speech and hearing research.

[90]  T. Sterling Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa , 1959 .

[91]  H. Beecher,et al.  The powerful placebo. , 1955, Journal of the American Medical Association.

[92]  W. G. Cochran The combination of estimates from different experiments. , 1954 .

[93]  Frederick Mosteller,et al.  Selected quantitative techniques and attitude measurement , 1954 .

[94]  L. V. Jones,et al.  Models for testing the significance of combined results. , 1953, Psychological bulletin.

[95]  F. Yates,et al.  Influence of changes in level of feeding on milk production , 1942, The Journal of Agricultural Science.

[96]  Frank Yates,et al.  Fertilizer policy in wartime. The fertilizer requirements of arable crops , 1941 .

[97]  William G. Cochran,et al.  The analysis of groups of experiments , 1938, The Journal of Agricultural Science.

[98]  W. G. Cochran Problems arising in the analysis of a series of similar experiments , 1937 .

[99]  C. Peters Summary of the Penn State Experiments on the Influence of Instruction in Character Education , 1933 .

[100]  K. Pearson ON A METHOD OF DETERMINING WHETHER A SAMPLE OF SIZE n SUPPOSED TO HAVE BEEN DRAWN FROM A PARENT POPULATION HAVING A KNOWN PROBABILITY INTEGRAL HAS PROBABLY BEEN DRAWN AT RANDOM , 1933 .

[101]  R. Fisher,et al.  Statistical Methods for Research Workers , 1930, Nature.

[102]  R. T. Birge,et al.  The Calculation of Errors by the Method of Least Squares , 1932 .

[103]  L. Simpson Report on Certain Enteric Fever Inoculation Statistics , 1904, British medical journal.