Review of methodology of quantitative reviews using meta‐analysis in ecology

Summary 1 Statistical methods for combination of independent results have been well publicized in the ecological literature, and have begun to be used for reviewing research. They provide a considerable advance in scientific rigour over traditional narrative or ‘vote-counting’ reviews. 2 However, other methodological developments for research synthesis have not yet been widely adopted. 3 This review briefly summarizes some of the techniques used for carrying out rigorous reviewing and synthesis of results in medical science, and surveys techniques used by ecological meta-analyses. 4 Many of the methods used to reduce bias and enhance the accuracy, reliability and usefulness of reviews in medical science have not yet been widely used by ecologists. 5 The quality of ecological reviews could be improved by adoption of some of these methods, such as specifying the methods used for literature searching, stating the types of study combined in the review and the strength of evidence they provide, presenting results as a point estimate with a confidence interval, investigating bias in selection of studies using funnel plots, making a clear distinction between the main analysis and subsidiary analyses and interpreting the latter with caution, and performing sensitivity analyses.

[1]  Peter S. Curtis,et al.  A meta-analysis of elevated CO2 effects on woody plant mass, form, and physiology , 1998, Oecologia.

[2]  Jessica Gurevitch,et al.  A Meta-Analysis of Competition in Field Experiments , 1992, The American Naturalist.

[3]  D. N. Byrne,et al.  The effects of crop diversification on herbivorous insects: a meta‐analysis approach , 1994 .

[4]  Anders Pape Møller,et al.  Sperm competition and sexual selection: a meta-analysis of paternity studies of birds , 1998, Behavioral Ecology and Sociobiology.

[5]  Isabelle M. Côté,et al.  Parasitism and group size in social animals: a meta-analysis , 1995 .

[6]  G. Smith,et al.  Bias in meta-analysis detected by a simple, graphical test , 1997, BMJ.

[7]  Michael T. Brett,et al.  Consumer Versus Resource Control in Freshwater Pelagic Food Webs , 1997, Science.

[8]  A. Palmer,et al.  Detecting Publication Bias in Meta‐analyses: A Case Study of Fluctuating Asymmetry and Sexual Selection , 1999, The American Naturalist.

[9]  J. Koricheva,et al.  Insect performance on experimentally stressed woody plants: a meta-analysis. , 1998, Annual review of entomology.

[10]  D. Wooster,et al.  Predator impacts on stream benthic prey , 1994, Oecologia.

[11]  Lenore Fahrig,et al.  Intra-specific abundance-distribution relationships , 1998 .

[12]  E. V. Werf,et al.  Lack's Clutch Size Hypothesis: An Examination of the Evidence Using Meta‐Analysis , 1992 .

[13]  William J. Hamilton,et al.  The Hamilton and Zuk Hypothesis Revisited: a Meta-Analytical Approach , 1997 .

[14]  Jessica Gurevitch,et al.  THE META‐ANALYSIS OF RESPONSE RATIOS IN EXPERIMENTAL ECOLOGY , 1999 .

[15]  S. Juliano,et al.  EFFECTS OF A PREDATOR ON PREY METAMORPHOSIS: PLASTIC RESPONSES BY PREY OR SELECTIVE MORTALITY? , 1997 .

[16]  R. Rosenthal Combining results of independent studies. , 1978 .

[17]  William J. Sutherland,et al.  The Effectiveness of Removing Predators to Protect Bird Populations , 1997 .

[18]  Daphne J. Fairbairn,et al.  A Comparative Analysis of Allometry for Sexual Size Dimorphism: Assessing Rensch's Rule , 1997, The American Naturalist.

[19]  A D Oxman,et al.  The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials , 1998, BMJ.

[20]  John E. Hunter,et al.  Methods of Meta-Analysis: Correcting Error and Bias in Research Findings , 1991 .

[21]  Peter A. Van Zandt,et al.  A Meta‐Analysis of Adaptive Deme Formation in Phytophagous Insect Populations , 1998, The American Naturalist.

[22]  J. Koricheva,et al.  Regulation of Woody Plant Secondary Metabolism by Resource Availability: Hypothesis Testing by Means of Meta-Analysis , 1998 .

[23]  Robert Poulin,et al.  Meta-analysis of parasite-induced behavioural changes , 1994, Animal Behaviour.

[24]  Harris Cooper,et al.  The Integrative Research Review: A Systematic Approach , 1984 .

[25]  Eduardo Fernandez-Duque,et al.  Meta‐Analysis: A Valuable Tool in Conservation Research , 1994 .

[26]  Dean C. Adams,et al.  RESAMPLING TESTS FOR META‐ANALYSIS OF ECOLOGICAL DATA , 1997 .

[27]  S. D. Cooper,et al.  Effect Size in Ecological Experiments: The Application of Biological Models in Meta‐Analysis , 1997, The American Naturalist.

[28]  Jonas Dahl,et al.  Impact on stream benthic prey by benthic vs drift feeding predators: a meta-analysis , 1996 .

[29]  T. Chalmers,et al.  Meta-analyses of randomized controlled trials. , 1987, The New England journal of medicine.

[30]  M. Forbes,et al.  Male biases in parasitism of mammals : effects of study type, host age, and parasite taxon , 1997 .

[31]  G H Guyatt,et al.  Guidelines for reading literature reviews. , 1988, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[32]  David Moher,et al.  The Medical Review Article Revisited: Has the Science Improved? , 1999, Annals of Internal Medicine.

[33]  Jonathan A C Sterne,et al.  Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. , 2001, BMJ.

[34]  G. Smith,et al.  Meta-analysis Spurious precision? Meta-analysis of observational studies , 1998, BMJ.

[35]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[36]  R. Thornhill,et al.  A meta‐analysis of the heritability of developmental stability , 1997 .

[37]  C. Lengeler,et al.  Language bias in randomised controlled trials published in English and German , 1997, The Lancet.

[38]  Brian Leung,et al.  Fluctuating asymmetry in relation to stress and fitness: Effects of trait type as revealed by meta-analysis , 1996 .

[39]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[40]  R. Rosenthal The file drawer problem and tolerance for null results , 1979 .

[41]  Cheryl L. Adkins Women & men in management: Gary N. Powell, Sage Publications, Newbury Park, CA. 1993 , 1994 .

[42]  P. Fiske,et al.  Mating success in lekking males: a meta-analysis , 1998 .

[43]  D. Moher,et al.  The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials , 2001, The Lancet.

[44]  Göran Englund,et al.  The importance of data-selection criteria: meta-analyses of stream predation experiments , 1999 .

[45]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[46]  Douglas G. Altman,et al.  Systematic Reviews in Health Care , 2001 .

[47]  M. Forbes,et al.  Sex-Biased Parasitism of Avian Hosts: Relations to Blood Parasite Taxon and Mating System , 1998 .

[48]  R. Rosenthal Meta-analytic procedures for social research , 1984 .

[49]  A. Mazumder,et al.  REVERSAL OF GRAZING IMPACT ON PLANT SPECIES RICHNESS IN NUTRIENT‐POOR VS. NUTRIENT‐RICH ECOSYSTEMS , 1998 .

[50]  Craig W. Osenberg,et al.  Meta‐analysis in Ecology: Concepts, Statistics, and Applications , 1999 .

[51]  C. Begg,et al.  Operating characteristics of a rank correlation test for publication bias. , 1994, Biometrics.

[52]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[53]  Jessica Gurevitch,et al.  STATISTICAL ISSUES IN ECOLOGICAL META‐ANALYSES , 1999 .

[54]  Peter S. Curtis,et al.  A meta‐analysis of leaf gas exchange and nitrogen in trees grown under elevated carbon dioxide , 1996 .

[55]  Robert Poulin,et al.  Sexual Inequalities in Helminth Infections: A Cost of Being a Male? , 1996, The American Naturalist.

[56]  Truman P. Young,et al.  Social and ecological influences on activity budgets of vervet monkeys, and their implications for group living , 1993, Behavioral Ecology and Sociobiology.

[57]  A D Oxman,et al.  Systematic Reviews: Checklists for review articles , 1994 .

[58]  G. Arnqvist,et al.  Meta-analysis: synthesizing research findings in ecology and evolution. , 1995, Trends in ecology & evolution.

[59]  Bradley J. Swanson,et al.  Spatial and temporal trends and effects of population size on the frequency of color phenotypes in the wild red fox (Vulpes vulpes) , 1996 .

[60]  K. Dickersin,et al.  Systematic Reviews: Identifying relevant studies for systematic reviews , 1994 .

[61]  Brad R. Murray,et al.  Density‐dependent germination and the role of seed leachate , 1998 .

[62]  Malcolm L. Hunter,et al.  A Meta‐Analysis of Forest Cover, Edge Effects, and Artificial Nest Predation Rates , 1998 .

[63]  David A. Andow,et al.  Limitations of Trichogramma nubilale (Hymenoptera: Trichogrammatidae) as an inundative biological control of Ostrinia nubilalis (Lepidoptera: Crambidae). , 1995 .

[64]  C. Goldman,et al.  A meta-analysis of the freshwater trophic cascade. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[65]  L. Hedges,et al.  Meta-analysis: Combining the results of independent experiments , 1993 .

[66]  L. Fahrig,et al.  HABITAT LOSS AND POPULATION DECLINE: A META‐ANALYSIS OF THE PATCH SIZE EFFECT , 1998 .

[67]  William J. Sutherland,et al.  Spatial patterns of depletion imposed by foraging vertebrates : theory, review and meta-analysis , 1997 .

[68]  C. Mulrow The medical review article: state of the science. , 1987, Annals of internal medicine.

[69]  S. Sharp,et al.  Explaining heterogeneity in meta-analysis: a comparison of methods. , 1999 .