Review Article

The present work is an overview of the main pitfalls which may occur when a researcher performs a meta-analysis. The main goal is to help clinicians evaluate published research results. Organizing and carrying out a meta-analysis is hard work, but the findings can be significant. Meta-analysis is a powerful tool to cumulate and summarize the knowledge in a research field, and to identify the overall measure of a treatment’s effect by combining several conclusions. However, it is a controversial tool, because even small violations of certain rules can lead to misleading conclusions. In fact, several decisions made when designing and performing a meta-analysis require personal judgment and expertise, thus creating personal biases or expectations that may influence the result. Meta-analysis’ conclusions should be interpreted in the light of various checks, discussed in this work, which can inform the readers of the likely reliability of the conclusions. Specifically, we explore the principal steps (from writing a prospective protocol of analysis to results’ interpretation) in order to minimize the risk of conducting a mediocre meta-analysis and to support researchers to accurately evaluate the published findings.

[1]  Jacob Stegenga,et al.  Is meta-analysis the platinum standard of evidence? , 2011, Studies in history and philosophy of biological and biomedical sciences.

[2]  J. Ioannidis,et al.  The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration , 2009, Annals of Internal Medicine [serial online].

[3]  M. Pittler Systematic Reviews in Health Care: Meta‐analysis in Context , 2010 .

[4]  L. Simpson Report on Certain Enteric Fever Inoculation Statistics , 1904, British medical journal.

[5]  D. Cook,et al.  Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? , 1998, The Lancet.

[6]  S. Yusuf,et al.  Overcoming the limitations of current meta-analysis of randomised controlled trials , 1998, The Lancet.

[7]  B. Takkouche,et al.  Meta-analysis protocol registration: sed quis custodiet ipsos custodes? [but who will guard the guardians?]. , 2010, Epidemiology.

[8]  G. Biondi-Zoccai,et al.  In Intensive Care Cardiovascular Anesthesia Review Article the Rough Guide to Systematic Reviews and Meta-analyses , 2022 .

[9]  Freya Harrison,et al.  Getting started with meta‐analysis , 2011 .

[10]  Stephen W Lagakos,et al.  Statistics in medicine--reporting of subgroup analyses in clinical trials. , 2007, The New England journal of medicine.

[11]  R. Bellomo,et al.  Why we should be wary of single-center trials , 2009, Critical care medicine.

[12]  Jonathan A C Sterne,et al.  Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. , 2001, BMJ.

[13]  G. Biondi-Zoccai,et al.  A journey into clinical evidence: from case reports to mixed treatment comparisons , 2011, HSR proceedings in intensive care & cardiovascular anesthesia.

[14]  Michael W Kattan,et al.  Meta-analysis: Its strengths and limitations. , 2008, Cleveland Clinic journal of medicine.

[15]  C. Ko,et al.  Meta-analysis in surgery: methods and limitations. , 2006, Archives of surgery.

[16]  J. Sterne,et al.  Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. , 2000, Journal of clinical epidemiology.

[17]  G. Glass Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[18]  Tianxi Cai,et al.  Meta‐analysis for rare events , 2010, Statistics in medicine.

[19]  S Greenland,et al.  Ecological bias, confounding, and effect modification. , 1989, International journal of epidemiology.

[20]  Michele Tarsilla Cochrane Handbook for Systematic Reviews of Interventions , 2010, Journal of MultiDisciplinary Evaluation.

[21]  H. Dubben,et al.  Systematic review of publication bias in studies on publication bias , 2005, BMJ : British Medical Journal.

[22]  Peter W Lane,et al.  Meta-analysis of incidence of rare events , 2013, Statistical methods in medical research.

[23]  Gary H Lyman,et al.  The strengths and limitations of meta-analyses based on aggregate data , 2005, BMC Medical Research Methodology.

[24]  George Davey Smith,et al.  Where now for meta-analysis? , 2002, International journal of epidemiology.

[25]  Douglas G Altman,et al.  Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study , 2010, BMJ : British Medical Journal.