Sufficiency and stability of evidence for public health interventions using cumulative meta-analysis.

We propose cumulative meta-analysis as the procedure of completing a new meta-analysis at each successive wave in a research database. Two facets of cumulative knowledge are considered: the first, sufficiency, refers to whether the meta-analytic database adequately demonstrates that a public health intervention works. The second, stability, refers to the shifts over time in the accruing evidence about whether a public health intervention works. We used a hypothetical data set to develop the indicators of sufficiency and stability, and then applied them to existing, published datasets. Our discussion centers on the implications of the use of this procedure in evaluating public health interventions.

[1]  B. Wampold,et al.  Intervention effects and relative variation as dimensions in experts' use of visual inference. , 1982, Journal of applied behavior analysis.

[2]  R. Rosenthal,et al.  Statistical Procedures and the Justification of Knowledge in Psychological Science , 1989 .

[3]  N. Miller,et al.  Alcohol and aggression: a meta-analysis on the moderating effects of inhibitory cues, triggering events, and self-focused attention. , 1996, Psychological bulletin.

[4]  L. Ku,et al.  Factors influencing first intercourse for teenage men. , 1993, Public health reports.

[5]  P. Sheeran,et al.  Do intentions predict condom use? Meta-analysis and examination of six moderator variables. , 1998, The British journal of social psychology.

[6]  Blair T. Johnson,et al.  Comparison of three major meta-analytic approaches. , 1995 .

[7]  Brian Mullen,et al.  Cumulative Meta-Analysis: A Consideration of Indicators of Sufficiency and Stability , 2001 .

[8]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[9]  Donald B. Rubin,et al.  Interpersonal expectancy effects: the first 345 studies , 1978, Behavioral and Brain Sciences.

[10]  S. Cohen,et al.  Depression and immunity: a meta-analytic review. , 1993, Psychological bulletin.

[11]  Cheryl Tatano Beck.  A meta-analysis of the relationship between postpartum depression and infant temperament. , 1996, Nursing research.

[12]  Judith A. Hall,et al.  Effectiveness of interventions to improve patient compliance: a meta-analysis. , 1998, Medical care.

[13]  W. Marsiglio,et al.  The impact of sex education on sexual activity, contraceptive use and premarital pregnancy among American teenagers. , 1986, Family planning perspectives.

[14]  Brian Mullen,et al.  Advanced BASIC meta-analysis , 1989 .

[15]  John E. Hunter,et al.  Comparison of Three Meta-Analysis Methods Revisited: An Analysis of Johnson, Mullen, and Salas (1995) , 1999 .

[16]  A. Deprospero,et al.  Inconsistent visual analyses of intrasubject data. , 1979, Journal of applied behavior analysis.

[17]  John M. Gottman,et al.  4 – Analysis of Interrupted Time-Series Experiments , 1978 .

[18]  A. Peterson,et al.  Meta‐Analysis of Interventions To Improve Drug Adherence in Patients with Hyperlipidemia , 2003, Pharmacotherapy.

[19]  W. Keith Campbell,et al.  Self-Esteem and Socioeconomic Status: A Meta-Analytic Review , 2002 .

[20]  Larry V. Hedges,et al.  Estimating Effect Size Under Publication Bias: Small Sample Properties and Robustness of a Random Effects Selection Model , 1996 .

[21]  M. Pitts,et al.  Educating young people about drugs: a systematic review. , 1998, Addiction.

[22]  David B. Pillemer,et al.  Summing Up: The Science of Reviewing Research , 1984 .

[23]  Stanton A Glantz,et al.  Effect of smoke-free workplaces on smoking behaviour: systematic review , 2002, BMJ : British Medical Journal.

[24]  G J Acton,et al.  Interventions to reduce the burden of caregiving for an adult with dementia: a meta-analysis. , 2001, Research in nursing & health.

[25]  Frank L. Schmidt,et al.  What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. , 1992 .

[26]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[27]  David E. Coleman,et al.  Elements of Graphing Data , 1987 .

[28]  H. Mcdonald,et al.  Interventions to enhance patient adherence to medication prescriptions: scientific review. , 2002, JAMA.

[29]  N. Freemantle,et al.  Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? , 1999, JAMA.

[30]  W W Tryon,et al.  A simplified time-series analysis for evaluating treatment interventions. , 1982, Journal of applied behavior analysis.

[31]  A. Greenwald.  Consequences of Prejudice Against the Null Hypothesis , 1975 .

[32]  Changes in incidence of sexual intercourse of unmarried teenagers following a community‐based sex education program , 1987 .

[33]  R. Rosenthal.  Meta-analytic procedures for social research , 1984 .

[34]  Robert McGill,et al.  The Many Faces of a Scatterplot , 1984 .

[35]  C. Furberg,et al.  Update of effects of calcium antagonists in myocardial infarction or angina in light of the second Danish Verapamil Infarction Trial (DAVIT-II) and other recent studies. , 1991, The American journal of cardiology.

[36]  S. Yusuf,et al.  Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. , 1997, Controlled clinical trials.

[37]  Donald B. Rubin,et al.  Issues in summarizing the first 345 studies of interpersonal expectancy effects , 1978, Behavioral and Brain Sciences.

[38]  M. Hojat,et al.  Impartial Judgment by the “Gatekeepers” of Science: Fallibility and Accountability in the Peer Review Process , 2003, Advances in health sciences education : theory and practice.

[39]  P. Sweeney,et al.  Attributional style in depression: a meta-analytic review. , 1986, Journal of personality and social psychology.

[40]  John W. Tukey,et al.  Data-Based Graphics: Visual Display in the Decades to Come , 1990 .

[41]  R. Kasuya,et al.  Caregiver Burden and Burnout: A Guide for Primary Care Physicians , 2000, Postgraduate medicine.

[42]  T C Chalmers,et al.  Cumulative meta-analysis of therapeutic trials for myocardial infarction. , 1992, The New England journal of medicine.

[43]  S. Koger,et al.  Is Music Therapy an Effective Intervention for Dementia?A Meta-Analytic Review of Literature. , 1999, Journal of music therapy.

[44]  D. McArthur.  Heart healthy eating behaviors of children following a school-based intervention: a meta-analysis. , 1998, Issues in comprehensive pediatric nursing.

[45]  K J Ottenbacher,et al.  Interrater agreement of visual analysis in single-subject decisions: quantitative review and analysis. , 1993, American journal of mental retardation : AJMR.

[46]  Harris Cooper,et al.  Statistically Combining Independent Studies: A Meta-Analysis of Sex Differences in Conformity Research , 1979 .

[47]  D. Rennie,et al.  Publication bias in editorial decision making. , 2002, JAMA.

[48]  Donald B. Rubin,et al.  Comment: Assumptions and Procedures in the File Drawer Problem , 1988 .

[49]  R. R. Jones,et al.  Effects of serial dependency on the agreement between visual and statistical inference. , 1978, Journal of applied behavior analysis.