A comparison of meta-analytic results using literature vs individual patient data. Paternal cell immunization for recurrent miscarriage.

OBJECTIVE To compare the meta-analytic results from published literature vs those obtained from pooling original patient data. DATA SOURCES Individual patient data from 15 completed or ongoing trials on paternal white blood cell immunization as treatment for recurrent miscarriage were obtained through the American Society for Reproductive Immunology. STUDY SELECTION Only randomized controlled trials were selected. Within these eight selected trials, 202 patients were from four published studies, 43 were added by the same investigators after publication, and 140 were from four unpublished trials. DATA EXTRACTION Individual patient data were collected using a standardized form and double data entry. DATA SYNTHESIS Using the fixed treatment effect model, we found that the effect of immunization, denoted as the relative live-birth ratio (RR), was greater by pooling summary data from published articles (RR, 1.29; 95% confidence interval [CI], 1.03 to 1.60) than by pooling all individual patient data from the same investigators (RR, 1.17; 95% CI, 0.97 to 1.37) or when individual patient data were pooled from four unpublished trials (RR, 1.01; 95% CI, 0.74 to 1.28). A similar diminishing treatment effect for the same comparisons was observed using the random treatment effect model (RR, 1.38; 95% CI, 0.89 to 1.87 using published summary data; RR, 1.18; 95% CI, 0.98 to 1.42 using individual patient data; RR, 1.01; 95% CI, 0.74 to 1.28 using unpublished trials). CONCLUSIONS Meta-analytic results may differ depending on the use of various data sources: whether the source was from summary statistics in the literature, from individual patient data provided by trialists, or from unpublished trials.

[1]  E. Korn,et al.  Regression analysis with clustered data. , 1994, Statistics in medicine.

[2]  K. Wachter,et al.  Disturbed by meta-analysis? , 1988, Science.

[3]  R. Simes,et al.  Treatment of Recurrent Spontaneous Abortion by Immunization With Paternal Lymphocytes: Results of a Controlled Trial , 1993, American journal of reproductive immunology.

[4]  Frederick Mosteller,et al.  A Survey of Current Problems in Meta‐Analysis: Discussion from the Agency for Health Care Policy and Research Inter‐PORT Work Group on Literature Review/Meta‐Analysis , 1995, Medical care.

[5]  I. Chalmers.  The Cochrane Collaboration: Preparing, Maintaining, and Disseminating Systematic Reviews of the Effects of Health Care , 1993, Annals of the New York Academy of Sciences.

[6]  J. Mowbray,et al.  CONTROLLED TRIAL OF TREATMENT OF RECURRENT SPONTANEOUS ABORTION BY IMMUNISATION WITH PATERNAL CELLS , 1985, The Lancet.

[7]  R. Pepperell,et al.  Treatment of Recurrent Aborters by Immunization With Paternal Cells—Controlled Trial , 1991, American journal of reproductive immunology.

[8]  F Mosteller,et al.  Some Statistical Methods for Combining Experimental Results , 1990, International Journal of Technology Assessment in Health Care.

[9]  T C Chalmers,et al.  Meta-analysis of clinical trials as a scientific discipline. II: Replicate variability and comparison of studies that agree and disagree. , 1987, Statistics in medicine.

[10]  R. Wittes.  Problems in the medical interpretation of overviews. , 1987, Statistics in medicine.

[11]  W. McIlroy,et al.  Should unpublished data be included in meta-analyses? Current convictions and controversies. , 1993, JAMA.

[12]  M. Parmar,et al.  Meta-analysis of the literature or of individual patient data: is there a difference? , 1993, The Lancet.

[13]  S B Thacker,et al.  Methods for pooled analyses of epidemiologic studies. , 1994, Epidemiology.

[14]  T R Fleming,et al.  Effects on overviews of early stopping rules for clinical trials. , 1987, Statistics in medicine.

[15]  H C Van Houwelingen,et al.  A bivariate approach to meta-analysis. , 1993, Statistics in medicine.

[16]  R Peto,et al.  Why do we need some large, simple randomized trials? , 1984, Statistics in medicine.

[17]  P. Easterbrook,et al.  Publication bias in clinical research , 1991, The Lancet.

[18]  S. Cowchock,et al.  What's a Mother to Do? Analysis of Trials Evaluating New Treatments for Unexplained Recurrent Miscarriages and Other Complaints , 1991, American journal of reproductive immunology.

[19]  S. Pocock,et al.  The impact of stopping rules on heterogeneity of results in overviews of clinical trials. , 1992, Biometrics.

[20]  S G Thompson,et al.  Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated , 1994, BMJ.

[21]  S. Daya,et al.  Trials and Tribulation in the Treatment of Recurrent Spontaneous Abortion , 1991, American journal of reproductive immunology.

[22]  C. Friedenreich.  Methods for Pooled Analyses of Epidemiologic Studies , 1993, Epidemiology.

[23]  I Olkin,et al.  Statistical and theoretical considerations in meta-analysis. , 1995, Journal of clinical epidemiology.

[24]  O. Christiansen,et al.  Placebo‐controlled trial of active immunization with third party leukocytes in recurrent miscarriage , 1994, Acta obstetricia et gynecologica Scandinavica.

[25]  R. Temple.  Problems in the Use of Large Data Sets to Assess Effectiveness , 1990, International Journal of Technology Assessment in Health Care.

[26]  D L Demets,et al.  Methods for combining randomized clinical trials: strengths and limitations. , 1987, Statistics in medicine.

[27]  S Greenland,et al.  Bias in the one-step method for pooling study results. , 1990, Statistics in medicine.

[28]  S G Thompson,et al.  The potential and limitations of meta-analysis. , 1991, Journal of epidemiology and community health.

[29]  E. Chelimsky,et al.  Cross-design Synthesis: A New Form of Meta-analysis for Combining Results from Randomized Clinical Trials and Medical-practice Databases , 1993, International Journal of Technology Assessment in Health Care.

[30]  R. Woolson,et al.  A Monte Carlo investigation of homogeneity tests of the odds ratio under various sample size configurations. , 1989, Biometrics.

[31]  M. Sandoz.  The logic and role of meta-analysis in clinical research , 1993 .

[32]  I. Chalmers,et al.  Retrospective and prospective identification of unpublished controlled trials: lessons from a survey of obstetricians and pediatricians. , 1989, Pediatrics.

[33]  D. Felson,et al.  Bias in meta-analytic research. , 1992, Journal of clinical epidemiology.

[34]  A. Grant,et al.  Preparing and updating systematic reviews of randomized controlled trials of health care. , 1993, The Milbank quarterly.

[35]  N. E. Breslow.  Statistical Methods in Cancer Research , 1986 .

[36]  F. Mosteller,et al.  How study design affects outcomes in comparisons of therapy. I: Medical. , 1989, Statistics in medicine.

[37]  O. Christiansen,et al.  Worldwide Collaborative Observational Study and Meta‐Analysis on Allogenic Leukocyte Immunotherapy for Recurrent Spontaneous Abortion 1 , 1994 .

[38]  David Hunt,et al.  ISIS-3 - A RANDOMIZED COMPARISON OF STREPTOKINASE VS TISSUE PLASMINOGEN-ACTIVATOR VS ANISTREPLASE AND OF ASPIRIN PLUS HEPARIN VS ASPIRIN ALONE AMONG 41,299 CASES OF SUSPECTED ACUTE MYOCARDIAL-INFARCTION , 1992 .

[39]  Frank T. Denton.  The Effects of Publication Selection on Test Probabilities and Estimator Distributions , 1990 .

[40]  K. Schulz,et al.  Immunization as therapy for recurrent spontaneous abortion: A review and meta‐analysis , 1993, Obstetrics and gynecology.

[41]  J. Matthews,et al.  Small clinical trials: are they all bad? , 1995, Statistics in medicine.

[42]  T C Chalmers,et al.  Cumulative meta-analysis of therapeutic trials for myocardial infarction. , 1992, The New England journal of medicine.

[43]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[44]  C. Coulam,et al.  Update on Immunotherapy for Recurrent Pregnancy Loss , 1992, American journal of reproductive immunology.

[45]  D. Cook,et al.  Methodologic guidelines for systematic reviews of randomized control trials in health care from the Potsdam Consultation on Meta-Analysis. , 1995, Journal of clinical epidemiology.

[46]  S. Pocock,et al.  Can meta-analyses be trusted? , 1991, The Lancet.

[47]  S G Thompson,et al.  Controversies in meta-analysis: the case of the trials of serum cholesterol reduction , 1993, Statistical methods in medical research.

[48]  J. Lau,et al.  Meta-analytic stimulus for changes in clinical trials , 1993, Statistical methods in medical research.

[49]  R. Simes,et al.  Confronting publication bias: a cohort design for meta-analysis. , 1987, Statistics in medicine.

[50]  P. Wortman,et al.  Inborn/outborn status and neonatal survival: a meta-analysis of non-randomized studies. , 1988, Statistics in medicine.

[51]  J. Fleiss.  Review papers : The statistical basis of meta-analysis , 1993 .

[52]  C. Hsieh,et al.  Immunotherapy for Recurrent Spontaneous Abortions in a Chinese Population , 1991, American journal of reproductive immunology.

[53]  M. Bracken,et al.  Clinically useful measures of effect in binary analyses of randomized trials. , 1994, Journal of clinical epidemiology.

[54]  H. Rockette,et al.  Limitations and advantages of meta-analysis in clinical trials. , 1988, Recent results in cancer research. Fortschritte der Krebsforschung. Progres dans les recherches sur le cancer.

[55]  U. Molander.  Urinary incontinence and related urogenital symptoms in elderly women , 1993, Acta obstetricia et gynecologica Scandinavica. Supplement.