False-Positive Results in Cancer Epidemiology: A Plea for Epistemological Modesty

False-positive results are inherent in the scientific process of testing hypotheses concerning the determinants of cancer and other human illnesses. Although much of what is known about the etiology of human cancers has arisen from well-conducted epidemiological studies, epidemiology has been increasingly criticized for producing findings that are often sensationalized in the media and fail to be upheld in subsequent studies. Herein we describe examples from cancer epidemiology of likely false-positive findings and discuss conditions under which such results may occur. We suggest general guidelines or principles, including the endorsement of editorial policies requiring the prominent listing of study caveats, which may help reduce the reporting of misleading results. Increased epistemological humility regarding findings in epidemiology would go a long way to diminishing the detrimental effects of false-positive results on the allocation of limited research resources, on the advancement of knowledge of the causes and prevention of cancer, and on the scientific reputation of epidemiology and would help to prevent oversimplified interpretations of results by the media and the public.

[1]  C. Phillips Commentary: Lack of scientific influences on epidemiology. , 2008, International journal of epidemiology.

[2]  P. McKeigue,et al.  Problems of reporting genetic associations with complex outcomes , 2003, The Lancet.

[3]  J. Ioannidis Why Most Published Research Findings Are False , 2005 .

[4]  A. Hartz,et al.  A comparison of observational studies and randomized, controlled trials. , 2000, The New England journal of medicine.

[5]  S. Pocock,et al.  Guideline The Strengthening the Reporting of Observational Studies in Epidemiology ( STROBE ) Statement : Guidelines for reporting observational studies * , 2014 .

[6]  T. Holford,et al.  1,1-Dichloro-2,2-bis(p-chlorophenyl)ethylene and polychlorinated biphenyls and breast cancer: combined analysis of five U.S. studies. , 2001, Journal of the National Cancer Institute.

[7]  K. Mossman The Wellcome Trust Case Control Consortium, U.K. , 2008 .

[8]  P. Brennan,et al.  Occupational Exposure to Vinyl Chloride, Acrylonitrile and Styrene and Lung Cancer Risk (Europe) , 2004, Cancer Causes & Control.

[9]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[10]  Shinto Eguchi,et al.  Confidence Intervals and P‐Values for Meta‐Analysis with Publication Bias , 2007, Biometrics.

[11]  J. Sterne,et al.  Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. , 2000, Journal of clinical epidemiology.

[12]  M. Wolff,et al.  DDT and Breast Cancer in Young Women: New Data on the Significance of Age at Exposure , 2007, Environmental health perspectives.

[13]  Francesco Forastiere,et al.  Methodological issues regarding confounding and exposure misclassification in epidemiological studies of occupational exposures. , 2007, American journal of industrial medicine.

[14]  D. Trichopoulos The future of epidemiology. , 1996, BMJ.

[15]  J. Tickner,et al.  Categorizing Mistaken False Positives in Regulation of Human and Environmental Health , 2007, Risk analysis : an official publication of the Society for Risk Analysis.

[16]  V. McCormack,et al.  Issues in the reporting of epidemiological studies: a survey of recent practice , 2004, BMJ : British Medical Journal.

[17]  N. Pearce Corporate influences on epidemiology. , 2008, International journal of epidemiology.

[18]  G. Mastrangelo,et al.  Mortality from tumours in workers in an acrylic fibre factory. , 1993, Occupational medicine.

[19]  M. O'berg Epidemiologic study of workers exposed to acrylonitrile. , 1980, Journal of occupational medicine. : official publication of the Industrial Medical Association.

[20]  J. Ioannidis,et al.  Comparison of evidence of treatment effects in randomized and nonrandomized studies. , 2001, JAMA.

[21]  John Hoey,et al.  Is this clinical trial fully registered? A statement from the International Committee of Medical Journal Editors , 2005, The Lancet.

[22]  John Hoey,et al.  Is this clinical trial fully registered?--A statement from the International Committee of Medical Journal Editors. , 2005, The New England journal of medicine.

[23]  G. Bond,et al.  Update of mortality among chemical workers with potential exposure to the higher chlorinated dioxins. , 1989, Journal of occupational medicine. : official publication of the Industrial Medical Association.

[24]  J. Godbold,et al.  Pesticides and polychlorinated biphenyl residues in human breast lipids and their relation to breast cancer. , 1992, Archives of environmental health.

[25]  P. Crosignani Re: False-positive results in cancer epidemiology: a plea for epistemological modesty. , 2009, Journal of the National Cancer Institute.

[26]  N. Dubin,et al.  Risk of breast cancer and organochlorine exposure. , 2000, Cancer epidemiology, biomarkers & prevention : a publication of the American Association for Cancer Research, cosponsored by the American Society of Preventive Oncology.

[27]  S. Parodi,et al.  Comparison Bias and Dilution Effect in Occupational Cohort Studies , 2007, International journal of occupational and environmental health.

[28]  G A Diamond,et al.  Clinical trials and statistical verdicts: probable grounds for appeal. , 1983, Annals of internal medicine.

[29]  N. Rothman,et al.  Serum organochlorine pesticides and PCBs and breast cancer risk: results from a prospective analysis (USA) , 1999, Cancer Causes & Control.

[30]  D. Hunter,et al.  Pesticide residues and breast cancer: the harvest of a silent spring? , 1993, Journal of the National Cancer Institute.

[31]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[32]  L. V. van Amelsvoort,et al.  False positive outcomes and design characteristics in occupational cancer epidemiology studies. , 2001, International journal of epidemiology.

[33]  J. Ioannidis,et al.  An exploratory test for an excess of significant findings , 2007, Clinical trials.

[34]  T. Benn,et al.  Mortality of United Kingdom acrylonitrile workers--an extended and updated study. , 1998, Scandinavian journal of work, environment & health.

[35]  Thomas A Trikalinos,et al.  Genetic associations in large versus small studies: an empirical assessment , 2003, The Lancet.

[36]  P. A. Honchar,et al.  Cancer mortality in workers exposed to 2,3,7,8-tetrachlorodibenzo-p-dioxin. , 1991, The New England journal of medicine.

[37]  A. Zuckerman,et al.  IARC Monographs on the Evaluation of Carcinogenic Risks to Humans , 1995, IARC monographs on the evaluation of carcinogenic risks to humans.

[38]  D. Christiani,et al.  Prevention of Work-Related Cancers , 2002, New solutions : a journal of environmental and occupational health policy : NS.

[39]  V. Chinchilli,et al.  Induced Abortion and Risk for Breast Cancer: Reporting (Recall) Bias in a Dutch Case-Control Study , 1996 .

[40]  L. Tomatis,et al.  Organochlorine compounds in neoplastic and adjacent apparently normal breast tissue , 1976, Bulletin of environmental contamination and toxicology.

[41]  R. Frentzel-Beyme,et al.  [Mortality study in chemical personnel of various industries exposed to acrylonitrile]. , 1980, Zentralblatt fur Arbeitsmedizin, Arbeitsschutz, Prophylaxe und Ergonomie.

[42]  J. B. Werner,et al.  Mortality of United Kingdom acrylonitrile polymerisation workers. , 1981, British journal of industrial medicine.

[43]  G. Taubes Epidemiology faces its limits. , 1995, Science.

[44]  S. Pocock,et al.  Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and Elaboration , 2007, PLoS medicine.

[45]  B. Macmahon,et al.  Coffee and cancer of the pancreas. , 1981, The New England journal of medicine.

[46]  F. V. van Leeuwen,et al.  Induced abortion and risk for breast cancer: reporting (recall) bias in a Dutch case-control study. , 1996, Journal of the National Cancer Institute.

[47]  J. Ioannidis Contradicted and initially stronger effects in highly cited clinical research. , 2005, JAMA.

[48]  J. Farman,et al.  Late lessons from early warnings: the precautionary principle 1896-2000 , 2002 .

[49]  P. Grandjean,et al.  Organochlorine exposure and risk of breast cancer , 1998, The Lancet.

[50]  N Dubin,et al.  Blood levels of organochlorine residues and risk of breast cancer. , 1993, Journal of the National Cancer Institute.

[51]  Collins Jj,et al.  Mortality patterns among employees exposed to acrylonitrile. , 1989, Journal of occupational medicine. : official publication of the Industrial Medical Association.

[52]  L. Tomatis,et al.  Business Bias: How Epidemiologic Studies May Underestimate or Fail to Detect Increased Risks of Cancer and Other Diseases , 2005, International journal of occupational and environmental health.

[53]  C. Begg,et al.  Operating characteristics of a rank correlation test for publication bias. , 1994, Biometrics.

[54]  Education Welfare.,et al.  Smoking and Health. Report of the Advisory Committee to the Surgeon General of the Public Health Service. , 1964 .

[55]  S. Goldstein,et al.  Surgical results of anomalous left coronary artery. , 1983, Archives of internal medicine.

[56]  Geoffrey C Kabat,et al.  Environmental toxins and breast cancer on Long Island. II. Organochlorine compound levels in blood. , 2002, Cancer epidemiology, biomarkers & prevention : a publication of the American Association for Cancer Research, cosponsored by the American Society of Preventive Oncology.

[57]  John A. Todd,et al.  Parameters for reliable results in genetic association studies in common disease , 2002, Nature Genetics.

[58]  Jonathan A C Sterne,et al.  The impact of residual and unmeasured confounding in epidemiologic studies: a simulation study. , 2007, American journal of epidemiology.

[59]  P. Buffler,et al.  Mortality and morbidity of workers exposed to acrylonitrile in fiber production. , 1998, Scandinavian journal of work, environment & health.

[60]  B C Amick,et al.  Relationship of job strain and iso-strain to health status in a cohort of women in the United States. , 1998, Scandinavian journal of work, environment & health.

[61]  Nathaniel Rothman,et al.  Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. , 2004, Journal of the National Cancer Institute.

[62]  Simon C. Potter,et al.  Genome-wide association study of 14,000 cases of seven common diseases and 3,000 shared controls , 2007, Nature.

[63]  R. Hiatt,et al.  Breast cancer and serum organochlorines: a prospective study among white, black, and Asian women. , 1994, Journal of the National Cancer Institute.

[64]  D. Mirabelli,et al.  'Environment' in cancer causation and etiological fraction: limitations and ambiguities. , 2006, Carcinogenesis.

[65]  M. O'berg,et al.  Cancer incidence and mortality among workers exposed to acrylonitrile. , 1987, American journal of industrial medicine.

[66]  R. Collins,et al.  Blood pressure, stroke, and coronary heart disease Part 2, short-term reductions in blood pressure: overview of randomised drug trials in their epidemiological context , 1990, The Lancet.

[67]  R. Collins,et al.  Blood pressure, stroke, and coronary heart disease Part 1, prolonged differences in blood pressure: prospective observational studies corrected for the regression dilution bias , 1990, The Lancet.

[68]  Jon Wakefield,et al.  A Bayesian measure of the probability of false discovery in genetic epidemiology studies. , 2007, American journal of human genetics.

[69]  V. Beral,et al.  Breast cancer and abortion: collaborative reanalysis of data from 53 epidemiological studies, including 83 000 women with breast cancer from 16 countries , 2004, The Lancet.

[70]  J. Ioannidis Why Most Published Research Findings Are False , 2019, CHANCE.

[71]  Coffee, tea, mate, methylxanthines and methylglyoxal. IARC Working Group on the Evaluation of Carcinogenic Risks to Humans. Lyon, 27 February to 6 March 1990. , 1991, IARC monographs on the evaluation of carcinogenic risks to humans.

[72]  J. Manson,et al.  Plasma organochlorine levels and the risk of breast cancer. , 1997, The New England journal of medicine.

[73]  M. O'berg,et al.  Epidemiologic study of workers exposed to acrylonitrile: an update. , 1985, Journal of occupational medicine. : official publication of the Industrial Medical Association.

[74]  J. Brock,et al.  Serum concentrations of organochlorine compounds and the subsequent development of breast cancer. , 1999, Cancer epidemiology, biomarkers & prevention : a publication of the American Association for Cancer Research, cosponsored by the American Society of Preventive Oncology.

[75]  S. Levin,et al.  False Alarm over Environmental False Alarms , 2003, Science.

[76]  P. Boyle,et al.  Risk of female breast cancer associated with serum polychlorinated biphenyls and 1,1-dichloro-2,2'-bis(p-chlorophenyl)ethylene. , 2000, Cancer epidemiology, biomarkers & prevention : a publication of the American Association for Cancer Research, cosponsored by the American Society of Preventive Oncology.

[77]  J. Ioannidis,et al.  Replication validity of genetic association studies , 2001, Nature Genetics.

[78]  A. Andersen,et al.  Serum organochlorine levels and breast cancer: a nested case-control study of Norwegian women. , 2000, Cancer epidemiology, biomarkers & prevention : a publication of the American Association for Cancer Research, cosponsored by the American Society of Preventive Oncology.

[79]  J. Blondell Organochlorine exposures and breast cancer risk in New York City women. , 2002, Environmental research.

[80]  S. Pocock,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. , 2007, Preventive medicine.

[81]  Thomas A Trikalinos,et al.  Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. , 2005, Journal of clinical epidemiology.

[82]  R. Monson,et al.  Mortality among rubber workers: VI. Men with potential exposure to acrylonitrile. , 1982, Journal of occupational medicine. : official publication of the Industrial Medical Association.

[83]  L. Brinton,et al.  The epidemiology of cervical carcinogenesis , 1995, Cancer.

[84]  Peter C Gøtzsche,et al.  Believability of relative risks and odds ratios in abstracts: cross sectional study , 2006, BMJ : British Medical Journal.

[85]  K. Popper,et al.  The Logic of Scientific Discovery , 1960 .

[86]  F. Speizer,et al.  Plasma organochlorine levels and the risk of breast cancer: An extended follow‐up in the Nurses' Health Study , 2001, International journal of cancer.

[87]  J. Clausen,et al.  Organochlorine compounds in human breast fat from deceased with and without breast cancer and in a biopsy material from newly diagnosed patients undergoing breast surgery. , 1984, Environmental research.

[88]  J. Freudenheim,et al.  Environmental organochlorine exposure and postmenopausal breast cancer risk. , 1998, Cancer epidemiology, biomarkers & prevention : a publication of the American Association for Cancer Research, cosponsored by the American Society of Preventive Oncology.

[89]  F. Sturmans,et al.  Mortality of workers exposed to acrylonitrile. , 1992, Journal of occupational medicine. : official publication of the Industrial Medical Association.

[90]  G. Eklund,et al.  Response bias in a case-control study: analysis utilizing comparative data concerning legal abortions from two independent Swedish studies. , 1991, American journal of epidemiology.

[91]  G. Swaen,et al.  Mortality Update of Workers Exposed to Acrylonitrile in The Netherlands , 2004, Scandinavian journal of work, environment & health.

[92]  H. Mussalo-Rauhamaa,et al.  Occurrence of beta‐hexachlorocyclohexane in breast cancer patients , 1990, Cancer.

[93]  S. Yufit Role of electronic factors in the formation of a standard, quasi-stable mixture of toxic polychlorinated dibenzo-para-dioxins and polychlorinated dibenzofurans , 1999 .

[94]  J. Fraumeni,et al.  Cancer epidemiology and prevention. , 2006 .

[95]  E. J. Schneider,et al.  A mortality survey of employees engaged in the development or manufacture of styrene-based products. , 1980, Journal of occupational medicine. : official publication of the Industrial Medical Association.

[96]  J. Lubin,et al.  Mortality of industrial workers exposed to acrylonitrile. , 1998, Scandinavian journal of work, environment & health.

[97]  Y. Benjamini,et al.  Quantitative Trait Loci Analysis Using the False Discovery Rate , 2005, Genetics.

[98]  P. Boyle,et al.  Epidemiology of pancreas cancer (1988) , 1989, International journal of pancreatology : official journal of the International Association of Pancreatology.