The False-positive to False-negative Ratio in Epidemiologic Studies

The ratio of false-positive to false-negative findings (FP:FN ratio) is an informative metric that warrants further evaluation. The FP:FN ratio varies greatly across different epidemiologic areas. In genetic epidemiology, it has varied from very high values (possibly even >100:1) for associations reported in candidate-gene studies to very low values (1:100 or lower) for associations with genome-wide significance. The substantial reduction over time in the FP:FN ratio in human genome epidemiology has corresponded to the routine adoption of stringent inferential criteria and comprehensive, agnostic reporting of all analyses. Most traditional fields of epidemiologic research more closely follow the practices of past candidate gene epidemiology, and thus have high FP:FN ratios. Further, FP and FN results do not necessarily entail the same consequences, and their relative importance may vary in different settings. This ultimately has implications for what is the acceptable FP:FN ratio and for how the results of published epidemiologic studies should be presented and interpreted.

[1]  P. Terpstra,et al.  Poor replication of candidate genes for major depressive disorder using genome-wide association data , 2011, Molecular Psychiatry.

[2]  J. Ioannidis,et al.  Genome-wide significant associations for variants with minor allele frequency of 5% or less--an overview: A HuGE review. , 2010, American journal of epidemiology.

[3]  Nilanjan Chatterjee,et al.  Estimation of effect size distribution from genome-wide association studies and implications for future discoveries , 2010, Nature Genetics.

[4]  G. Gibson Hints of hidden heritability in GWAS , 2010, Nature Genetics.

[5]  P. Visscher,et al.  Common SNPs explain a large proportion of heritability for human height , 2011 .

[6]  Ayellet V. Segrè,et al.  Twelve type 2 diabetes susceptibility loci identified through large-scale association analysis , 2010, Nature Genetics.

[7]  S. Young Acknowledge and fix the multiple testing problem. , 2010, International journal of epidemiology.

[8]  Atul J. Butte,et al.  An Environment-Wide Association Study (EWAS) on Type 2 Diabetes Mellitus , 2010, PloS one.

[9]  J. Ioannidis,et al.  Replication of past candidate loci for common diseases and phenotypes in 100 genome-wide association studies , 2010, European Journal of Human Genetics.

[10]  S. London,et al.  Evaluation of candidate genes in a genome-wide association study of childhood asthma in Mexicans. , 2010, The Journal of allergy and clinical immunology.

[11]  J. Hirschhorn Genomewide association studies--illuminating biologic pathways. , 2009, The New England journal of medicine.

[12]  John P A Ioannidis,et al.  Researching Genetic Versus Nongenetic Determinants of Disease: A Comparison and Proposed Unification , 2009, Science Translational Medicine.

[13]  Peter Kraft,et al.  Replication in genome-wide association studies. , 2009, Statistical science : a review journal of the Institute of Mathematical Statistics.

[14]  D. Kiel,et al.  Collaborative Meta-analysis: Associations of 150 Candidate Genes With Osteoporosis and Osteoporotic Fracture , 2009, Annals of Internal Medicine.

[15]  Paolo Vineis,et al.  Epidemiology, Public Health, and the Rhetoric of False Positives , 2009, Environmental health perspectives.

[16]  High marks for GWAS , 2009, Nature Genetics.

[17]  Christian Gieger,et al.  Targeting 160 Candidate Genes for Blood Pressure Regulation with a Genome-Wide Genotyping Array , 2009, PloS one.

[18]  F. Collins,et al.  Potential etiologic and functional implications of genome-wide association loci for human diseases and traits , 2009, Proceedings of the National Academy of Sciences.

[19]  Siobhan M. Dolan,et al.  Genome-Wide Association Studies, Field Synopses, and the Development of the Knowledge Base on Genetic Variation and Human Diseases , 2009, American journal of epidemiology.

[20]  John P. A. Ioannidis,et al.  Validating, augmenting and refining genome-wide association signals , 2009, Nature Reviews Genetics.

[21]  W. Maziak Point-counterpoint. The triumph of the null hypothesis: epidemiology in an age of change. , 2009, International journal of epidemiology.

[22]  William Wheeler,et al.  Genome-Wide and Candidate Gene Association Study of Cigarette Smoking Behaviors , 2009, PloS one.

[23]  John P A Ioannidis,et al.  Meta-analysis in genome-wide association studies. , 2009, Pharmacogenomics.

[24]  Sally Hopewell,et al.  Publication bias in clinical trials due to statistical significance or direction of trial results. , 2009, The Cochrane database of systematic reviews.

[25]  James M. Robins,et al.  Observational Studies Analyzed Like Randomized Experiments: An Application to Postmenopausal Hormone Therapy and Coronary Heart Disease , 2008, Epidemiology.

[26]  J. Ioannidis Why Most Discovered True Associations Are Inflated , 2008, Epidemiology.

[27]  J. Ioannidis,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias , 2008, PloS one.

[28]  Paolo Boffetta,et al.  False-Positive Results in Cancer Epidemiology: A Plea for Epistemological Modesty , 2008, Journal of the National Cancer Institute.

[29]  Iztok Hozo,et al.  Decision-making when data and inferences are not conclusive: risk-benefit and acceptable regret approach. , 2008, Seminars in hematology.

[30]  I. Hozo,et al.  When Is Diagnostic Testing Inappropriate or Irrational? Acceptable Regret Approach , 2008, Medical decision making : an international journal of the Society for Medical Decision Making.

[31]  M. Daly,et al.  Estimation of the multiple testing burden for genomewide association studies of nearly all common variants , 2008, Genetic epidemiology.

[32]  M. McCarthy,et al.  Meta-analysis of genome-wide association data and large-scale replication identifies additional susceptibility loci for type 2 diabetes , 2008, Nature Genetics.

[33]  Jan P Vandenbroucke,et al.  Observational Research, Randomised Trials, and Two Views of Medical Science , 2008, PLoS medicine.

[34]  C. Hoggart,et al.  Genome‐wide significance for dense SNP and resequencing data , 2008, Genetic epidemiology.

[35]  Ronald E. Doel,et al.  :The Honest Broker: Making Sense of Science in Policy and Politics , 2008 .

[36]  J. Ioannidis,et al.  Persistence of contradicted claims in the literature. , 2007, JAMA.

[37]  John P A Ioannidis,et al.  Almost all articles on cancer prognostic markers report statistically significant results. , 2007, European journal of cancer.

[38]  C. Gieger,et al.  Genomewide association analysis of coronary artery disease. , 2007, The New England journal of medicine.

[39]  M. McCarthy,et al.  Replication of Genome-Wide Association Signals in UK Samples Reveals Risk Loci for Type 2 Diabetes , 2007, Science.

[40]  James Strait,et al.  Genome-Wide Association Scan Shows Genetic Variants in the FTO Gene Are Associated with Obesity-Related Traits , 2007, PLoS genetics.

[41]  J. Pielke The Honest Broker: Making Sense of Science in Policy and Politics , 2007 .

[42]  Harlan M Krumholz,et al.  Nonvalidation of reported genetic risk factors for acute coronary syndrome in a large-scale replication study. , 2007, JAMA.

[43]  George Liberopoulos,et al.  Selection in Reported Epidemiological Risks: An Empirical Assessment , 2007, PLoS medicine.

[44]  Iztok Hozo,et al.  When Should Potentially False Research Findings Be Considered Acceptable? , 2007, PLoS medicine.

[45]  Peter C Austin,et al.  Testing multiple statistical hypotheses resulted in spurious associations: a study of astrological signs and health. , 2006, Journal of clinical epidemiology.

[46]  Garnet L Anderson,et al.  Statistical Issues Arising in the Women's Health Initiative , 2005, Biometrics.

[47]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[48]  J. Ioannidis Contradicted and initially stronger effects in highly cited clinical research. , 2005, JAMA.

[49]  Thomas A Trikalinos,et al.  Early extreme contradictory estimates may appear in published research: the Proteus phenomenon in molecular genetics research and randomized trials. , 2005, Journal of clinical epidemiology.

[50]  M. A. Best Bayesian Approaches to Clinical Trials and Health‐Care Evaluation , 2005 .

[51]  Carl V Phillips,et al.  Publication bias in situ , 2004 .

[52]  Looking to the 21st century: have we learned from our mistakes, or are we doomed to compound them? , 2004, Pharmacoepidemiology and drug safety.

[53]  Nathaniel Rothman,et al.  Assessing the Probability That a Positive Report is False: An Approach for Molecular Epidemiology Studies , 2004 .

[54]  Elio Riboli,et al.  Epidemiologic evidence of the protective effect of fruit and vegetables on cancer risk. , 2003, The American journal of clinical nutrition.

[55]  P. McKeigue,et al.  For Personal Use. Only Reproduce with Permission from the Lancet Publishing Group. Problems of Reporting Genetic Associations with Complex Outcomes , 2022 .

[56]  E. Lander,et al.  Meta-analysis of genetic association studies supports a contribution of common variants to susceptibility to common disease , 2003, Nature Genetics.

[57]  S. Ebrahim,et al.  Data dredging, bias, or confounding , 2002, BMJ : British Medical Journal.

[58]  J. Sterne,et al.  What is newsworthy? Longitudinal study of the reporting of medical research in two British newspapers , 2002, BMJ : British Medical Journal.

[59]  J. Hirschhorn,et al.  A comprehensive review of genetic association studies , 2002, Genetics in Medicine.

[60]  Jonathan A C Sterne,et al.  Sifting the evidence—what's wrong with significance tests? , 2001, BMJ : British Medical Journal.

[61]  Steven Goodman Toward Evidence-Based Medical Statistics. 2: The Bayes Factor , 1999, Annals of Internal Medicine.

[62]  M. Stampfer Observational epidemiology is the preferred means of evaluating effects of behavioral and lifestyle modification. , 1997, Controlled Clinical Trials.

[63]  P. Skrabanek Has risk-factor epidemiology outlived its usefulness? , 1993, American journal of epidemiology.

[64]  A R Feinstein,et al.  Scientific standards in epidemiologic studies of the menace of daily life. , 1988, Science.

[65]  R. Merton The Normative Structure of Science , 1973 .