Why your new cancer biomarker may never work: recurrent patterns and remarkable diversity in biomarker failures.

Less than 1% of published cancer biomarkers actually enter clinical practice. Although best practices for biomarker development are published, optimistic investigators may not appreciate the statistical near-certainty and diverse modes by which the other 99% (likely including your favorite new marker) do indeed fail. Here, patterns of failure were abstracted for classification from publications and an online database detailing marker failures. Failure patterns formed a hierarchical logical structure, or outline, of an emerging, deeply complex, and arguably fascinating science of biomarker failure. A new cancer biomarker under development is likely to have already encountered one or more of the following fatal features encountered by prior markers: lack of clinical significance, hidden structure in the source data, a technically inadequate assay, inappropriate statistical methods, unmanageable domination of the data by normal variation, implausibility, deficiencies in the studied population or in the investigator system, and its disproof or abandonment for cause by others. A greater recognition of the science of biomarker failure and its near-complete ubiquity is constructive and celebrates a seemingly perpetual richness of biologic, technical, and philosophical complexity, the full appreciation of which could improve the management of scarce research resources.

[1]  A. Dupuy,et al.  Critical review of published microarray studies for cancer outcome and guidelines on statistical analysis and reporting. , 2007, Journal of the National Cancer Institute.

[2]  V. Beneš,et al.  The MIQE guidelines: minimum information for publication of quantitative real-time PCR experiments. , 2009, Clinical chemistry.

[3]  M. Radmacher,et al.  Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. , 2003, Journal of the National Cancer Institute.

[4]  Thorsten Henrich,et al.  Minimum information specification for in situ hybridization and immunohistochemistry experiments (MISFISHIE) , 2008, Nature Biotechnology.

[5]  G. Omenn,et al.  Evolution of Translational Omics: Lessons Learned and the Path Forward , 2013 .

[6]  E. Diamandis.  Analysis of serum proteomic patterns for early cancer diagnosis: drawing attention to potential problems. , 2004, Journal of the National Cancer Institute.

[7]  D. Berry.  Multiplicities in cancer research: ubiquitous and necessary evils. , 2012, Journal of the National Cancer Institute.

[8]  M S Pepe,et al.  Phases of biomarker development for early detection of cancer. , 2001, Journal of the National Cancer Institute.

[9]  David Moher,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. , 2003, Clinical chemistry.

[10]  S. Kern,et al.  Stagnation and herd mentality in the biomedical sciences , 2004, Cancer biology & therapy.

[11]  J. Brooks.  Why most published research findings are false: Ioannidis JP, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece , 2008 .

[12]  E. Diamandis,et al.  The failure of protein cancer biomarkers to reach the clinic: why, and what can be done to address the problem? , 2012, BMC Medicine.

[13]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[14]  Biomarker studies and other difficult inferential problems: statistical caveats. , 2007, Seminars in oncology.

[15]  H. Welch,et al.  Overdiagnosis in cancer. , 2010, Journal of the National Cancer Institute.

[16]  D. Rennie,et al.  The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. , 2003, Annals of internal medicine.

[17]  Jocelyn Kaiser,et al.  Clinical medicine. Biomarker tests need closer scrutiny, IOM concludes. , 2012, Science.

[18]  Sudhir Srivastava,et al.  Ovarian Cancer Biomarker Performance in Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial Specimens , 2011, Cancer Prevention Research.

[19]  D. Ransohoff.  Bias as a threat to the validity of cancer molecular-marker research , 2005, Nature reviews. Cancer.

[20]  E. Diamandis,et al.  Mining the Ovarian Cancer Ascites Proteome for Potential Ovarian Cancer Biomarkers*S , 2009, Molecular & Cellular Proteomics.

[21]  John P. A. Ioannidis,et al.  An empirical assessment of validation practices for molecular classifiers , 2011, Briefings Bioinform..

[22]  David Moher,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. , 2003, Radiology.

[23]  David F Ransohoff,et al.  Evaluating discovery-based research: when biologic reasoning cannot work. , 2004, Gastroenterology.

[24]  E. Diamandis,et al.  Cancer biomarkers: can we turn recent failures into success? , 2010, Journal of the National Cancer Institute.

[25]  Christine M. Micheel,et al.  COMMITTEE ON THE REVIEW OF OMICS-BASED TESTS FOR PREDICTING PATIENT OUTCOMES IN CLINICAL TRIALS , 2012 .

[26]  Douglas G Altman,et al.  Reporting recommendations for tumor marker prognostic studies. , 2005, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[27]  Clive R Taylor,et al.  Recommendations for Improved Standardization of Immunohistochemistry , 2007, Applied immunohistochemistry & molecular morphology : AIMM.

[28]  D. Berry,et al.  Cancer and Leukemia Group B Pathology Committee guidelines for tissue microarray construction representing multicenter prospective clinical trial tissues. , 2011, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[29]  Siri J. Carpenter.  Psychology research. Psychology's bold initiative. , 2012, Science.

[30]  Raymond Vanholder,et al.  Implementation of proteomic biomarkers: making it work , 2012, European journal of clinical investigation.