Metrics to find a surrogate endpoint of OS in metastatic oncology trials: a simulation study

Surrogate endpoint (SE) for overall survival (OS) in cancer patients is essential to improving the efficiency of oncology drug development. In practice, we may discover a new patient level association with OS in a discovery cohort, and then measure the trial level association across studies in a meta-analysis to validate the SE. In this work, we simulated pairs of metrics to quantify the surrogacy at the patient level and the trial level and evaluated their association, and to understand how well various patient level metrics from the initial discovery would indicate the eventual utility as a SE. Across all the simulation scenarios, we found tight correlation among all the patient level metrics, including C index, integrated brier score and log hazard ratio between SE values and OS; and similar correlation between any of them and the trial level association metric. Despite the continual increase in the true biological link between SE and OS, both patient and trial level metrics often plateaued coincidentally in many scenarios; their association always decreased quickly. Under the SE development framework and data generation models considered here, all patient level metrics are similar in ranking a candidate SE according to its eventual trial level association; incorporating additional biological factors into a SE are likely to have diminished return in improving both patient level and trial level association.

[1]  Ash A. Alizadeh,et al.  Circulating Tumor DNA Measurements As Early Outcome Predictors in Diffuse Large B-Cell Lymphoma. , 2018, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[2]  N. Obuchowski,et al.  Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[3]  Michael J. Crowther,et al.  Simulating Survival Data Using the simsurv R Package , 2021, J. Stat. Softw..

[4]  Patrick Royston,et al.  Discrimination-based sample size calculations for multivariable prognostic models for time-to-event data , 2015, BMC Medical Research Methodology.

[5]  J. Ware The limitations of risk factors as prognostic tools. , 2006, The New England journal of medicine.

[6]  M. Pepe,et al.  Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. , 2004, American journal of epidemiology.

[7]  Hemant Ishwaran,et al.  Evaluating Random Forests for Survival Analysis using Prediction Error Curves. , 2012, Journal of statistical software.

[8]  Michael J. Pencina,et al.  Understanding increments in model performance metrics , 2013, Lifetime data analysis.

[9]  Geert Molenberghs,et al.  The validation of surrogate end points by using data from randomized clinical trials: a case‐study in advanced colorectal cancer , 2004 .

[10]  J. Somberg,et al.  Biomarkers and surrogate endpoints. , 1999, American journal of therapeutics.

[11]  J R Anderson,et al.  Analysis of survival by tumor response. , 1983, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[12]  S. Bates,et al.  Analyzing the Pivotal Trial That Compared Sunitinib and IFN-α in Renal Cell Carcinoma, Using a Method That Assesses Tumor Regression and Growth , 2012, Clinical Cancer Research.

[13]  T. Fojo,et al.  Tumor Regression and Growth Rates Determined in Five Intramural NCI Prostate Cancer Trials: The Growth Rate Constant as an Indicator of Therapeutic Efficacy , 2010, Clinical Cancer Research.

[14]  J. Soria,et al.  A Model of Overall Survival Predicts Treatment Outcomes with Atezolizumab versus Chemotherapy in Non–Small Cell Lung Cancer Based on Early Tumor Kinetics , 2018, Clinical Cancer Research.

[15]  P. Heagerty,et al.  Survival Model Predictive Accuracy and ROC Curves , 2005, Biometrics.

[16]  P. Dennis,et al.  Relationship Between Progression‐Free Survival, Objective Response Rate, and Overall Survival in Clinical Trials of PD‐1/PD‐L1 Immune Checkpoint Blockade: A Meta‐Analysis , 2020, Clinical pharmacology and therapeutics.

[17]  A. Gupta,et al.  Early Reduction in ctDNA Predicts Survival in Patients with Lung and Bladder Cancer Treated with Durvalumab , 2018, Clinical Cancer Research.

[18]  E Graf,et al.  Assessment and comparison of prognostic classification schemes for survival data. , 1999, Statistics in medicine.

[19]  F. Harrell,et al.  Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors , 2005 .

[20]  Patrick Royston,et al.  A new measure of prognostic separation in survival data , 2004, Statistics in medicine.

[21]  A. Bauer-Mehren,et al.  An Enhanced Prognostic Score for Overall Survival of Patients with Cancer Derived from a large Real World Cohort. , 2020, Annals of oncology : official journal of the European Society for Medical Oncology.

[22]  Erich P Huang,et al.  RECIST 1.1-Update and clarification: From the RECIST committee. , 2016, European journal of cancer.

[23]  Sean Khozin,et al.  Overall response rate, progression-free survival, and overall survival with targeted and standard therapies in advanced non-small-cell lung cancer: US Food and Drug Administration trial-level and patient-level analyses. , 2015, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[24]  Paul Blanche,et al.  An R2‐curve for evaluating the accuracy of dynamic predictions , 2018, Statistics in medicine.

[25]  G. Molenberghs,et al.  The validation of surrogate endpoints in meta-analyses of randomized experiments. , 2000, Biostatistics.

[26]  Liang Li,et al.  DYNAMIC PREDICTION FOR MULTIPLE REPEATED MEASURES AND EVENT TIME DATA: AN APPLICATION TO PARKINSON'S DISEASE. , 2016, The annals of applied statistics.

[27]  Bradley P. Carlin,et al.  Combining Dynamic Predictions From Joint Models for Longitudinal and Time-to-Event Data Using Bayesian Model Averaging , 2013, 1303.2797.

[28]  Matthias Schmid,et al.  Boosting the Concordance Index for Survival Data – A Unified Framework To Derive and Evaluate Biomarker Combinations , 2013, PloS one.

[29]  Martin Schumacher,et al.  An easy mathematical proof showed that time-dependent bias inevitably leads to biased effect estimation. , 2008, Journal of clinical epidemiology.

[30]  D. Sargent,et al.  Biomarkers and surrogate end points—the challenge of statistical validation , 2010, Nature Reviews Clinical Oncology.

[31]  Antoni Ribas,et al.  The “cancer immunogram” , 2016, Science.

[32]  Vinay Prasad,et al.  Cancer Drugs Approved on the Basis of a Surrogate End Point and Subsequent Overall Survival: An Analysis of 5 Years of US Food and Drug Administration Approvals. , 2015, JAMA internal medicine.