Validation of Models Used to Inform Colorectal Cancer Screening Guidelines

Background. Microsimulation models synthesize evidence about disease processes and interventions, providing a method for predicting long-term benefits and harms of prevention, screening, and treatment strategies. Because models often require assumptions about unobservable processes, assessing a model’s predictive accuracy is important. Methods. We validated 3 colorectal cancer (CRC) microsimulation models against outcomes from the United Kingdom Flexible Sigmoidoscopy Screening (UKFSS) Trial, a randomized controlled trial that examined the effectiveness of one-time flexible sigmoidoscopy screening to reduce CRC mortality. The models incorporate different assumptions about the time from adenoma initiation to development of preclinical and symptomatic CRC. Analyses compare model predictions to study estimates across a range of outcomes to provide insight into the accuracy of model assumptions. Results. All 3 models accurately predicted the relative reduction in CRC mortality 10 years after screening (predicted hazard ratios, with 95% percentile intervals: 0.56 [0.44, 0.71], 0.63 [0.51, 0.75], 0.68 [0.53, 0.83]; estimated with 95% confidence interval: 0.56 [0.45, 0.69]). Two models with longer average preclinical duration accurately predicted the relative reduction in 10-year CRC incidence. Two models with longer mean sojourn time accurately predicted the number of screen-detected cancers. All 3 models predicted too many proximal adenomas among patients referred to colonoscopy. Conclusion. Model accuracy can only be established through external validation. Analyses such as these are therefore essential for any decision model. Results supported the assumptions that the average time from adenoma initiation to development of preclinical cancer is long (up to 25 years), and mean sojourn time is close to 4 years, suggesting the window for early detection and intervention by screening is relatively long. Variation in dwell time remains uncertain and could have important clinical and policy implications.

[1]  A. Zauber,et al.  A Comparison of the Cost-Effectiveness of Fecal Occult Blood Tests with Different Test Characteristics in the Context of Annual Screening in the Medicare Population , 2003 .

[2]  Carolyn M. Rutter,et al.  Clarifying Differences in Natural History between Models of Screening , 2011, Medical decision making : an international journal of the Society for Medical Decision Making.

[3]  Rob Boer,et al.  The MISCAN-COLON Simulation Model for the Evaluation of Colorectal Cancer Screening , 1999, Comput. Biomed. Res..

[4]  D. Lam-Himlin,et al.  Serrated Lesions of the Colorectum: Review and Recommendations From an Expert Panel , 2013 .

[5]  J. Savarino,et al.  Bayesian Calibration of Microsimulation Models , 2009, Journal of the American Statistical Association.

[6]  Chyke A Doubeni,et al.  Screening Colonoscopy and Risk for Incident Late-Stage Colorectal Cancer Diagnosis in Average-Risk Adults , 2013, Annals of Internal Medicine.

[7]  T. Stukel,et al.  Association between colonoscopy and colorectal cancer mortality in a US cohort according to site of cancer and colonoscopist specialty. , 2012, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[8]  William Hazelton,et al.  Benefits and Harms of Computed Tomography Lung Cancer Screening Strategies: A Comparative Modeling Study for the U.S. Preventive Services Task Force , 2014, Annals of Internal Medicine.

[9]  D. Early,et al.  Prevalence of missed adenomas in patients with inadequate bowel preparation on screening colonoscopy. , 2012, Gastrointestinal endoscopy.

[10]  Laurence L. George,et al.  The Statistical Analysis of Failure Time Data , 2003, Technometrics.

[11]  J. Wardle,et al.  Single flexible sigmoidoscopy screening to prevent colorectal cancer: baseline findings of a UK multicentre randomised trial , 2002, The Lancet.

[12]  P. Prorok,et al.  Evidence of a healthy volunteer effect in the prostate, lung, colorectal, and ovarian cancer screening trial. , 2007, American journal of epidemiology.

[13]  H. Brenner,et al.  Protection from right- and left-sided colorectal neoplasms after colonoscopy: population-based study. , 2010, Journal of the National Cancer Institute.

[14]  J. Banks,et al.  Disease prevalence, disease incidence, and mortality in the United States and in England , 2010, Demography.

[15]  U. P. S. T. Force,et al.  Screening for Colorectal Cancer: Recommendation and Rationale , 2002, Annals of Internal Medicine.

[16]  R. Lukes,et al.  Prevalence of undiagnosed cancer of the large bowel found at autopsy in different races , 1970, Cancer.

[17]  J. Habbema,et al.  Should colorectal cancer screening be considered in elderly persons without previous screening? A cost-effectiveness analysis. , 2014, Annals of internal medicine.

[18]  P. Bossuyt,et al.  Polyp Miss Rate Determined by Tandem Colonoscopy: A Systematic Review , 2006, The American Journal of Gastroenterology.

[19]  J. Caro,et al.  Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--7. , 2012, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[20]  N. Weiss,et al.  Case-control studies of the efficacy of screening tests designed to prevent the incidence of cancer. , 1999, American journal of epidemiology.

[21]  Eric J Feuer,et al.  Dynamic Microsimulation Models for Health Outcomes , 2011, Medical decision making : an international journal of the Society for Medical Decision Making.

[22]  Michael J. Tildesley,et al.  A Bayesian Ensemble Approach for Epidemiological Projections , 2015, PLoS Comput. Biol..

[23]  Mark Jit,et al.  Calibration of Complex Models through Bayesian Evidence Synthesis , 2015, Medical decision making : an international journal of the Society for Medical Decision Making.

[24]  J. Wardle,et al.  Gender differences in utilization of colorectal cancer screening , 2005, Journal of medical screening.

[25]  Natasha K. Stout,et al.  Calibration Methods Used in Cancer Simulation Models and Suggested Reporting Guidelines , 2012, PharmacoEconomics.

[26]  Amy B. Knudsen,et al.  Cost-effectiveness of computed tomographic colonography screening for colorectal cancer in the medicare population. , 2010, Journal of the National Cancer Institute.

[27]  J. Kalbfleisch,et al.  The Statistical Analysis of Failure Time Data , 1980 .

[28]  Jonathan Karnon,et al.  Model Performance Evaluation (Validation and Calibration) in Model-based Studies of Therapeutic Interventions for Cardiovascular Diseases , 2013, Applied Health Economics and Health Policy.

[29]  J. Carethers.  One colon lumen but two organs. , 2011, Gastroenterology.

[30]  Adrian E. Raftery,et al.  Bayesian Model Averaging: A Tutorial , 2016 .

[31]  H. Brenner,et al.  Colonoscopy and Polyp Characteristics , 2013, Annals of Internal Medicine.

[32]  C. Williams,et al.  Depth of insertion at flexible sigmoidoscopy: implications for colorectal cancer screening and instrument design. , 1999, Endoscopy.

[33]  Rongwei Fu,et al.  Screening for Colorectal Cancer: A Targeted, Updated Systematic Review for the U.S. Preventive Services Task Force , 2008, Annals of Internal Medicine.

[34]  Hsiu‐Po Wang,et al.  Different Bowel Preparation Schedule Leads to Different Diagnostic Yield of Proximal and Nonpolypoid Colorectal Neoplasm at Screening Colonoscopy in Average-Risk Population , 2011, Diseases of the colon and rectum.

[35]  Cancer,et al.  Once-only flexible sigmoidoscopy screening in prevention of colorectal cancer: a multicentre randomised controlled trial , 2010, The Lancet.

[36]  Carolyn M Rutter,et al.  An Evidence-Based Microsimulation Model for Colorectal Cancer: Validation and Application , 2010, Cancer Epidemiology, Biomarkers & Prevention.

[37]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[38]  Iris Lansdorp-Vogelaar,et al.  A Systematic Comparison of Microsimulation Models of Colorectal Cancer , 2011, Medical decision making : an international journal of the Society for Medical Decision Making.

[39]  Amy B. Knudsen,et al.  Evaluating Test Strategies for Colorectal Cancer Screening: A Decision Analysis for the U.S. Preventive Services Task Force , 2008, Annals of Internal Medicine.

[40]  Graham A. Colditz,et al.  Cost-effectiveness of screening for colorectal cancer in the general population. , 2000, JAMA.

[41]  Eric J Feuer,et al.  Secular trends in colon and rectal cancer relative survival. , 2013, Journal of the National Cancer Institute.

[42]  Onchee Yu,et al.  A hierarchical non‐homogenous Poisson model for meta‐analysis of adenoma counts , 2007, Statistics in medicine.

[43]  J. Potter,et al.  Colorectal Endoscopy, Advanced Adenomas, and Sessile Serrated Polyps: Implications for Proximal Colon Cancer , 2012, The American Journal of Gastroenterology.

[44]  N Segnan,et al.  European guidelines for quality assurance in colorectal cancer screening and diagnosis. First Edition – Colonoscopic surveillance following adenoma removal , 2012, Endoscopy.

[45]  Amy B. Knudsen,et al.  Stool DNA Testing to Screen for Colorectal Cancer in the Medicare Population , 2010, Annals of Internal Medicine.