Learning Stroke Treatment Progression Models for an MDP Clinical Decision Support System

This paper describes a clinical decision support framework in multi-step health care domains that can dynamically recommend optimal treatment plans with respect to both patient outcomes and expected treatment cost. Our system uses a modified POMDP framework in which hidden states are not explicitly modeled, but rather, probabilistic models for predicting future observables given observation and action histories are learned directly from electronic health record (EHR) data. High quality treatment recommendations are found using a sampling-based tree growing approach which produces good results despite only exploring a fraction of the observation and action spaces. We describe the application of the approach to an ischemic stroke domain with clinical trial data (International Stroke Trial Dataset, 1993-1996). The dataset is of moderate size (N=19,435) and exhibits many characteristics of real EHR data, including noise, missing values, and idiosyncratic coding. The system’s predictive model was chosen using cross-validated model selection from a set of several candidate learning methods, including logistic regression, Naive Bayes, Bayes nets, and random forests. Simulations suggest that the optimized decisions improve patient outcomes, such as 6-month survival rate, compared to the decisions of human doctors during the study.

[1]  Evert de Jonge,et al.  Prognostic Bayesian networks: II: An application in the domain of cardiac surgery , 2007, J. Biomed. Informatics.

[2]  Jeffrey L Saver,et al.  Stroke Declines From Third to Fourth Leading Cause of Death in the United States: Historical Perspective and Challenges Ahead , 2011, Stroke.

[3]  D. Bates,et al.  Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. , 2003, Archives of internal medicine.

[4]  Peter Sandercock,et al.  The International Stroke Trial (IST): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19 435 patients with acute ischaemic stroke , 1997, The Lancet.

[5]  E. Berge,et al.  Low molecular-weight heparin versus aspirin in patients with acute ischaemic stroke and atrial fibrillation: a double-blind randomised study , 2000, The Lancet.

[6]  Peter Sandercock,et al.  Previous Use of Aspirin and Baseline Stroke Severity: An Analysis of 17 850 Patients in the International Stroke Trial , 2006, Stroke.

[7]  E. Balas,et al.  Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success , 2005, BMJ : British Medical Journal.

[8]  P A Wolf,et al.  Survival and recurrence following stroke. The Framingham study. , 1982, Stroke.

[9]  Byoung-Tak Zhang,et al.  AptaCDSS-E: A classifier ensemble-based clinical decision support system for cardiovascular disease level prediction , 2008, Expert Syst. Appl..

[10]  Evert de Jonge,et al.  Prognostic Bayesian networks: I: Rationale, learning procedure, and clinical use , 2007, J. Biomed. Informatics.

[11]  Kris K. Hauser,et al.  Artificial intelligence framework for simulating clinical decision-making: A Markov decision process approach , 2013, Artif. Intell. Medicine.

[12]  K Asplund,et al.  Haemodilution for acute ischaemic stroke. , 2000, The Cochrane database of systematic reviews.

[13]  David W. Bates,et al.  Synthesis of Research Paper: Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a Reality , 2003, J. Am. Medical Informatics Assoc..

[14]  K. Davis,et al.  Mirror, Mirror on the Wall Mirror, Mirror on the Wall How the Performance of the U.S. Health Care System Compares Internationally , 2010 .

[15]  Caroline Leigh Watkins.  The International Stroke Trial (IST): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19 435 patients with acute ischaemic stroke , 1997 .

[16]  P. Sandercock,et al.  The International Stroke Trial database , 2011, Trials.

[17]  C Warlow,et al.  Indications for early aspirin use in acute ischemic stroke : A combined analysis of 40 000 randomized patients from the chinese acute stroke trial and the international stroke trial. On behalf of the CAST and IST collaborative groups. , 2000, Stroke.

[18]  L. Hayden,et al.  Ten Commandments for Effective Clinical Decision Support: Making the Practice of Evidence-based Medicine a Reality , 2011 .

[19]  H. R. Taylor,et al.  The International Stroke Trial (IST): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19 435 patients with acute ischaemic stroke , 1997, The Lancet.

[20]  Yishay Mansour,et al.  A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes , 1999, Machine Learning.

[21]  K. Davis,et al.  Mirror, Mirror on the Wall: How the Performance of the U.S. Health Care System Compares Internationally, 2010 Update , 2010 .

[22]  Richard S. Sutton,et al.  Predictive Representations of State , 2001, NIPS.

[23]  ZhengMingChen,et al.  Indications for Early Aspirin Use in Acute Ischemic Stroke , 2000 .

[24]  Athanassios,et al.  Australia and New Zealand Health Policy Open Access Medical Decision Making for Patients with Parkinson Disease under Average Cost Criterion , 2022 .

[25]  Oguzhan Alagoz,et al.  Markov Decision Processes: A Tool for Sequential Decision Making under Uncertainty , 2010, Medical decision making : an international journal of the Society for Medical Decision Making.