Desiderata for sharable computable biomedical knowledge for learning health systems

In this commentary, we work out the specific desired functions required for sharing knowledge objects (based on statistical models) presumably to be used for clinical decision support derived from a learning health system, and, in so doing, discuss the implications for novel knowledge architectures. We will demonstrate how decision models, implemented as influence diagrams, satisfy the desiderata. The desiderata include locally validate discrimination, locally validate calibration, locally recalculate thresholds by incorporating local preferences, provide explanation, enable monitoring, enable debiasing, account for generalizability, account for semantic uncertainty, shall be findable, and others as necessary and proper. We demonstrate how formal decision models, especially when implemented as influence diagrams based on Bayesian networks, support both the knowledge artifact itself (the “primary decision”) and the “meta‐decision” of whether to deploy the knowledge artifact. We close with a research and development agenda to put this framework into place.

[1]  William Marsh,et al.  Value of information analysis for interventional and counterfactual Bayesian networks in forensic medical sciences , 2016, Artif. Intell. Medicine.

[2]  R. Saunders,et al.  Best Care at Lower Cost: The Path to Continuously Learning Health Care in America , 2013 .

[3]  J. Kassirer,et al.  The threshold approach to clinical decision making. , 1980, The New England journal of medicine.

[4]  Steven G. Johnson,et al.  A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data , 2016, EGEMS.

[5]  P. Plsek,et al.  The challenge of complexity in health care , 2001, BMJ : British Medical Journal.

[6]  Martin Eccles,et al.  Implementing clinical guidelines: current evidence and future implications. , 2004, The Journal of continuing education in the health professions.

[7]  H P Lehmann,et al.  Developmental dysplasia of the hip practice guideline: technical report. Committee on Quality Improvement, and Subcommittee on Developmental Dysplasia of the Hip. , 2000, Pediatrics.

[8]  H C Sox,et al.  Setting the optimal erythrocyte protoporphyrin screening decision threshold for lead poisoning: a decision analytic approach. , 1991, Pediatrics.

[9]  S D Walter,et al.  The partial area under the summary ROC curve , 2005, Statistics in medicine.

[10]  David Gur,et al.  On use of partial area under the ROC curve for evaluation of diagnostic performance , 2013, Statistics in medicine.

[11]  P. A. Mileman Decision making in health care. Theory, psychology, and applications , 2001 .

[12]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[13]  Carl Lagoze,et al.  The Knowledge Object Reference Ontology (KORO): A formalism to support management and sharing of computable biomedical knowledge for learning health systems , 2018, Learning health systems.

[14]  J Charlet,et al.  Efficient Results in Semantic Interoperability for Health Care. Findings from the Section on Knowledge Representation and Management. , 2016, Yearbook of medical informatics.

[15]  D. Feeny,et al.  Utilities and Quality-Adjusted Life Years , 1989, International Journal of Technology Assessment in Health Care.

[16]  Harold P. Lehmann,et al.  Developmental Dysplasia of the Hip Practice Guideline: Technical Report , 2000, Pediatrics.

[17]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[18]  Yolanda Gil,et al.  PROV-DM: The PROV Data Model , 2013 .

[19]  Ross D. Shachter,et al.  Representation and Analysis of Medical Decision Problems with Influence Diagrams , 1997, Medical decision making : an international journal of the Society for Medical Decision Making.

[20]  Philip R. O. Payne,et al.  Evidence generating medicine: redefining the research-practice relationship to complete the evidence cycle. , 2013, Medical care.

[21]  A. Gelman,et al.  Stan , 2015 .

[22]  J. Q. Smith Influence Diagrams for Statistical Modelling , 1989 .

[23]  Marc Berg,et al.  Overriding of drug safety alerts in computerized physician order entry. , 2006, Journal of the American Medical Informatics Association : JAMIA.

[24]  M. Szklo,et al.  Epidemiology: Beyond the Basics , 1999 .

[25]  David J. Lunn,et al.  The BUGS Book: A Practical Introduction to Bayesian Analysis , 2013 .

[26]  Lawrence M. Fagan,et al.  A Methodology for Generating Computer-based Explanations of Decision-theoretic Advice , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[27]  Ferdinand T. Velasco,et al.  Improving Outcomes with Clinical Decision Support: An Implementer's Guide , 2012 .

[28]  Vibha Anand,et al.  An Empirical Validation of Recursive Noisy OR (RNOR) Rule for Asthma Prediction. , 2010, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[29]  M. Graffar [Modern epidemiology]. , 1971, Bruxelles medical.

[30]  M. Gold Cost-effectiveness in health and medicine , 2016 .

[31]  J. Schreiber Foundations Of Statistics , 2016 .

[32]  Erik Schultes,et al.  The FAIR Guiding Principles for scientific data management and stewardship , 2016, Scientific Data.

[33]  M. Boyle,et al.  Multiattribute and Single‐Attribute Utility Functions for the Health Utilities Index Mark 3 System , 2002, Medical care.

[34]  Mor Peleg,et al.  The Ontology of Clinical Research (OCRe): An informatics foundation for the science of clinical research , 2014, J. Biomed. Informatics.

[35]  Guilherme Del Fiol,et al.  Context-sensitive decision support (infobuttons) in electronic health records: a systematic review , 2017, J. Am. Medical Informatics Assoc..

[36]  Trends in blood lead levels among children--Boston, Massachusetts, 1994-1999. , 2001, MMWR. Morbidity and mortality weekly report.

[37]  Jonathan S. Austrian,et al.  Impact of an emergency department electronic sepsis surveillance system on patient mortality and length of stay , 2018, J. Am. Medical Informatics Assoc..

[38]  Stephanie Taylor,et al.  How do physicians decide to treat: an empirical evaluation of the threshold model , 2014, BMC Medical Informatics and Decision Making.

[39]  M. Cabana,et al.  Why don't physicians follow clinical practice guidelines? A framework for improvement. , 1999, JAMA.

[40]  Maarten Joost IJzerman,et al.  A Comparison of Analytic Hierarchy Process and Conjoint Analysis Methods in Assessing Treatment Alternatives for Stroke Rehabilitation , 2012, The Patient - Patient-Centered Outcomes Research.

[41]  Lucila Ohno-Machado,et al.  Discrimination and calibration of mortality risk prediction models in interventional cardiology , 2005, J. Biomed. Informatics.

[42]  C. Mc Donald,et al.  Use of a computer to detect and respond to clinical events: its effect on clinician behavior. , 1976, Annals of internal medicine.

[43]  Ewout W. Steyerberg,et al.  Big Data and Predictive Analytics: Recalibrating Expectations , 2018, JAMA.

[44]  Moshe Leshno,et al.  The value of information for decision-making in the healthcare environment. , 2007, Studies in health technology and informatics.

[45]  George Hripcsak,et al.  Beyond discrimination: A comparison of calibration methods and clinical usefulness of predictive models of readmission risk , 2017, J. Biomed. Informatics.

[46]  R. Brooks,et al.  The EuroQol Group after 25 years , 2012, Springer Netherlands.

[47]  E. Shortliffe,et al.  An analysis of physician attitudes regarding computer-based clinical consultation systems. , 1981, Computers and biomedical research, an international journal.

[48]  T A Pryor,et al.  Rationale for the Arden Syntax. , 1994, Computers and biomedical research, an international journal.

[49]  John Sarisky,et al.  Blood Lead Levels Among Children Aged <6 Years - Flint, Michigan, 2013-2016. , 2016, MMWR. Morbidity and mortality weekly report.

[50]  Ross D. Shachter,et al.  End-User Construction of Influence Diagrams for Bayesian Statistics , 1993, UAI.

[51]  Shuang Wang,et al.  GIST 2.0: A scalable multi-trait metric for quantifying population representativeness of individual clinical studies , 2016, J. Biomed. Informatics.

[52]  David Feeny,et al.  A Checklist for Reporting Valuation Studies of Multi-Attribute Utility-Based Instruments (CREATE) , 2015, PharmacoEconomics.

[53]  Andrew Thomas,et al.  The BUGS project: Evolution, critique and future directions , 2009, Statistics in medicine.

[54]  Tommi Tervonen,et al.  Clinical trials information in drug development and regulation , 2012 .

[55]  David O Meltzer,et al.  Minimal Modeling Approaches to Value of Information Analysis for Health Research , 2011, Medical decision making : an international journal of the Society for Medical Decision Making.

[56]  James G Dolan,et al.  Patients’ Preferences and Priorities Regarding Colorectal Cancer Screening , 2013, Medical decision making : an international journal of the Society for Medical Decision Making.

[57]  Preventing lead poisoning in young children--United States. , 1985, MMWR. Morbidity and mortality weekly report.

[58]  王德伦 英语-翻译-Internet , 2000 .

[59]  Jonathan M. Teich,et al.  Improving Outcomes with Clinical Decision Support , 2012 .

[60]  et al.,et al.  Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs) , 2011, BMC Medical Informatics Decis. Mak..