Mechanistic machine learning: how data assimilation leverages physiologic knowledge using Bayesian inference to forecast the future, infer the present, and phenotype

Abstract We introduce data assimilation as a computational method that uses machine learning to combine data with human knowledge in the form of mechanistic models in order to forecast future states, to impute missing data from the past by smoothing, and to infer measurable and unmeasurable quantities that represent clinically and scientifically important phenotypes. We demonstrate the advantages it affords in the context of type 2 diabetes by showing how data assimilation can be used to forecast future glucose values, to impute previously missing glucose values, and to infer type 2 diabetes phenotypes. At the heart of data assimilation is the mechanistic model, here an endocrine model. Such models can vary in complexity, contain testable hypotheses about important mechanics that govern the system (eg, nutrition’s effect on glucose), and, as such, constrain the model space, allowing for accurate estimation using very little data.

[1]  David T. Westwick,et al.  Identification of Nonlinear Physiological Systems: Westwick/Identification of Nonlinear Physiological Systems , 2005 .

[2]  Christopher E. Hann,et al.  A physiological Intensive Control Insulin-Nutrition-Glucose (ICING) model validated in critically ill patients , 2011, Comput. Methods Programs Biomed..

[3]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[4]  James V. Candy,et al.  Bayesian Signal Processing , 2009 .

[5]  Kazuyuki Aihara,et al.  Development of a mathematical model that predicts the outcome of hormone therapy for prostate cancer. , 2010, Journal of theoretical biology.

[6]  Lena Mamykina,et al.  Offline and online data assimilation for real-time blood glucose forecasting in type 2 diabetes , 2017, 1709.00163.

[7]  R S Parker,et al.  The intravenous route to blood glucose control. , 2001, IEEE engineering in medicine and biology magazine : the quarterly magazine of the Engineering in Medicine & Biology Society.

[8]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[9]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[10]  Arthur P. Dempster,et al.  Upper and Lower Probabilities Induced by a Multivalued Mapping , 1967, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[11]  Colin J. Cotter,et al.  Probabilistic Forecasting and Bayesian Data Assimilation , 2015 .

[12]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[13]  P. Glass,et al.  Pharmacokinetic–pharmacodynamic modeling in anesthesia, intensive care and pain medicine , 2009, Current opinion in anaesthesiology.

[14]  Geir Evensen,et al.  The Ensemble Kalman Filter: theoretical formulation and practical implementation , 2003 .

[15]  Lena Mamykina,et al.  Data-driven health management: reasoning about personally generated data in diabetes with information technologies , 2016, J. Am. Medical Informatics Assoc..

[16]  H. Musoff,et al.  Unscented Kalman Filter , 2015 .

[17]  Bruce J. Gluckman,et al.  Data assimilation of glucose dynamics for use in the intensive care unit , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[18]  C. Hann,et al.  Adaptive Bolus-Based Set-Point Regulation of Hyperglycemia in Critical Care , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[19]  Branko Ristic,et al.  Beyond the Kalman Filter: Particle Filters for Tracking Applications , 2004 .

[20]  David J. Christini,et al.  Introduction: Mapping and control of complex cardiac arrhythmias. , 2002, Chaos.

[21]  Elizabeth M. Heitkemper,et al.  Designing for engagement with self-monitoring: A user-centered approach with low-income, Latino adults with Type 2 Diabetes , 2019, Int. J. Medical Informatics.

[22]  Peter L. Bonate,et al.  Recommended reading in population pharmacokinetic pharmacodynamics , 2005, The AAPS Journal.

[23]  C. Cobelli,et al.  In Silico Preclinical Trials: A Proof of Concept in Closed-Loop Control of Type 1 Diabetes , 2009, Journal of diabetes science and technology.

[24]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[25]  G. Hripcsak,et al.  Population Physiology: Leveraging Electronic Health Record Data to Understand Human Endocrine Dynamics , 2011, PloS one.

[26]  DJ Albers,et al.  The Parameter Houlihan: a solution to high-throughput identifiability indeterminacy for brutally ill-posed problems. , 2019, Mathematical biosciences.

[27]  Francis J. Doyle,et al.  Robust H∞ glucose control in diabetes using a physiological model , 2000 .

[28]  Roman Hovorka,et al.  Feasibility of fully automated closed-loop glucose control using continuous subcutaneous glucose measurements in critical illness: a randomized controlled trial , 2013, Critical Care.

[29]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[30]  A. H. Murphy,et al.  What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting , 1993 .

[31]  Nils Lid Hjort,et al.  Model Selection and Model Averaging , 2001 .

[32]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[33]  George Hripcsak,et al.  High-fidelity phenotyping: richness and freedom from bias , 2017, J. Am. Medical Informatics Assoc..

[34]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[35]  Florian Nadel,et al.  Stochastic Processes And Filtering Theory , 2016 .

[36]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[37]  Roman Hovorka,et al.  Home Use of an Artificial Beta Cell in Type 1 Diabetes. , 2015, The New England journal of medicine.

[38]  J. F. Selgrade,et al.  A model for hormonal control of the menstrual cycle: structural consistency but sensitivity with regard to data. , 2009, Journal of Theoretical Biology.

[39]  Ralph C. Smith,et al.  Uncertainty Quantification: Theory, Implementation, and Applications , 2013 .

[40]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[41]  Bruce J. Gluckman,et al.  Reconstructing Mammalian Sleep Dynamics with Data Assimilation , 2012, PLoS Comput. Biol..

[42]  W. Briggs Statistical Methods in the Atmospheric Sciences , 2007 .

[43]  D. Nychka Data Assimilation” , 2006 .

[44]  Nicholas G. Polson,et al.  Tracking Epidemics With Google Flu Trends Data and a State-Space SEIR Model , 2012, Journal of the American Statistical Association.

[45]  S. F. Schmidt APPLICATION OF STATISTICAL FILTER THEORY TO THE OPTIMAL ESTIMATION OF POSITION AND VELOCITY ON BOARD A CIRCUMLUNAR VEHICLE , 2022 .

[46]  Niklas Hartung,et al.  Bayesian Data Assimilation to Support Informed Decision Making in Individualized Chemotherapy , 2019, CPT: pharmacometrics & systems pharmacology.

[47]  Xiaohua Xia,et al.  On Identifiability of Nonlinear ODE Models and Applications in Viral Dynamics , 2011, SIAM Rev..

[48]  G. Roberts,et al.  MCMC Methods for Functions: ModifyingOld Algorithms to Make Them Faster , 2012, 1202.0709.

[49]  Arthur P. Dempster,et al.  A Generalization of Bayesian Inference , 1968, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[50]  E. Mosekilde,et al.  Computer model for mechanisms underlying ultradian oscillations of insulin and glucose. , 1991, The American journal of physiology.

[51]  Kazuyuki Aihara,et al.  Hybrid optimal scheduling for intermittent androgen suppression of prostate cancer. , 2010, Chaos.

[52]  Gilles Clermont,et al.  From Inverse Problems in Mathematical Physiology to Quantitative Differential Diagnoses , 2007, PLoS Comput. Biol..

[53]  Steven M. Markowitz Cardiac Arrhythmias and Device Therapy: Results and Perspectives for the New Century , 2000 .

[54]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[55]  L. Glass,et al.  DYNAMIC CONTROL OF CARDIAC ALTERNANS , 1997 .

[56]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[57]  V L Gott,et al.  Termination of malignant ventricular arrhythmias with an implanted automatic defibrillator in human beings. , 1980, The New England journal of medicine.

[58]  David T. Westwick,et al.  Identification of nonlinear physiological systems , 2003 .

[59]  Lena Mamykina,et al.  Personalized glucose forecasting for type 2 diabetes using data assimilation , 2017, PLoS Comput. Biol..

[60]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[61]  Glenn Shafer,et al.  Belief Functions and Parametric Models , 1982, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[62]  H. Madsen,et al.  Using Stochastic Differential Equations for PK/PD Model Development , 2005, Journal of Pharmacokinetics and Pharmacodynamics.

[63]  James V. Candy,et al.  Bayesian Signal Processing: Classical, Modern and Particle Filtering Methods , 2009 .

[64]  Bruce A. McElhoe,et al.  An Assessment of the Navigation and Course Corrections for a Manned Flyby of Mars or Venus , 1966, IEEE Transactions on Aerospace and Electronic Systems.

[65]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[66]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[67]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[68]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[69]  G. Wahba Smoothing noisy data with spline functions , 1975 .

[70]  R. Reynolds,et al.  The NCEP/NCAR 40-Year Reanalysis Project , 1996, Renewable Energy.

[71]  L. Magni,et al.  Diabetes: Models, Signals, and Control , 2010, IEEE Reviews in Biomedical Engineering.

[72]  Jianying Hu,et al.  Artificial Intelligence in Health in 2018: New Opportunities, Challenges, and Practical Implications , 2019, Yearbook of Medical Informatics.

[73]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[74]  M. Morari,et al.  Closed-Loop Control of Blood Glucose , 2007 .

[75]  Harvey Thomas Banks,et al.  Modeling and Inverse Problems in the Presence of Uncertainty , 2014 .

[76]  E. Tabak,et al.  Dynamical Phenotyping: Using Temporal Analysis of Clinically Collected Physiologic Data to Stratify Populations , 2014, PloS one.

[77]  S. Lauritzen Time Series Analysis in 1880: A Discussion of Contributions Made by T.N. Thiele , 1981 .

[78]  Sophie Donnet,et al.  A review on estimation of stochastic differential equations for pharmacokinetic/pharmacodynamic models. , 2013, Advanced drug delivery reviews.

[79]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .