Planning as Inference in Epidemiological Dynamics Models

In this work we demonstrate how to automate parts of the infectious disease-control policy-making process via performing inference in existing epidemiological models. The kind of inference tasks undertaken include computing the posterior distribution over controllable, via direct policy-making choices, simulation model parameters that give rise to acceptable disease progression outcomes. Among other things, we illustrate the use of a probabilistic programming language that automates inference in existing simulators. Neither the full capabilities of this tool for automating inference nor its utility for planning is widely disseminated at the current time. Timely gains in understanding about how such simulation-based models and inference automation tools applied in support of policy-making could lead to less economically damaging policy prescriptions, particularly during the current COVID-19 pandemic.

[1]  Mark W. Schmidt,et al.  Robust Asymmetric Learning in POMDPs , 2020, ICML.

[2]  G. Toraldo,et al.  Tracing day-zero and forecasting the COVID-19 outbreak in Lombardy, Italy: A compartmental modelling and numerical optimization approach , 2020, PloS one.

[3]  S. Eubank,et al.  Commentary on Ferguson, et al., “Impact of Non-pharmaceutical Interventions (NPIs) to Reduce COVID-19 Mortality and Healthcare Demand” , 2020, Bulletin of Mathematical Biology.

[4]  Frank D. Wood,et al.  Coping With Simulators That Don't Always Return , 2020, AISTATS.

[5]  Akshaya Srikanth Bhagavathula,et al.  Modeling the Epidemiological Trend and Behavior of COVID-19 in Italy , 2020, medRxiv.

[6]  J. Gómez-Gardeñes,et al.  A mathematical model for the spatiotemporal epidemic spreading of COVID19 , 2020, medRxiv.

[7]  Diego Caccavo,et al.  Chinese and Italian COVID-19 outbreaks can be correctly described by a modified SIRD model , 2020, medRxiv.

[8]  S. Mei,et al.  Epidemiological and clinical characteristics of COVID-19 in Shenzhen, the largest migrant city of China , 2020, medRxiv.

[9]  P. Teles,et al.  PREDICTING THE EVOLUTION OF COVID-19 IN PORTUGAL USING AN ADAPTED SIR MODEL PREVIOUSLY USED IN SOUTH KOREA FOR THE MERS OUTBREAK , 2020, medRxiv.

[10]  Qiao Wang,et al.  Assessing the Global Tendency of COVID-19 Outbreak , 2020, medRxiv.

[11]  Gerardo Toraldo,et al.  Tracing DAY-ZERO and Forecasting the Fade out of the COVID-19 Outbreak in Lombardy, Italy: A Compartmental Modelling and Numerical Optimization Approach. , 2020, medRxiv.

[12]  Flavio Iannelli,et al.  Trend analysis of the COVID-19 pandemic in China and the rest of the world , 2020, medRxiv.

[13]  Snehal M. Shekatkar,et al.  Multi-city modeling of epidemics using spatial networks: Application to 2019-nCov (COVID-19) coronavirus in India , 2020, medRxiv.

[14]  M. Traini,et al.  Modelling the epidemic 2019-nCoV event in Italy: a preliminary note , 2020, medRxiv.

[15]  M. Magdon-Ismail Machine Learning the Phenomenology of COVID-19 From Early Infection Dynamics , 2020, medRxiv.

[16]  Jonathan Le Roux,et al.  COVID-19: Forecasting short term hospital needs in France , 2020, medRxiv.

[17]  Jia Wangping,et al.  Extended SIR Prediction of the Epidemics Trend of COVID-19 in Italy and Compared With Hunan, China , 2020, Frontiers in Medicine.

[18]  Hannah R. Meredith,et al.  The Incubation Period of Coronavirus Disease 2019 (COVID-19) From Publicly Reported Confirmed Cases: Estimation and Application , 2020, Annals of Internal Medicine.

[19]  Xuefeng Li,et al.  Modeling the situation of COVID-19 and effects of different containment strategies in China with dynamic differential equations and parameters estimation , 2020, medRxiv.

[20]  Tobias Bleicker,et al.  Clinical presentation and virological assessment of hospitalized cases of coronavirus disease 2019 in a travel-associated transmission cluster , 2020, medRxiv.

[21]  Pierre Magal,et al.  Predicting the cumulative number of cases for the COVID-19 epidemic in China from early data , 2020, medRxiv.

[22]  Zunyou Wu,et al.  Characteristics of and Important Lessons From the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72 314 Cases From the Chinese Center for Disease Control and Prevention. , 2020, JAMA.

[23]  Liangrong Peng,et al.  Epidemic analysis of COVID-19 in China by dynamical modeling , 2020, medRxiv.

[24]  N. Hengartner,et al.  The Novel Coronavirus, 2019-nCoV, is Highly Contagious and More Infectious Than Initially Estimated , 2020, medRxiv.

[25]  G. Röst,et al.  Risk Assessment of Novel Coronavirus COVID-19 Outbreaks Outside China , 2020, medRxiv.

[26]  C. Althaus,et al.  Pattern of early human-to-human transmission of Wuhan 2019 novel coronavirus (2019-nCoV), December 2019 to January 2020 , 2020, Euro surveillance : bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin.

[27]  Sebastian Funk,et al.  Choices and trade-offs in inference with infectious disease models. , 2019, Epidemics.

[28]  Amanda Minter,et al.  Approximate Bayesian Computation for infectious disease modelling. , 2019, Epidemics.

[29]  Frank D. Wood,et al.  Deep Probabilistic Surrogate Networks for Universal Simulator Approximation , 2019, ArXiv.

[30]  Frank D. Wood,et al.  Amortized Rejection Sampling in Universal Probabilistic Programming , 2019, AISTATS.

[31]  Prabhat,et al.  Etalumis: bringing probabilistic programming to scientific simulators at scale , 2019, SC.

[32]  Andrew Plummer,et al.  School dismissal as a pandemic influenza response: When, where and for how long? , 2019, Epidemics.

[33]  Yee Whye Teh,et al.  Hijacking Malaria Simulators with Probabilistic Programming , 2019, ArXiv.

[34]  Frank D. Wood,et al.  LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models , 2019, AISTATS.

[35]  Marc Baguelin,et al.  Contemporary statistical inference for infectious disease models using Stan. , 2019, Epidemics.

[36]  John D. Kelleher,et al.  An open-data-driven agent-based model to simulate infectious disease outbreaks , 2018, PloS one.

[37]  Jennifer Badham,et al.  Developing agent-based models of complex health behaviour , 2018, Health & place.

[38]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.

[39]  Hongseok Yang,et al.  An Introduction to Probabilistic Programming , 2018, ArXiv.

[40]  L. Childs,et al.  An introduction to compartmental modeling for the budding infectious disease modeler , 2018 .

[41]  Prabhat,et al.  Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model , 2018, NeurIPS.

[42]  Ye Chen,et al.  Implementation and applications of EMOD, an individual-based multi-disease modeling platform , 2018, Pathogens and disease.

[43]  Yee Whye Teh,et al.  Revisiting Reweighted Wake-Sleep for Models with Stochastic Control Flow , 2018, UAI.

[44]  Sergey Levine,et al.  Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review , 2018, ArXiv.

[45]  Jeremy E. Oakley,et al.  Approximate Bayesian Computation and simulation based inference for complex stochastic epidemic models , 2018 .

[46]  Magdalena Cerdá,et al.  Agent-Based Modeling in Public Health: Current Applications and Future Directions , 2018, Annual review of public health.

[47]  Hongseok Yang,et al.  On Nesting Monte Carlo Estimators , 2017, ICML.

[48]  D. Krewski,et al.  Effectiveness of personal protective measures in reducing pandemic influenza transmission: A systematic review and meta-analysis. , 2017, Epidemics.

[49]  John D. Kelleher,et al.  A Taxonomy for Agent-Based Models in Human Infectious Disease Epidemiology , 2017, J. Artif. Soc. Soc. Simul..

[50]  Vanessa Sochat,et al.  Singularity: Scientific containers for mobility of compute , 2017, PloS one.

[51]  Theodore Kypraios,et al.  A tutorial introduction to Bayesian inference for stochastic epidemic models using Approximate Bayesian Computation. , 2017, Mathematical biosciences.

[52]  O. Sharomi,et al.  Optimal control in epidemiology , 2017, Ann. Oper. Res..

[53]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[54]  Frank D. Wood,et al.  Inference Compilation and Universal Probabilistic Programming , 2016, AISTATS.

[55]  David Tolpin,et al.  Design and Implementation of Probabilistic Programming Language Anglican , 2016, IFL 2016.

[56]  J. Lessler,et al.  Trends in the Mechanistic and Dynamic Modeling of Infectious Diseases , 2016, Current Epidemiology Reports.

[57]  Frank D. Wood,et al.  Inference Networks for Sequential Monte Carlo in Graphical Models , 2016, ICML.

[58]  Fredrik Lindsten,et al.  Interacting Particle Markov Chain Monte Carlo , 2016, ICML.

[59]  Ohad Kammar,et al.  Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints , 2016, 2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS).

[60]  J. Grefenstette,et al.  Is population structure sufficient to generate area-level inequalities in influenza rates? An examination using agent-based models , 2015, BMC Public Health.

[61]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[62]  Wayne T. A. Enanoria,et al.  The role of vaccination coverage, individual behaviors, and the public health response in the control of measles epidemics: an agent-based simulation for California , 2015, BMC Public Health.

[63]  Frank D. Wood,et al.  Particle Gibbs with Ancestor Sampling for Probabilistic Programs , 2015, AISTATS.

[64]  David Tolpin,et al.  Output-Sensitive Adaptive Metropolis-Hastings for Probabilistic Programs , 2015, ECML/PKDD.

[65]  A. Lloyd,et al.  Nine challenges for deterministic epidemic models , 2014, Epidemics.

[66]  Yee Whye Teh,et al.  Asynchronous Anytime Sequential Monte Carlo , 2014, NIPS.

[67]  Frank D. Wood,et al.  A New Approach to Probabilistic Programming Inference , 2014, AISTATS.

[68]  Shawn T. Brown,et al.  FRED (A Framework for Reconstructing Epidemic Dynamics): an open-source software system for modeling infectious diseases and control strategies using census-based populations , 2013, BMC Public Health.

[69]  Donald S Burke,et al.  Policies to reduce influenza in the workplace: impact assessments using an agent-based model. , 2013, American journal of public health.

[70]  Lawrence M. Murray Bayesian State-Space Modelling on High-Performance Hardware Using LibBi , 2013, 1306.3277.

[71]  Shawn T. Brown,et al.  School closure as an influenza mitigation strategy: how variations in legal authority and plan criteria can alter the impact , 2012, BMC Public Health.

[72]  Frank L. Lewis,et al.  Optimal Control: Lewis/Optimal Control 3e , 2012 .

[73]  Jon Parker,et al.  A Distributed Platform for Global-Scale Agent-Based Models of Disease Transmission , 2011, TOMC.

[74]  Noah D. Goodman,et al.  Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation , 2011, AISTATS.

[75]  Shanta M Zimmer,et al.  The benefits to all of ensuring equal and timely access to influenza vaccines in poor communities. , 2011, Health affairs.

[76]  Stephen P. Boyd,et al.  Receding Horizon Control , 2011, IEEE Control Systems.

[77]  Yang Wang,et al.  Code generation for receding horizon control , 2010, 2010 IEEE International Symposium on Computer-Aided Control System Design.

[78]  Shawn T. Brown,et al.  Simulating school closure strategies to mitigate an influenza epidemic. , 2010, Journal of public health management and practice : JPHMP.

[79]  Marc Toussaint,et al.  Robot trajectory optimization using approximate inference , 2009, ICML '09.

[80]  Laxminarayana Ganapathi,et al.  Synthesized Population Databases: A US Geospatial Database for Agent-Based Models. , 2009, Methods report.

[81]  David Welch,et al.  Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems , 2009, Journal of The Royal Society Interface.

[82]  Vicenç Gómez,et al.  Optimal control as a graphical model inference problem , 2009, Machine Learning.

[83]  Emanuel Todorov,et al.  General duality between optimal control and estimation , 2008, 2008 47th IEEE Conference on Decision and Control.

[84]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[85]  B. Finkenstädt,et al.  Statistical Inference in a Stochastic Epidemic SEIR Model with Control Intervention: Ebola as a Case Study , 2006, Biometrics.

[86]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[87]  Pejman Rohani,et al.  Appropriate Models for the Management of Infectious Diseases , 2005, PLoS medicine.

[88]  Gerald DeJong,et al.  The Influence of Reward on the Speed of Reinforcement Learning: An Analysis of Shaping , 2003, ICML.

[89]  T. Sargent,et al.  Robust Control and Model Uncertainty , 2001 .

[90]  James B. Rawlings,et al.  Tutorial overview of model predictive control , 2000 .

[91]  Jay H. Lee,et al.  Model predictive control: past, present and future , 1999 .

[92]  J. Doyle,et al.  Essentials of Robust Control , 1997 .

[93]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[94]  David J. N. Limebeer,et al.  Linear Robust Control , 1994 .

[95]  Manfred Morari,et al.  Model predictive control: Theory and practice - A survey , 1989, Autom..

[96]  W. O. Kermack,et al.  A contribution to the mathematical theory of epidemics , 1927 .

[97]  J. Sureshkumar,et al.  Novel Coronavirus , 2020, Research Journal of Pharmacology and Pharmacodynamics.

[98]  Frank Wood,et al.  Planning as Inference in Epidemiological Models , 2020, ArXiv.

[99]  D. Bertsekas Reinforcement Learning and Optimal ControlA Selective Overview , 2018 .

[100]  Wenlong Fu,et al.  Model-based reinforcement learning: A survey , 2018 .

[101]  P. Koya,et al.  Mathematical Modeling and Simulation Study of SEIR disease and Data Fitting of Ebola Epidemic spreading in West Africa , 2015 .

[102]  David B. Dunson,et al.  Bayesian data analysis, third edition , 2013 .

[103]  Peter Stone,et al.  Reinforcement learning , 2019, Scholarpedia.

[104]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[105]  Marko Bacic,et al.  Model predictive control , 2003 .

[106]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[107]  M. Köppen,et al.  The Curse of Dimensionality , 2010 .

[108]  HighWire Press Proceedings of the Royal Society of London. Series A, Containing papers of a mathematical and physical character , 1934 .