Efficient Evaluation of Influenza Mitigation Strategies Using Preventive Bandits

Pandemic influenza has the epidemic potential to kill millions of people. While different preventive measures exist, it remains challenging to implement them in an effective and efficient way. To improve preventive strategies, it is necessary to thoroughly understand their impact on the complex dynamics of influenza epidemics. To this end, epidemiological models provide an essential tool to evaluate such strategies in silico. Epidemiological models are frequently used to assist the decision making concerning the mitigation of ongoing epidemics. Therefore, rapidly identifying the most promising preventive strategies is crucial to adequately inform public health officials. To this end, we formulate the evaluation of prevention strategies as a multi-armed bandit problem. Through experiments, we demonstrate that it is possible to identify the optimal strategy using only a limited number of model evaluations, even if there is a large number of preventive strategies to consider.

[1]  Flute , 1941 .

[2]  J. Bather,et al.  Multi‐Armed Bandit Allocation Indices , 1990 .

[3]  K. D. Patterson,et al.  The geography and mortality of the 1918 influenza pandemic. , 1991, Bulletin of the history of medicine.

[4]  A. Abbas,et al.  Cellular and Molecular Immunology , 1991 .

[5]  K. Stöhr,et al.  Influenza--WHO cares. , 2002, The Lancet. Infectious diseases.

[6]  A. Nizam,et al.  Containing Bioterrorist Smallpox , 2002, Science.

[7]  Vijay Kumar,et al.  Online learning in online auctions , 2003, SODA '03.

[8]  M. Newman,et al.  Applying Network Theory to Epidemics: Control Measures for Mycoplasma pneumoniae Outbreaks , 2003, Emerging infectious diseases.

[9]  Aravind Srinivasan,et al.  Structure of Social Contact Networks and Their Impact on Epidemics , 2004, Discrete Methods in Epidemiology.

[10]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[11]  Martin Enserink,et al.  Crisis Underscores Fragility of Vaccine Production System , 2004, Science.

[12]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[13]  Henry Nicholls,et al.  Pandemic Influenza: The Inside Story , 2006, PLoS biology.

[14]  C. Macken,et al.  Mitigation strategies for pandemic influenza in the United States. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[15]  C. Bridges,et al.  The annual impact of seasonal influenza in the US: measuring disease burden and costs. , 2007, Vaccine.

[16]  H. Robbins Some aspects of the sequential design of experiments , 1952 .

[17]  Hazhir Rahmandad,et al.  Heterogeneity and Network Structure in the Dynamics of Diffusion: Comparing Agent-Based and Differential Equation Models , 2004, Manag. Sci..

[18]  William H Press,et al.  Bandit solutions provide unified ethical models for randomized clinical trials and comparative effectiveness research , 2009, Proceedings of the National Academy of Sciences.

[19]  M. Halloran,et al.  Strategies for Pandemic and Seasonal Influenza Vaccination of Schoolchildren in the United States , 2009, American journal of epidemiology.

[20]  Gail E. Potter,et al.  The Transmissibility and Control of Pandemic Influenza A (H1N1) Virus , 2009, Science.

[21]  Cecile Viboud,et al.  The signature features of influenza pandemics--implications for policy. , 2009, The New England journal of medicine.

[22]  J. Medlock,et al.  Optimizing Influenza Vaccine Distribution , 2009, Science.

[23]  Dennis L. Chao,et al.  FluTE, a Publicly Available Stochastic Influenza Epidemic Simulation Model , 2010, PLoS Comput. Biol..

[24]  Evan Dekker,et al.  Empirical evaluation methods for multiobjective reinforcement learning algorithms , 2011, Machine Learning.

[25]  Kevin D. Glazebrook,et al.  Multi-Armed Bandit Allocation Indices: Gittins/Multi-Armed Bandit Allocation Indices , 2011 .

[26]  Neff Walker,et al.  Mathematical models in the evaluation of health programmes , 2011, The Lancet.

[27]  S. Halstead,et al.  Controlling Dengue with Vaccines in Thailand , 2012, PLoS neglected tropical diseases.

[28]  Ann Nowé,et al.  Designing multi-objective multi-armed bandits algorithms: A study , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[29]  Shimon Whiteson,et al.  A Survey of Multi-Objective Sequential Decision-Making , 2013, J. Artif. Intell. Res..

[30]  .. Lesslera,et al.  Seven challenges for model-driven data collection in experimental and observational studies , 2014 .

[31]  Aurélien Garivier,et al.  On the Complexity of A/B Testing , 2014, COLT.

[32]  M. Biggerstaff,et al.  Estimating the Potential Effects of a Vaccine Program Against an Emerging Influenza Pandemic—United States , 2015, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[33]  Nicholas Jackson,et al.  Efficacy and Long-Term Safety of a Dengue Vaccine in Regions of Endemic Disease. , 2015, The New England journal of medicine.

[34]  Frans A. Oliehoek,et al.  Quality Assessment of MORL Algorithms: A Utility-Based Approach , 2015 .

[35]  Lander Willem Agent-based models for Infectious Disease Transmission: Exploration, Estimation & Computational Efficiency , 2015 .

[36]  Richard K. Zimmerman,et al.  Influenza Vaccine Effectiveness in the United States During 2012–2013: Variable Protection by Age and Virus Type , 2014, The Journal of infectious diseases.

[37]  Diederik M. Roijers Multi-objective decision-theoretic planning , 2016 .

[38]  Alessandro Vespignani,et al.  Spatiotemporal dynamics of the Ebola epidemic in Guinea and implications for vaccination and disease elimination: a computational modeling analysis , 2016, BMC Medicine.

[39]  Diederik M. Roijers Multi-objective decision-theoretic planning: abstract , 2016, SIGAI.

[40]  Daniel J. Laydon,et al.  Benefits and risks of the Sanofi-Pasteur dengue vaccine: Modeling optimal deployment , 2016, Science.

[41]  Alberto Revelli,et al.  Cancer and fertility preservation: international recommendations from an expert meeting , 2016, BMC Medicine.

[42]  Simon Cauchemez,et al.  Model-Based Comprehensive Analysis of School Closure Policies for Mitigating Influenza Epidemics and Pandemics , 2016, PLoS Comput. Biol..

[43]  Jon Cohen INFECTIOUS DISEASE. The race for a Zika vaccine is on. , 2016, Science.

[44]  Simon Cauchemez,et al.  Spread of yellow fever virus outbreak in Angola and the Democratic Republic of the Congo 2015–16: a modelling study , 2017, The Lancet. Infectious diseases.