Reinforced Contact Tracing and Epidemic Intervention

The recent outbreak of COVID-19 poses a serious threat to people’s lives. Epidemic control strategies have also caused damage to the economy by cutting off humans’ daily commute. In this paper, we develop an Individual-based Reinforcement Learning Epidemic Control Agent (IDRLECA) to search for smart epidemic control strategies that can simultaneously minimize infections and the cost of mobility intervention. IDRLECA first hires an infection probability model to calculate the current infection probability of each individual. Then, the infection probabilities together with individuals’ health status and movement information are fed to a novel GNN to estimate the spread of the virus through human contacts. The estimated risks are used to further support an RL agent to select individual-level epidemic-control actions. The training of IDRLECA is guided by a specially designed reward function considering both the cost of mobility intervention and the effectiveness of epidemic control. Moreover, we design a constraint for control-action selection that eases its difficulty and further improve exploring efficiency. Extensive experimental results demonstrate that IDRLECA can suppress infections at a very low level and retain more than 95% of human mobility.

[1]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[2]  Qingchu Wu,et al.  An individual-based modeling framework for infectious disease spreading in clustered complex networks , 2020 .

[3]  Madhav V. Marathe,et al.  EpiFast: a fast algorithm for large scale realistic epidemic simulations on distributed memory systems , 2009, ICS.

[4]  Hao Wu,et al.  Mastering Complex Control in MOBA Games with Deep Reinforcement Learning , 2019, AAAI.

[5]  Jakub W. Pachocki,et al.  Dota 2 with Large Scale Deep Reinforcement Learning , 2019, ArXiv.

[6]  Aravind Srinivasan,et al.  Modelling disease outbreaks in realistic urban social networks , 2004, Nature.

[7]  John Augustine,et al.  Economy Versus Disease Spread: Reopening Mechanisms for COVID 19 , 2020, ArXiv.

[8]  Davide Bacciu,et al.  Contextual Graph Markov Model: A Deep and Generative Approach to Graph Processing , 2018, ICML.

[9]  W. Liang,et al.  Modified SEIR and AI prediction of the epidemics trend of COVID-19 in China under public health interventions , 2020, Journal of thoracic disease.

[10]  S. Barua.  Understanding Coronanomics: The Economic Implications of the Coronavirus (COVID-19) Pandemic , 2020, SSRN Electronic Journal.

[11]  HighWire Press.  Proceedings of the Royal Society of London. Series A, Containing papers of a mathematical and physical character , 1934 .

[12]  J. Blumenstock,et al.  The strength of long-range ties in population-scale social networks , 2018, Science.

[13]  G. Milne,et al.  A Small Community Model for the Transmission of Infectious Diseases: Comparison of School Closure as an Intervention in Individual-Based Models of an Influenza Pandemic , 2008, PloS one.

[14]  Phillip D. Stroud,et al.  EpiSimS simulation of a multi-component strategy for pandemic influenza , 2008, SpringSim '08.

[15]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[16]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[17]  A. L. Schmidt,et al.  Economic and social consequences of human mobility restrictions under COVID-19 , 2020, Proceedings of the National Academy of Sciences.

[18]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[19]  Jieping Ye,et al.  Spatiotemporal Multi-Graph Convolution Network for Ride-Hailing Demand Forecasting , 2019, AAAI.

[20]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[21]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[22]  Naoki Masuda,et al.  Individual-based approach to epidemic processes on arbitrary dynamic contact networks , 2015, Scientific Reports.

[23]  Le Song,et al.  Heterogeneous Graph Neural Networks for Malicious Account Detection , 2018, CIKM.

[24]  W. O. Kermack,et al.  A contribution to the mathematical theory of epidemics , 1927 .

[25]  S. Rizzo.  Balancing Precision and Recall for Cost-effective Epidemic Containment , 2020 .

[26]  Alessandro Vespignani,et al.  Modeling the spatial spread of infectious diseases: The GLobal Epidemic and Mobility computational model , 2010, J. Comput. Sci..

[27]  Xiaoyang Yu,et al.  Mining community and inferring friendship in mobile social networks , 2016, Neurocomputing.