Reinforcement learning approach for adapting complex agent-based model of evacuation to fast linear model

Nowadays, most coastal regions face a potential risk of tsunami. The evacuation is one of the most effective mitigation procedures. However, there are always the part of evacuees (e.g. the tourist) who lack information of the evacuation map, which motivates us to focus on the problem of optimizing of guidance sign placement for tsunami evacuation. Concretely, we must find out an optimal sign placement in order to have as many evacuees as possible to reach shelters before tsunami arrival. In fact, most studies focus on two approaches: Agent-Based modeling and Equation-Based modeling. Each approach has its own pros and cons. While the Agent-Based modeling introduces an accurate but very slow model, the Equation-Based one provides a very fast but inaccurate model. The idea of this study is that we learn the accurate Agent-Based model and adapt it into very fast Equation-Based model in order to solve the optimizing problem. In this paper, we present clearly two models representing the two approaches and pros and cons of each model. We then propose a reinforcement learning approach for adapting complex Agent-Based model into a very fast Linear Model (representing Equation-Based modeling approach). By experimentation, our proposed approach shows that we can replace a slow complex model by a very fast model with an acceptable level of accuracy in order to solve optimizing problem.

[1]  JI Qingge,et al.  Simulating Crowd Evacuation with a Leader-Follower Model , 2006 .

[2]  Keith M. Christensen,et al.  Agent-Based Emergency Evacuation Simulation with Individuals with Disabilities in the Population , 2008, J. Artif. Soc. Soc. Simul..

[3]  Benoit Gaudou,et al.  GAMA: multi-level and complex environment for agent-based models and simulations , 2013, AAMAS.

[4]  Andrew G. Barto,et al.  Improving Elevator Performance Using Reinforcement Learning , 1995, NIPS.

[5]  Yann Chevaleyre,et al.  Speeding up the evaluation of casualties in multi-agent simulations with Linear Programming application to optimization of sign placement for tsunami evacuation , 2013, The 2013 RIVF International Conference on Computing & Communication Technologies - Research, Innovation, and Vision for Future (RIVF).

[6]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7]  V. Ca,et al.  Tsunami risk along Vietnamese coast , 2008 .

[8]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[9]  Léon Bottou,et al.  The Tradeoffs of Large Scale Learning , 2007, NIPS.

[10]  Norman I. Badler,et al.  Modeling Crowd and Trained Leader Behavior during Building Evacuation , 2006, IEEE Computer Graphics and Applications.

[11]  Shunichi Koshimura,et al.  Tsunami Hazard Mitigation at Palabuhanratu, Indonesia , 2012 .

[12]  Norio Okada,et al.  Dynamic Route Decision Model-based Multi-agent Evacuation Simulation - Case Study of Nagata Ward, Kobe , 2008 .

[13]  Fumihiko Imamura,et al.  TSUNAMI HAZARD AND CASUALTY ESTIMATION IN A COASTAL AREA THAT NEIGHBORS THE INDIAN OCEAN AND SOUTH CHINA SEA , 2012 .

[14]  Andrew R. McIntyre,et al.  Resource Review: Three Open Source Systems for Evolving Programs–Lilgp, ECJ and Grammatical Evolution , 2004, Genetic Programming and Evolvable Machines.

[15]  Joseph L. Smith,et al.  Agent Based Simulation of Human Movements During Emergency Evacuations of Facilities , 2008 .

[16]  Andrea Lodi,et al.  Mixed integer nonlinear programming tools: a practical overview , 2011, 4OR.

[17]  Yann Chevaleyre,et al.  Optimizing the Placement of Evacuation Signs on Road Network with Time and Casualties in Case of a Tsunami , 2012, 2012 IEEE 21st International Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises.

[18]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[19]  Ardiansyah,et al.  Tsunami Evacuation Simulation for Disaster Education and City Planning , 2012 .

[20]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[21]  Alexis Drogoul,et al.  Simulation of Emergency Evacuation of Pedestrians along the Road Networks in Nhatrang City , 2012, 2012 IEEE RIVF International Conference on Computing & Communication Technologies, Research, Innovation, and Vision for the Future.

[22]  Wayne A. Wickelgren,et al.  Speed-accuracy tradeoff and information processing dynamics , 1977 .

[23]  Michael J. North,et al.  Agent-based modeling and simulation: introductory tutorial , 2013, WSC '13.