A Hybrid Learning Algorithm for Generating Multi-Agent Daily Activity Plans

This paper proposes a hybrid learning algorithm based on the competing risk model and the cross entropy method for generating complete one-day activity plans for multi-agent traffic simulations. An agent’s activity plan generation process is modeled using the Markov decision process. As generating complete activity plans of agents using a reinforcement learning approach is computationally expensive and inefficient, we propose a hybrid method that first estimates the activity type of agents and the scheduled ending time sequences from empirical data based on a competing risk model. The activity plans obtained are then completed by the cross entropy method for the optimal destination choice learning of agents. The performance of the proposed method is compared with the Q-learning algorithm. The numerical result shows that the proposed method generates consistent daily activity plans for multi-agent traffic simulations.

[1]  J. Kalbfleisch,et al.  The Statistical Analysis of Failure Time Data , 1980 .

[2]  Kay W. Axhausen,et al.  Continuous Activity Planning for Continuous Traffic Simulation , 2011 .

[3]  Davy Janssens,et al.  Simulation of sequential data: An enhanced reinforcement learning approach , 2009, Expert Syst. Appl..

[4]  Davy Janssens,et al.  The development of an adapted Markov chain modelling heuristic and simulation framework in the context of transportation research , 2005, Expert Syst. Appl..

[5]  Marcel Rieser,et al.  Agent-Oriented Coupling of Activity-Based Demand Generation with Multiagent Traffic Simulation , 2007 .

[6]  Jean-Patrick Lebacque,et al.  A cross entropy based multiagent approach for multiclass activity chain modeling and simulation , 2013 .

[7]  Tai-Yu Ma,et al.  Modèle dynamique de transport basé sur les activités , 2007 .

[8]  C H Chang-Hyeon Joh,et al.  Measuring and predicting adaptation in multidimensional activity-travel patterns , 2004 .

[9]  Kay W. Axhausen,et al.  New approaches to generating comprehensive all-day activity-travel schedules , 2009 .

[10]  Jin-Hyuk Chung,et al.  An Empirical Investigation of the Underlying Behavioral Processes of Trip Chaining , 2002 .

[11]  Jean-Patrick Lebacque,et al.  A Cross Entropy Based Multi-Agent Approach to Traffic Assignment Problems , 2009 .

[12]  Eric Cornelis,et al.  Multistate Nonhomogeneous Semi-Markov Model of Daily Activity Type, Timing, and Duration Sequence , 2009 .

[13]  K. Nagel,et al.  Generating complete all-day activity plans with genetic algorithms , 2005 .

[14]  Tai-Yu Ma.  A Cross Entropy Multiagent Learning Algorithm for Solving Vehicle Routing Problems with Time Windows , 2011, ICCL.

[15]  Kay W. Axhausen,et al.  A GA-based household scheduler , 2005 .

[16]  Torsten Hägerstraand.  WHAT ABOUT PEOPLE IN REGIONAL SCIENCE , 1970 .

[17]  C. Gawron,et al.  An Iterative Algorithm to Determine the Dynamic User Equilibrium in a Traffic Simulation Model , 1998 .

[18]  Masao Kuwahara,et al.  Decomposition of the reactive dynamic assignments with queues for a many-to-many origin-destination pattern , 1997 .

[19]  Hjp Harry Timmermans,et al.  A learning-based transportation oriented simulation system , 2004 .

[20]  David Charypar,et al.  Q-Learning for Flexible Learning of Daily Activity Plans , 2005 .

[21]  Franziska Klügl-Frohnmeyer,et al.  Multi Agent System Simulation for the Generation of Individual Activity Programs , 2005, Applications of Agent Technology in Traffic and Transportation.

[22]  R. Rubinstein.  The Cross-Entropy Method for Combinatorial and Continuous Optimization , 1999 .