A Hybrid Algorithm for Fast Learning Individual Daily Activity Plans for Multiagent Transportation Simulation

This paper propose a hybrid learning algorithm based on the competing risk duration model and the cross entropy method for generating complete all-day activity plan in multiagent transportation simulation. We formulate agent's activity scheduling problem as a sequential Markov decision process. By initially generating individual's activity type and duration sequence from empirical data based on the competing risk duration model, the obtained plans can be efficiently improved by reinforcement learning technique towards near-optimal activity plan. We apply the cross entropy method to efficiently learn near-optimal activity plan. The numerical result shows that the proposed method generates consistent daily activity plans for multiagent transportation simulation.

[1]  Masao Kuwahara,et al.  Decomposition of the reactive dynamic assignments with queues for a many-to-many origin-destination pattern , 1997 .

[2]  Hjp Harry Timmermans,et al.  A learning-based transportation oriented simulation system , 2004 .

[3]  Kay W. Axhausen,et al.  A Simultaneous Simulation of Activity Chains and Traffic Flow , 1988 .

[4]  Torsten Hägerstraand.  WHAT ABOUT PEOPLE IN REGIONAL SCIENCE , 1970 .

[5]  David Charypar,et al.  Q-Learning for Flexible Learning of Daily Activity Plans , 2005 .

[6]  J. Kalbfleisch,et al.  The Statistical Analysis of Failure Time Data: Kalbfleisch/The Statistical , 2002 .

[7]  Eric Cornelis,et al.  Multistate Nonhomogeneous Semi-Markov Model of Daily Activity Type, Timing, and Duration Sequence , 2009 .

[8]  Jean-Patrick Lebacque,et al.  A cross entropy based multiagent approach for multiclass activity chain modeling and simulation , 2013 .

[9]  Tai-Yu Ma,et al.  Modèle dynamique de transport basé sur les activités , 2007 .

[10]  Laurence L. George,et al.  The Statistical Analysis of Failure Time Data , 2003, Technometrics.

[11]  Franziska Klügl-Frohnmeyer,et al.  Multi Agent System Simulation for the Generation of Individual Activity Programs , 2005, Applications of Agent Technology in Traffic and Transportation.

[12]  R. Rubinstein.  The Cross-Entropy Method for Combinatorial and Continuous Optimization , 1999 .

[13]  Jin-Hyuk Chung,et al.  An Empirical Investigation of the Underlying Behavioral Processes of Trip Chaining , 2002 .

[14]  F. Koppelman,et al.  Activity-Based Modeling of Travel Demand , 2003 .

[15]  Kay W. Axhausen,et al.  Continuous Activity Planning for a Continuous Traffic Simulation , 2010 .

[16]  Kay W. Axhausen,et al.  New approaches to generating comprehensive all-day activity-travel schedules , 2009 .

[17]  Marcel Rieser,et al.  Agent-Oriented Coupling of Activity-Based Demand Generation with Multiagent Traffic Simulation , 2007 .

[18]  C H Chang-Hyeon Joh,et al.  Measuring and predicting adaptation in multidimensional activity-travel patterns , 2004 .

[19]  K. Nagel,et al.  Generating complete all-day activity plans with genetic algorithms , 2005 .

[20]  Tai-Yu Ma.  A Cross Entropy Multiagent Learning Algorithm for Solving Vehicle Routing Problems with Time Windows , 2011, ICCL.

[21]  Kay W. Axhausen,et al.  A GA-based household scheduler , 2005 .

[22]  C. Gawron,et al.  An Iterative Algorithm to Determine the Dynamic User Equilibrium in a Traffic Simulation Model , 1998 .