REAL-TIME COORDINATED SIGNAL CONTROL USING AGENTS WITH ONLINE REINFORCEMENT LEARNING

This paper introduces a multi-agent architecture for real-time coordinated signal control in an urban traffic network. The multi- agent architecture consists of three hierarchical layers of controller agents: intersection, zone and regional controllers. Each controller agent is implemented by applying artificial intelligence concepts namely fuzzy logic, neural network and evolutionary algorithm. Based on the fuzzy rule base, each individual controller agent recommends an appropriate signal policy at the end of each signal phase. These policies are later processed in a policy repository before being selected and implemented into the traffic network. In order to cope with the changing dynamics of the complex traffic processes within the network, an online reinforcement learning module is used to updating the knowledge base and inference rules of the agents. This multi-agent system with online reinforcement learning concept has been implemented in a network consisting of 25 signalized intersections, in a microscopic traffic simulator. Our initial test results have shown that the multi-agent system has improved the traffic condition in terms of average delay and total vehicle stoppage time, compared to that of a fixed-time traffic signal control

[1]  Gilberto Nakamiti,et al.  Fuzzy sets in distributed traffic control , 1996, Proceedings of IEEE 5th International Fuzzy Systems.

[2]  Reda Alhajj,et al.  Multiagent reinforcement learning using function approximation , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[3]  S. Chand,et al.  Self-organizing traffic control via fuzzy logic , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[4]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[5]  D. Levinson.  Detecting the Breakdown of Traffic Trb 2003 Annual Meeting Cd-rom Paper Revised from Original Submittal , 2002 .

[6]  L. Magdalena.  A rst approach to a Taxonomy of Fuzzy Neural Systems � , 1995 .

[7]  Klaus Bogenberger,et al.  A neuro-fuzzy algorithm for coordinated traffic responsive ramp metering , 2001, ITSC 2001. 2001 IEEE Intelligent Transportation Systems. Proceedings (Cat. No.01TH8585).

[8]  Harpreet Singh,et al.  Generating optimal adaptive fuzzy-neural models of dynamical systems with applications to control , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[9]  Yinghua Lin,et al.  A new approach to fuzzy-neural system modeling , 1995, IEEE Trans. Fuzzy Syst..

[10]  Jarkko Niittymäki,et al.  General fuzzy rule base for isolated traffic signal control‐rule formulation , 2001 .

[11]  Fernando Gomide,et al.  A neural network for fuzzy decision making problems , 1993, [Proceedings 1993] Second IEEE International Conference on Fuzzy Systems.

[12]  Hyung Lee-Kwang,et al.  Distributed and cooperative fuzzy controllers for traffic intersections group , 1999, IEEE Trans. Syst. Man Cybern. Part C.

[13]  W Reilly,et al.  HIGHWAY CAPACITY MANUAL 2000 , 1997 .

[14]  M Hoops,et al.  COORDINATED TRAFFIC MANAGEMENT IN THE GREATER AREA OF MUNICH , 1998 .

[15]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[16]  David J. Lovell,et al.  Autonomous Agents for Traffic Simulation and Control , 2001 .

[17]  Yukinori Kakazu,et al.  Genetic reinforcement learning for cooperative traffic signal control , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[18]  Yoshiki Uchikawa,et al.  On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm , 1992, IEEE Trans. Neural Networks.