Emotional temporal difference Q-learning signals in multi-agent system cooperation: real case studies

Chaotic non-linear dynamics approach is now the most powerful tool for scientists to deal with complexities in real cases; and artificial neural networks and neuro-fuzzy models are widely used for their capabilities in non-linear modelling of chaotic systems. Chaos, uncertain behaviours, demanding fluctuation, complexity of the traffic flow situations and the problems with those methods, however, caused the forecasting traffic flow values to lack robustness and precision. In this study, the traffic flow forecasting is analysed by emotional concepts and multi-agent systems (MASs) points of view as a new method. Its architecture is based on a temporal difference (TD) Q -learning with a neuro-fuzzy structure. The performance of TD Q -learning method is improved by emotional learning. The concept of emotional TD Q -learning method is discussed for the first time in this study. The forecasting algorithm which uses the Q -learning algorithm is capable of finding the optimal forecasting approach as the one obtained by the reinforcement learning. In addition, in order to study in a more practical situation, the neuro-fuzzy behaviours can be modelled by MAS. The real traffic flow signals used for fitting the proposed methods are obtained from interstate I-494 in Minnesota City in USA and the E17 motorway Gent-Antwerp in Belgium.

[1]  Zhirui Ye,et al.  Short-Term Traffic Flow Forecasting Using Fuzzy Logic System Methods , 2008, J. Intell. Transp. Syst..

[2]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[3]  Bart van Arem,et al.  Recent advances and applications in the field of short-term traffic forecasting. , 1997 .

[4]  John Patrick Aggleton,et al.  Emotion: Sensory Representation, Reinforcement, and the Temporal Lobe , 1990 .

[5]  F. Rashidi,et al.  Emotional temporal difference learning based intelligent controller , 2003, Proceedings of 2003 IEEE Conference on Control Applications, 2003. CCA 2003..

[6]  Eleni I. Vlahogianni,et al.  Spatio‐Temporal Short‐Term Urban Traffic Volume Forecasting Using Genetically Optimized Modular Networks , 2007, Comput. Aided Civ. Infrastructure Eng..

[7]  E. Chang,et al.  Traffic flow forecasting neural networks based on exponential smoothing method , 2011, 2011 6th IEEE Conference on Industrial Electronics and Applications.

[8]  Christian Balkenius,et al.  A Computational Model of Context Processing , 2000 .

[9]  Susan Grant-Muller,et al.  Use of sequential learning for short-term traffic flow forecasting , 2001 .

[10]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[11]  Ji-xiang Yang,et al.  A Dish Parallel BP for Traffic Flow Forecasting , 2007 .

[12]  Baher Abdulhai,et al.  Short-Term Traffic Flow Prediction Using Neuro-Genetic Algorithms , 2002, J. Intell. Transp. Syst..

[13]  Yuanchang Xie,et al.  A Wavelet Network Model for Short-Term Traffic Volume Forecasting , 2006, J. Intell. Transp. Syst..

[14]  Baher Abdulhai,et al.  Forecasting of short-term traffic-flow based on improved neurofuzzy models via emotional temporal difference learning algorithm , 2012, Eng. Appl. Artif. Intell..

[15]  Ying Lee,et al.  Sequential forecast of incident duration using Artificial Neural Network models. , 2007, Accident; analysis and prevention.

[16]  W. Y. Szeto,et al.  Multivariate Traffic Forecasting Technique Using Cell Transmission Model and SARIMA Model , 2009 .

[17]  Yiannis Kamarianakis,et al.  Modeling Traffic Volatility Dynamics in an Urban Network , 2005 .

[18]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[19]  Eleni I. Vlahogianni,et al.  Short‐term traffic forecasting: Overview of objectives and methods , 2004 .

[20]  Biswajit Basu,et al.  Multivariate Short-Term Traffic Flow Forecasting Using Time-Series Analysis , 2009, IEEE Transactions on Intelligent Transportation Systems.

[21]  Sherif Ishak,et al.  Optimizing traffic prediction performance of neural networks under various topological, input, and traffic condition settings , 2004 .

[22]  Michael Wooldridge,et al.  Property-based Slicing for Agent Verification , 2009, J. Log. Comput..

[23]  Alexander Skabardonis,et al.  A spatial queuing model for the emergency vehicle districting and location problem , 2009 .