Short-term traffic flow forecasting : parametric and nonparametric approaches via emotional temporal difference learning

Information signal from real case and natural complex dynamical systems such as traffic flow are usually specified by irregular motions. Chaotic nonlinear dynamics approach is now the most powerful tool for scientists to deal with complexities in real cases, and neural networks and neuro-fuzzy models are widely used for their capabilities in nonlinear modeling of chaotic systems more than the traditional methods. As mentioned, the traffic flow conditions caused the forecasting values of traffic flow to lack robustness and accuracy. In this paper, the traffic flow forecasting is analyzed with emotional concepts and multi-agent systems (MASs) points of view as a new method in this field. The findings enabled the researchers to develop a newly object-oriented method of forecasting traffic flow. Its architecture is based on a temporal difference (TD) Q-learning with a neuro-fuzzy structure, which is the nonparametric approach. The performance of TD Q-learning is improved by emotional learning. The proposed method on the present conditions and the action of the system according to the criteria could forecast traffic signals so that the objectives are reached in minimum time. The ability of presented learning algorithm to prospect gains from future actions and obtain rewards from its past experiences allows emotional TD Q-learning algorithm to improve its decisions for the best possible actions. In addition, to study in a more practical situation, the neuro-fuzzy behaviors could be modeled by MAS. The proposed method (intelligent/nonparametric approach) is compared by parametric approach, autoregressive integrated moving average (ARIMA) method, which is implemented by multi-layer perceptron neural networks and called ARIMANN. Here, the ARIMANN is updated by backpropagation and temporal difference backpropagation for the first time. The simulation results revealed that the studied forecaster could discover the optimal forecasting by means of the Q-learning algorithm. Difficult to handle through parametric and classic methods, the real traffic flow signals used for fitting the algorithms is obtained from a two-lane street I-494 in Minnesota City.

[1]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[2]  Pattie Maes,et al.  Designing autonomous agents: Theory and practice from biology to engineering and back , 1990, Robotics Auton. Syst..

[3]  Xiaoyan Zhang,et al.  Short-Term Travel Time Prediction Using A Time-Varying Coecient Linear Model ? , 2001 .

[4]  Paul A. Fishwick,et al.  Feedforward Neural Nets as Models for Time Series Forecasting , 1993, INFORMS J. Comput..

[5]  Eleni I. Vlahogianni,et al.  Advanced Computational Approaches for Predicting Tourist Arrivals: the Case of Charter Air-Travel , 2010 .

[6]  Héctor Pomares,et al.  Hybridization of intelligent techniques and ARIMA models for time series prediction , 2008, Fuzzy Sets Syst..

[7]  C J Messer.  ADVANCED FREEWAY SYSTEM RAMP METERING STRATEGIES FOR TEXAS , 1993 .

[8]  Billy M. Williams,et al.  Comparison of parametric and nonparametric models for traffic flow forecasting , 2002 .

[9]  Eleni I. Vlahogianni,et al.  Statistical methods for detecting nonlinearity and non-stationarity in univariate short-term time-series of traffic volume , 2006 .

[10]  F. Rashidi,et al.  Emotional temporal difference learning based intelligent controller , 2003, Proceedings of 2003 IEEE Conference on Control Applications, 2003. CCA 2003..

[11]  Ping-Feng Pai,et al.  A hybrid ARIMA and support vector machines model in stock price forecasting , 2005 .

[12]  Pawan Lingras,et al.  Short-term traffic prediction on different types of roads with genetically designed regression and time delay neural network models , 2005 .

[13]  Eleni I. Vlahogianni,et al.  Memory properties and fractional integration in transportation time-series , 2009 .

[14]  Ying Lee,et al.  Sequential forecast of incident duration using Artificial Neural Network models. , 2007, Accident; analysis and prevention.

[15]  Milton S. Boyd,et al.  Designing a neural network for forecasting financial and economic time series , 1996, Neurocomputing.

[16]  Fan Wang,et al.  Short-Term Traffic Flow Prediction Based on Parallel Quasi-Newton Neural Network , 2009, 2009 International Conference on Measuring Technology and Mechatronics Automation.

[17]  Hamid R. Berenji,et al.  Learning and tuning fuzzy logic controllers through reinforcements , 1992, IEEE Trans. Neural Networks.

[18]  Mascha C. van der Voort,et al.  Combining kohonen maps with arima time series models to forecast traffic flow , 1996 .

[19]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[20]  Ji-xiang Yang,et al.  A Dish Parallel BP for Traffic Flow Forecasting , 2007 .

[21]  Héctor Pomares,et al.  Soft-computing techniques and ARMA model for time series prediction , 2008, Neurocomputing.

[22]  Matthew G. Karlaftis,et al.  A multivariate state space approach for urban traffic flow modeling and prediction , 2003 .

[23]  Eleni I. Vlahogianni,et al.  Temporal Evolution of Short‐Term Urban Traffic Flow: A Nonlinear Dynamics Approach , 2008, Comput. Aided Civ. Infrastructure Eng..

[24]  James W. Denton,et al.  How Good Are Neural Networks for Causal Forecasting , 1995 .

[25]  Jih-Jeng Huang,et al.  Model identification of ARIMA family using genetic algorithms , 2005, Appl. Math. Comput..

[26]  Eleni I. Vlahogianni,et al.  Short‐term traffic forecasting: Overview of objectives and methods , 2004 .

[27]  Chuntian Cheng,et al.  Combining a fuzzy optimal model with a genetic algorithm to solve multi-objective rainfall–runoff model calibration , 2002 .

[28]  Liangyue Cao,et al.  Predicting chaotic time series with wavelet networks , 1995 .

[29]  Babak Nadjar Araabi,et al.  Enhancing the Performance of Neurofuzzy Predictors by Emotional Learning Algorith , 2003, Informatica.

[30]  Eleni I. Vlahogianni,et al.  Spatio‐Temporal Short‐Term Urban Traffic Volume Forecasting Using Genetically Optimized Modular Networks , 2007, Comput. Aided Civ. Infrastructure Eng..

[31]  Xiaoyan Zhang,et al.  Short-term travel time prediction , 2003 .

[32]  Ibrahim El-Amin,et al.  Artificial neural networks as applied to long-term demand forecasting , 1999, Artif. Intell. Eng..

[33]  Paul A. Fishwick,et al.  Time series forecasting using neural networks vs. Box- Jenkins methodology , 1991, Simul..

[34]  Jenq-Neng Hwang,et al.  Temporal difference method for multi-step prediction: application to power load forecasting , 1991, Proceedings of the First International Forum on Applications of Neural Networks to Power Systems.

[35]  Michael Y. Hu,et al.  A simulation study of artificial neural networks for nonlinear time-series forecasting , 2001, Comput. Oper. Res..

[36]  Chuntian Cheng,et al.  A comparison of performance of several artificial intelligence , 2009 .

[37]  Terry R. Rakes,et al.  The effect of sample size and variability of data on the comparative performance of artificial neural networks and regression , 1998, Comput. Oper. Res..

[38]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[39]  Zhirui Ye,et al.  Short-Term Traffic Flow Forecasting Using Fuzzy Logic System Methods , 2008, J. Intell. Transp. Syst..

[40]  Eleni I. Vlahogianni,et al.  Optimized and meta-optimized neural networks for short-term traffic flow prediction: A genetic approach , 2005 .

[41]  John Guerard,et al.  Naïve, Arima, Nonparametric, Transfer Function, and VAR Models: A Comparison of Forecasting Performance , 2002 .

[42]  H. Brian Hwarng,et al.  Insights into neural-network forecasting of time series corresponding to ARMA(p,q) structures , 2001 .

[43]  Eleni I. Vlahogianni,et al.  Enhancing Predictions in Signalized Arterials with Information on Short-Term Traffic Flow Dynamics , 2009, J. Intell. Transp. Syst..

[44]  George S. Fishman,et al.  Estimating Sample Size in Computing Simulation Experiments , 1971 .

[45]  Shephali Aggarwal.  A Modern Approach to Artificial Intelligence Technologies , 2012 .

[46]  Bart De Moor,et al.  Identification of influential observations on total least squares estimates , 2002 .

[47]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[48]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[49]  Krzysztof J. Cios,et al.  Time series forecasting by combining RBF networks, certainty factors, and the Box-Jenkins model , 1996, Neurocomputing.

[50]  M. Posner.  The Brain and Emotion , 1999, Nature Medicine.

[51]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[52]  Baher Abdulhai,et al.  Short-Term Traffic Flow Prediction Using Neuro-Genetic Algorithms , 2002, J. Intell. Transp. Syst..

[53]  Weili Li,et al.  Calculation and Analysis of Double-Fed VSCF Induction Generator Operating Performance , 2007 .

[54]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[55]  Michael J Demetsky,et al.  SHORT-TERM TRAFFIC FLOW PREDICTION: NEURAL NETWORK APPROACH , 1994 .

[56]  Stewart W. Wilson,et al.  From Animals to Animats 5. Proceedings of the Fifth International Conference on Simulation of Adaptive Behavior , 1997 .

[57]  Michael J Demetsky,et al.  TRAFFIC FLOW FORECASTING: COMPARISON OF MODELING APPROACHES , 1997 .

[58]  Limin Jia,et al.  Road Traffic Flow Prediction with a Time-Oriented ARIMA Model , 2009, 2009 Fifth International Joint Conference on INC, IMS and IDC.

[59]  Man-Chun Tan,et al.  An Aggregation Approach to Short-Term Traffic Flow Prediction , 2009, IEEE Transactions on Intelligent Transportation Systems.

[60]  Susan Grant-Muller,et al.  Use of sequential learning for short-term traffic flow forecasting , 2001 .

[61]  Kwok-wing Chau,et al.  A hybrid adaptive time-delay neural network model for multi-step-ahead prediction of sunspot activity , 2006 .

[62]  Durdu Ömer Faruk.  A hybrid neural network and ARIMA model for water quality time series prediction , 2010, Eng. Appl. Artif. Intell..

[63]  Hojjat Adeli,et al.  Dynamic Wavelet Neural Network Model for Traffic Flow Forecasting , 2005 .

[64]  Xu Jianmin,et al.  Traffic Flow Prediction Based on Hybrid ARIMA and ANN Model , 2007 .

[65]  Chin-Teng Lin,et al.  Neural-Network-Based Fuzzy Logic Control and Decision System , 1991, IEEE Trans. Computers.

[66]  A. R. Cook,et al.  ANALYSIS OF FREEWAY TRAFFIC TIME-SERIES DATA BY USING BOX-JENKINS TECHNIQUES , 1979 .

[67]  Byungkyu Park,et al.  Hybrid Neuro-Fuzzy Application in Short-Term Freeway Traffic Volume Forecasting , 2002 .

[68]  Michael I. Jordan,et al.  MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES , 1996 .

[69]  John Patrick Aggleton,et al.  Emotion: Sensory Representation, Reinforcement, and the Temporal Lobe , 1990 .

[70]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[71]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .

[72]  F. Rashidi,et al.  COMBINING CONTEXT AND EMOTIONAL TEMPORAL DIFFERENCE LEARNING IN CONTROL ENGINEERING , 2005 .

[73]  A Khaki Sedigh,et al.  Control of Multivariable Systems Based on Emotional Temporal Difference Learning Controller , 2004 .

[74]  Mecit Cetin,et al.  Short-Term Traffic Flow Prediction with Regime Switching Models , 2006 .

[75]  I Okutani,et al.  Dynamic prediction of traffic volume through Kalman Filtering , 1984 .

[76]  R. W. Andrews,et al.  Arma-Based Confidence Intervals for Simulation Output Analysis , 1984 .

[77]  Nicholas R. Jennings,et al.  Intelligent agents: theory and practice , 1995, The Knowledge Engineering Review.

[78]  Christian Balkenius,et al.  A Computational Model of Context Processing , 2000 .