AST-MTL: An Attention-Based Multi-Task Learning Strategy for Traffic Forecasting

Road traffic forecasting is crucial in Intelligent Transportation Systems (ITS). To achieve accurate results, it is necessary to model the dynamic nature and the complex non-linear dependencies governing traffic. The goal is particularly challenging when the prediction involves more than just one traffic variable. This paper proposes a novel multi-task learning model, called AST-MTL, to perform multi-horizon predictions of the traffic flow and speed at the road network scale. The strategy combines a multilayer fully-connected neural network (FNN) and a multi-head attention mechanism to learn related tasks while improving generalization performance. The model also includes the graph convolutional network (GCNs) and the gated recurrent unit network (GRUs) to extract the spatial and temporal features of traffic conditions. Our experiments employ new sets of GPS data, called OBU data, to perform traffic prediction in the freeway and urban contexts. The experimental results prove our model can effectively perform multi-horizon traffic forecasting for different types of roads and outperform state-of-the-art models.

[1]  Weiming Zhang,et al.  Interactive Temporal Recurrent Convolution Network for Traffic Prediction in Data Centers , 2018, IEEE Access.

[2]  Kejiang Ye,et al.  How to Build a Graph-Based Deep Learning Architecture in Traffic Domain: A Survey , 2020, ArXiv.

[3]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[4]  Yu Liu,et al.  T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction , 2018, IEEE Transactions on Intelligent Transportation Systems.

[5]  Rob J Hyndman.  Measuring forecast accuracy , 2014 .

[6]  Chenglu Wen,et al.  DeepSTD: Mining Spatio-Temporal Disturbances of Multiple Context Factors for Citywide Traffic Flow Prediction , 2020, IEEE Transactions on Intelligent Transportation Systems.

[7]  Geoff Boeing,et al.  OSMnx: New Methods for Acquiring, Constructing, Analyzing, and Visualizing Complex Street Networks , 2016, Comput. Environ. Urban Syst..

[8]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[9]  Leonard J. Tashman,et al.  Out-of-sample tests of forecasting accuracy: an analysis and review , 2000 .

[10]  Eleni I. Vlahogianni,et al.  Road Traffic Forecasting: Recent Advances and New Challenges , 2018, IEEE Intelligent Transportation Systems Magazine.

[11]  Shiliang Sun,et al.  Neural network multitask learning for traffic flow forecasting , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[12]  Mihaela van der Schaar,et al.  Attentive State-Space Modeling of Disease Progression , 2019, NeurIPS.

[13]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[14]  Zijian Liu,et al.  A deep learning based multitask model for network-wide traffic speed prediction , 2020, Neurocomputing.

[15]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[16]  Wenhu Chen,et al.  Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting , 2019, NeurIPS.

[17]  Zhiyong Cui,et al.  Deep Bidirectional and Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed Prediction , 2018, ArXiv.

[18]  Fei-Yue Wang,et al.  Traffic Flow Prediction With Big Data: A Deep Learning Approach , 2015, IEEE Transactions on Intelligent Transportation Systems.

[19]  Wenhao Huang,et al.  Deep Architecture for Traffic Flow Prediction: Deep Belief Networks With Multitask Learning , 2014, IEEE Transactions on Intelligent Transportation Systems.

[20]  Xiaogang Wang,et al.  Residual Attention Network for Image Classification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Michael Crawshaw,et al.  Multi-Task Learning with Deep Neural Networks: A Survey , 2020, ArXiv.

[22]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[23]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[24]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[25]  Gianluca Bontempi,et al.  A tutorial on network-wide multi-horizon traffic forecasting with deep learning , 2021, EDBT/ICDT Workshops.

[26]  Krzysztof Janowicz,et al.  Traffic transformer: Capturing the continuity and periodicity of time series for traffic forecasting , 2020, Trans. GIS.

[27]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[28]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[29]  Li Li,et al.  Using LSTM and GRU neural network methods for traffic flow prediction , 2016, 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC).

[30]  Antonio D. Masegosa,et al.  A Taxonomy of Traffic Forecasting Regression Problems From a Supervised Learning Perspective , 2019, IEEE Access.

[31]  Cory Stephenson,et al.  A Comparison of Loss Weighting Strategies for Multi task Learning in Deep Neural Networks , 2019, IEEE Access.

[32]  Kunpeng Zhang,et al.  A Multitask Learning Model for Traffic Flow and Speed Forecasting , 2020, IEEE Access.

[33]  Jimeng Sun,et al.  RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism , 2016, NIPS.

[34]  Guan Wei,et al.  A Summary of Traffic Flow Forecasting Methods , 2004 .

[35]  Nicolas Loeff,et al.  Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting , 2021, International Journal of Forecasting.

[36]  Sercan O. Arik,et al.  TabNet: Attentive Interpretable Tabular Learning , 2019, AAAI.

[37]  Fang Liu,et al.  An Improved Fuzzy Neural Network for Traffic Speed Prediction Considering Periodic Characteristic , 2017, IEEE Transactions on Intelligent Transportation Systems.

[38]  David M Levinson,et al.  Spatiotemporal short-term traffic forecasting using the network weight matrix and systematic detrending , 2019, Transportation Research Part C: Emerging Technologies.