A parallel-res GRU architecture and its application to road network traffic flow forecasting

Gated Recurrent Units (GRU) is an effective architecture on time series analysis. But deeper GRU models are more difficult to train and degrade rapidly. In this paper, we proposed a Parallel-Res GRU architecture for road network traffic flow forecasting. We explicitly reformulate the Parallel-Res Path as learning residual functions with reference to the input layer and output layer, instead of stacked plain GRU layers. And we construct multi-level residual architecture. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can improve accuracy from considerably increased depth and reduce the degradation. On UCI's PEM-SF dataset, we construct a 20 layers Parallel-Res GRU model for road network traffic flow forecasting, and the experiments indicate a considerable improvement comparing with normal stacked GRU models.

[1]  Steve B. Jiang,et al.  Nonlinear Systems Identification Using Deep Dynamic Neural Networks , 2016, ArXiv.

[2]  Nicholas G. Polson,et al.  Deep learning for short-term traffic flow prediction , 2016, 1604.04527.

[3]  Ramez Elmasri,et al.  Scalable deep traffic flow neural networks for urban traffic congestion prediction , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[4]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[5]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[6]  Li Li,et al.  Using LSTM and GRU neural network methods for traffic flow prediction , 2016, 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC).

[7]  Fei-Yue Wang,et al.  DeepTrend: A Deep Hierarchical Neural Network for Traffic Flow Prediction , 2017, ArXiv.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Fei-Yue Wang,et al.  Traffic Flow Prediction With Big Data: A Deep Learning Approach , 2015, IEEE Transactions on Intelligent Transportation Systems.

[10]  Alex Graves,et al.  Supervised Sequence Labelling with Recurrent Neural Networks , 2012, Studies in Computational Intelligence.

[11]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Peter C. Y. Chen,et al.  LSTM network: a deep learning approach for short-term traffic forecast , 2017 .

[13]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[14]  Huachun Tan,et al.  Short-term traffic flow forecasting with spatial-temporal correlation in a hybrid deep learning framework , 2016, ArXiv.

[15]  Said M. Easa,et al.  Supervised Weighting-Online Learning Algorithm for Short-Term Traffic Flow Prediction , 2013, IEEE Transactions on Intelligent Transportation Systems.

[16]  Tharam S. Dillon,et al.  Optimized Structure of the Traffic Flow Forecasting Model With a Deep Learning Approach , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Guy Pujolle,et al.  A Long Short-Term Memory Recurrent Neural Network Framework for Network Traffic Matrix Prediction , 2017, ArXiv.

[18]  Jungwon Lee,et al.  Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition , 2017, INTERSPEECH.

[19]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[20]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.