DALC: Distributed Automatic LSTM Customization for Fine-Grained Traffic Speed Prediction

Over the past decade, several approaches have been introduced for short-term traffic prediction. However, providing fine-grained traffic prediction for large-scale transportation networks where numerous detectors are geographically deployed to collect traffic data is still an open issue. To address this issue, in this paper, we formulate the problem of customizing an LSTM model for a single detector into a finite Markov decision process and then introduce an Automatic LSTM Customization (ALC) algorithm to automatically customize an LSTM model for a single detector such that the corresponding prediction accuracy can be as satisfactory as possible and the time consumption can be as low as possible. Based on the ALC algorithm, we introduce a distributed approach called Distributed Automatic LSTM Customization (DALC) to customize an LSTM model for every detector in large-scale transportation networks. Our experiment demonstrates that the DALC provides higher prediction accuracy than several approaches provided by Apache Spark MLlib.

[1]  Yunpeng Wang,et al.  Long short-term memory neural network for traffic speed prediction using remote microwave sensor data , 2015 .

[2]  A. R. Cook,et al.  ANALYSIS OF FREEWAY TRAFFIC TIME-SERIES DATA BY USING BOX-JENKINS TECHNIQUES , 1979 .

[3]  Ming-Chang Lee,et al.  Performance evaluation of job schedulers on Hadoop YARN , 2016, Concurr. Comput. Pract. Exp..

[4]  Ugur Demiryurek,et al.  Deep Learning: A Generic Approach for Extreme Condition Traffic Forecasting , 2017, SDM.

[5]  Ramin Yahyapour,et al.  Hybrid Job-Driven Scheduling for Virtual MapReduce Clusters , 2016, IEEE Transactions on Parallel and Distributed Systems.

[6]  Hashem R Al-Masaeid,et al.  Short-Term Prediction of Traffic Volume in Urban Arterials , 1995 .

[7]  Gary A. Davis,et al.  Nonparametric Regression and Short‐Term Freeway Traffic Forecasting , 1991 .

[8]  L Poole David,et al.  Artificial Intelligence: Foundations of Computational Agents , 2010 .

[9]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[10]  Mascha C. van der Voort,et al.  Combining kohonen maps with arima time series models to forecast traffic flow , 1996 .

[11]  Fei-Yue Wang,et al.  Traffic Flow Prediction With Big Data: A Deep Learning Approach , 2015, IEEE Transactions on Intelligent Transportation Systems.

[12]  Zili Zhang,et al.  A distributed spatial-temporal weighted model on MapReduce for short-term traffic flow forecasting , 2016, Neurocomputing.

[13]  Ming-Chang Lee,et al.  RePAD: Real-time Proactive Anomaly Detection for Time Series , 2020, AINA.

[14]  Yi-Chang Chiu,et al.  Real-Time Freeway-Experienced Travel Time Prediction Using N-Curve and k Nearest Neighbor Methods , 2011 .

[15]  Peter C. Y. Chen,et al.  LSTM network: a deep learning approach for short-term traffic forecast , 2017 .

[16]  Billy M. Williams,et al.  Modeling and Forecasting Vehicular Traffic Flow as a Seasonal ARIMA Process: Theoretical Basis and Empirical Results , 2003, Journal of Transportation Engineering.