Real-time Traffic Data Prediction with Basic Safety Messages using Kalman-Filter based Noise Reduction Model and Long Short-term Memory Neural Network

The accurate prediction of traffic data, such as average speed and average space-headway between vehicles in real-time is important for route planning and scheduling to reduce travel time, for future traffic condition assessment, and for vehicle’s energy optimization to reduce fuel consumption. Unfortunately, the stochastic change of traffic flow over time greatly complicates the development of such a real-time traffic data prediction method. With the development of Connected Vehicle (CV) technology, temporal variation of roadway traffic can be captured by sharing Basic Safety Messages (BSMs) from each vehicle using the communication between vehicles as well as with transportation roadside infrastructures (e.g., traffic signal, roadside unit) and traffic management centers. However, the penetration of connected vehicles in the near future will be limited. BSMs from limited CVs could provide an inaccurate estimation of current speed or space-headway. This inaccuracy in the estimated current average speed and average space-headway data is termed as noise. This noise in the traffic data significantly reduces the prediction accuracy of a machinelearning model, such as the accuracy of long short-term memory (LSTM) model in predicting traffic condition. To improve the real-time prediction accuracy with low penetration of CVs, we developed a traffic data prediction model that combines the LSTM with a noise reduction model. We first investigated the standard Kalman filter and Kalman filter based Rauch–Tung–Striebel (RTS) noise reduction techniques to reduce the noise from the current traffic data measured from BSMs. We next used the filtered data to evaluate the performance of the LSTM prediction model. The average speed and space-headway used in this study were generated from the Enhanced Next Generation Simulation (NGSIM) dataset, which contains vehicle trajectory data for every onetenth of a second. Compared to a baseline LSTM model without any noise reduction, for 5% penetration of CVs, the analyses revealed that combined LSTM/RTS model reduced the mean absolute percentage error (MAPE) from 19% to 5% for speed prediction and from 27% to 9% for space-headway prediction. The overall reduction of MAPE value ranged from 1% to 14% for speed and 2% to 18% for space headway prediction compared to the baseline model. The statistical significance test with a 95% confidence interval confirmed no significant difference in predicted average speed and average space headway using this LSTM/RTS combination for different CV penetration rates.