Two-Stream Networks for Lane-Change Prediction of Surrounding Vehicles

In highway scenarios, an alert human driver will typically anticipate early cut-in and cut-out maneuvers of surrounding vehicles using only visual cues. An automated system must anticipate these situations at an early stage too, to increase the safety and the efficiency of its performance. To deal with lane-change recognition and prediction of surrounding vehicles, we pose the problem as an action recognition/prediction problem by stacking visual cues from video cameras. Two video action recognition approaches are analyzed: two-stream convolutional networks and spatiotemporal multiplier networks. Different sizes of the regions around the vehicles are analyzed, evaluating the importance of the interaction between vehicles and the context information in the performance. In addition, different prediction horizons are evaluated. The obtained results demonstrate the potential of these methodologies to serve as robust predictors of future lane-changes of surrounding vehicles in time horizons between 1 and 2 seconds.

[1]  Wolfgang Rosenstiel,et al.  Object-oriented Bayesian networks for detection of lane change maneuvers , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[2]  Klaus-Dieter Kuhnert,et al.  When will it change the lane? A probabilistic regression approach for rarely occurring events , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[3]  Richard P. Wildes,et al.  Spatiotemporal Multiplier Networks for Video Action Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Dizan Vasquez,et al.  A survey on motion prediction and risk assessment for intelligent vehicles , 2014, ROBOMECH Journal.

[5]  Ignacio Parra,et al.  Vehicle trajectory and lane change prediction using ANN and SVM classifiers , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[6]  Ruigang Yang,et al.  The ApolloScape Open Dataset for Autonomous Driving and Its Application , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Sara McMains,et al.  Convolution neural network-based lane change intention prediction of surrounding vehicles for ACC , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[10]  Huijing Zhao,et al.  On-Road Vehicle Trajectory Collection and Scene-Based Lane Change Analysis: Part I , 2017, IEEE Transactions on Intelligent Transportation Systems.

[11]  Andreas Lawitzky,et al.  A Combined Model- and Learning-Based Framework for Interaction-Aware Maneuver Prediction , 2016, IEEE Transactions on Intelligent Transportation Systems.

[12]  Ignacio Parra,et al.  Experimental validation of lane-change intention prediction methodologies based on CNN and LSTM , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[13]  Torsten Bertram,et al.  Probabilistic Lane Change Prediction using Gaussian Process Neural Networks , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[14]  Klaus-Dieter Kuhnert,et al.  A lane change detection approach using feature ranking with maximized predictive power , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[15]  Lutz Eckstein,et al.  The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systems , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[16]  Klaus C. J. Dietmayer,et al.  A learning concept for behavior prediction in traffic situations , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[17]  Dongsuk Kum,et al.  The multilayer perceptron approach to lateral motion prediction of surrounding vehicles for autonomous vehicles , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[18]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[19]  Mohan M. Trivedi,et al.  Convolutional Social Pooling for Vehicle Trajectory Prediction , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  Junxiang Li,et al.  A Dynamic Bayesian Network for Vehicle Maneuver Prediction in Highway Driving Scenarios: Framework and Verification , 2019, Electronics.

[21]  Ignacio Parra,et al.  The PREVENTION dataset: a novel benchmark for PREdiction of VEhicles iNTentIONs , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[22]  Huijun Di,et al.  Manifold Learning for Lane-changing Behavior Recognition in Urban Traffic , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[23]  Andrew Zisserman,et al.  Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Yun Fu,et al.  Human Action Recognition and Prediction: A Survey , 2018, International Journal of Computer Vision.

[25]  Jason J. Corso,et al.  Predicting Future Lane Changes of Other Highway Vehicles using RNN-based Deep Models , 2018, ArXiv.

[26]  Huijing Zhao,et al.  On-Road Vehicle Trajectory Collection and Scene-Based Lane Change Analysis: Part II , 2017, IEEE Trans. Intell. Transp. Syst..

[27]  Mohan M. Trivedi,et al.  Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[28]  Masayoshi Tomizuka,et al.  INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps , 2019, ArXiv.

[29]  Ümit Özgüner,et al.  Trajectory prediction of a lane changing vehicle based on driver behavior estimation and classification , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).