Towards Tactical Maneuver Detection for Autonomous Driving Based on Vision Only

The detection of tactical maneuvers performed by other traffic participants is a key component for modern autonomous driving systems. We present a novel vision-based approach to detect tactical maneuvers for vehicles, pedestrians, and bicycles. Our approach uses neural networks with temporal convolutions to incorporate temporal information into the detection. The training and evaluation of the presented architecture is performed on a video dataset obtained in a city and industrial environment spanning 3.5 hours. Our approach detects nine distinct maneuver classes with an average accuracy of 54.21%, with single detection accuracies of up to 88.17%.

[1]  P. Mathias,et al.  Infrastructure-based vehicle maneuver estimation at urban intersections , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[2]  Xiaojuan Qi,et al.  ICNet for Real-Time Semantic Segmentation on High-Resolution Images , 2017, ECCV.

[3]  Deva Ramanan,et al.  Attentional Pooling for Action Recognition , 2017, NIPS.

[4]  Volker Willert,et al.  Bayesian, maneuver-based, long-term trajectory prediction and criticality assessment for driver assistance systems , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[5]  Shih-Fu Chang,et al.  CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  I. Dagli,et al.  Cutting-in vehicle recognition for ACC systems- towards feasible situation analysis methodologies , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[7]  Thomas Brox,et al.  FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Klaus-Dieter Kuhnert,et al.  A lane change detection approach using feature ranking with maximized predictive power , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[9]  Ming Shao,et al.  A Multi-stream Bi-directional Recurrent Neural Network for Fine-Grained Action Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Rehder Tobias,et al.  Learning lane change intentions through lane contentedness estimation from demonstrated driving , 2016 .

[11]  Markus Maurer,et al.  Graph-based context representation, environment modeling and information aggregation for automated driving , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[12]  Horst-Michael Groß,et al.  Learning driving situations and behavior models from data , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[13]  Luc Van Gool,et al.  Two-Stream SR-CNNs for Action Recognition in Videos , 2016, BMVC.

[14]  Andrew Zisserman,et al.  Convolutional Two-Stream Network Fusion for Video Action Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Jitendra Malik,et al.  Contextual Action Recognition with R*CNN , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Andreas Geiger,et al.  Object scene flow for autonomous vehicles , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Thomas Brox,et al.  Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  Matthew J. Hausknecht,et al.  Beyond short snippets: Deep networks for video classification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Cordelia Schmid,et al.  Action Tubelet Detector for Spatio-Temporal Action Localization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Julian Eggert,et al.  Using Context Information and Probabilistic Classification for Making Extended Long-Term Trajectory Predictions , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[22]  Jonas Firl,et al.  Online maneuver recognition and multimodal trajectory prediction for intersection assistance using non-parametric regression , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[23]  Amaury Nègre,et al.  Probabilistic Analysis of Dynamic Scenes and Collision Risks Assessment to Improve Driving Safety , 2011, IEEE Intelligent Transportation Systems Magazine.

[24]  Christoph Stiller,et al.  Assessing map-based maneuver hypotheses using probabilistic methods and evidence theory , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[25]  Franz Kummert,et al.  General Behavior Prediction by a Combination of Scenario-Specific Models , 2014, IEEE Transactions on Intelligent Transportation Systems.

[26]  Cordelia Schmid,et al.  Multi-region Two-Stream R-CNN for Action Detection , 2016, ECCV.

[27]  Jonas Fredriksson,et al.  Rule-Based Highway Maneuver Intention Recognition , 2015, 2015 IEEE 18th International Conference on Intelligent Transportation Systems.

[28]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Jens Rieken,et al.  Behaviour recognition of traffic participants by using manoeuvre primitives for automated vehicles in urban traffic , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[30]  Klaus-Dieter Kuhnert,et al.  When will it change the lane? A probabilistic regression approach for rarely occurring events , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[31]  C. V. Jawahar,et al.  First Person Action Recognition Using Deep Learned Descriptors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[33]  Christian Wolf,et al.  Sequential Deep Learning for Human Action Recognition , 2011, HBU.

[34]  Jonathan P. How,et al.  Driver Behavior Classification at Intersections and Validation on Large Naturalistic Data Set , 2012, IEEE Transactions on Intelligent Transportation Systems.

[35]  Cordelia Schmid,et al.  Long-Term Temporal Convolutions for Action Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Véronique Berge-Cherfaoui,et al.  Vehicle trajectory prediction based on motion model and maneuver recognition , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Mathias Perrollaz,et al.  Learning-based approach for online lane change intention prediction , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).