Visual Perception Based Situation Analysis of Traffic Scenes for Autonomous Driving Applications

The major challenges for analyzing the situation of traffic scenes include defining proper metrics and achieving computation efficiency. This paper proposes two new situation metrics, a multimodality scene model, and a metrics computing network for traffic scene analysis. The main novelty is threefold. (1) The planning complexity and perception complexity are proposed as the situation metrics of traffic senes. (2) A multimodality model is proposed to describe traffic scenes, which combines the information of the static environment, dynamic objects, and ego-vehicle. (3) A deep neural network (DNN) based computing network is proposed to compute the two situation metrics based on scene models. Using the Nuscenes dataset, a high-level dataset for traffic scene analysis is developed to validate the scene model and the situation metrics computing network. The experiment results show that the proposed scene model is effective for situation analysis and the proposed situation metrics computing network outperforms than traditional CNN methods.

[1]  Sejoon Lim,et al.  Development of Complexity Index and Predictions of Accident Risks for Mixed Autonomous Driving Levels , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[2]  Jing Li,et al.  Graph CNNs for Urban Traffic Passenger Flows Prediction , 2018, 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[3]  Kenzo Lobos-Tsunekawa,et al.  Collision Avoidance for Indoor Service Robots Through Multimodal Deep Reinforcement Learning , 2019, RoboCup.

[4]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[5]  Alejandro Ribeiro,et al.  Graph Policy Gradients for Large Scale Unlabeled Motion Planning with Constraints , 2019, ArXiv.

[6]  Koyel Banerjee,et al.  Online Camera LiDAR Fusion and Object Detection on Hybrid Data for Autonomous Driving , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[7]  Ernst D. Dickmanns,et al.  Dynamic Vision for Perception and Control of Motion , 2007 .

[8]  Zhiwei Li,et al.  Multi-Feature-Based Road Complexity Calculation Model for the Evaluation of Unmanned Ground Vehicles , 2018, 2018 10th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC).

[9]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Ruigang Yang,et al.  The ApolloScape Dataset for Autonomous Driving , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Soumith Chintala,et al.  A MultiPath Network for Object Detection , 2016, BMVC.

[14]  Hui Kong,et al.  Road Detection through CRF based LiDAR-Camera Fusion , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[15]  Hongdong Li,et al.  Unsupervised Deep Epipolar Flow for Stationary or Dynamic Scenes , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Xiaokun Li,et al.  A hidden Markov model framework for traffic event detection using video features , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[17]  Bin Yang,et al.  Deep Continuous Fusion for Multi-sensor 3D Object Detection , 2018, ECCV.

[18]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[20]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[23]  Ilja Radusch,et al.  Early Fusion of Camera and Lidar for robust road detection based on U-Net FCN , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[24]  Marcelo H. Ang,et al.  Situation-aware decision making for autonomous driving on urban road using online POMDP , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[25]  Chi Zhang,et al.  Traffic Sensory Data Classification by Quantifying Scenario Complexity , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[26]  François Chollet,et al.  Xception: Deep Learning with Depthwise Separable Convolutions , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  M. Elif Karsligil,et al.  Severity classification of abnormal traffic events at intersections , 2011, 2011 18th IEEE International Conference on Image Processing.