Feature Sharing and Integration for Cooperative Cognition and Perception with Volumetric Sensors

The recent advancement in computational and communication systems has led to the introduction of high-performing neural networks and high-speed wireless vehicular communication networks. As a result, new technologies such as cooperative perception and cognition have emerged, addressing the inherent limitations of sensory devices by providing solutions for the detection of partially occluded targets and expanding the sensing range. However, designing a reliable cooperative cognition or perception system requires addressing the challenges caused by limited network resources and discrepancies between the data shared by different sources. In this paper, we examine the requirements, limitations, and performance of different cooperative perception techniques, and present an in-depth analysis of the notion of Deep Feature Sharing (DFS). We explore different cooperative object detection designs and evaluate their performance in terms of average precision. We use the Volony dataset for our experimental study. The results confirm that the DFS methods are significantly less sensitive to the localization error caused by GPS noise. Furthermore, the results attest that detection gain of DFS methods caused by adding more cooperative participants in the scenes is comparable to raw information sharing technique while DFS enables flexibility in design toward satisfying communication requirements.

[1]  Klaus C. J. Dietmayer,et al.  Towards Safe Autonomous Driving: Capture Uncertainty in the Deep Neural Network For Lidar 3D Vehicle Detection , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[2]  Klaus C. J. Dietmayer,et al.  Car2X-based perception in a high-level fusion architecture for cooperative perception systems , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[3]  Matthew C. Valenti,et al.  Adaptive content control for communication amongst cooperative automated vehicles , 2014, 2014 IEEE 6th International Symposium on Wireless Vehicular Communications (WiVeC 2014).

[4]  Bin Yang,et al.  PIXOR: Real-time 3D Object Detection from Point Clouds , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Gustavo de Veciana,et al.  Performance and Scaling of Collaborative Sensing and Networking for Automated Driving Applications , 2018, 2018 IEEE International Conference on Communications Workshops (ICC Workshops).

[6]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[7]  Emilio Frazzoli,et al.  Multivehicle Cooperative Driving Using Cooperative Perception: Design and Experimental Validation , 2015, IEEE Transactions on Intelligent Transportation Systems.

[8]  Yaser P. Fallah,et al.  Message content control for distributed map sharing in vehicle safety communications , 2017, 2017 IEEE 36th International Performance Computing and Communications Conference (IPCCC).

[9]  Klaus C. J. Dietmayer,et al.  Inter-vehicle object association for cooperative perception systems , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[10]  Horst-Michael Groß,et al.  Complex-YOLO: An Euler-Region-Proposal for Real-Time 3D Object Detection on Point Clouds , 2018, ECCV Workshops.

[11]  Qi Chen,et al.  F-cooper: feature based cooperative perception for autonomous vehicle edge computing system using 3D point clouds , 2019, SEC.

[12]  Jun Won Choi,et al.  Enhanced Object Detection in Bird's Eye View Using 3D Global Context Inferred From Lidar Point Data , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[13]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Tian Xia,et al.  Vehicle Detection from 3D Lidar Using Fully Convolutional Network , 2016, Robotics: Science and Systems.

[15]  Kazunori Ohno,et al.  Vehicle detection and localization on bird's eye view elevation images using convolutional neural network , 2017, 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR).

[16]  Amir Emad Marvasti,et al.  Bandwidth-Adaptive Feature Sharing for Cooperative LIDAR Object Detection , 2020, 2020 IEEE 3rd Connected and Automated Vehicles Symposium (CAVS).

[17]  Lars C. Wolf,et al.  Realizing collective perception in a vehicle , 2016, 2016 IEEE Vehicular Networking Conference (VNC).

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Fernando García,et al.  BirdNet: A 3D Object Detection Framework from LiDAR Information , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[20]  Yang Xu,et al.  Hybrid Sensing Data Fusion of Cooperative Perception for Autonomous Driving With Augmented Vehicular Reality , 2021, IEEE Systems Journal.

[21]  Ramesh Govindan,et al.  AVR: Augmented Vehicular Reality , 2018, MobiSys.

[22]  Marcelo H. Ang,et al.  Motion planning using cooperative perception on urban road , 2013, 2013 6th IEEE Conference on Robotics, Automation and Mechatronics (RAM).

[23]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Gustavo de Veciana,et al.  Deployment and Performance of Infrastructure to Assist Vehicular Collaborative Sensing , 2018, 2018 IEEE 87th Vehicular Technology Conference (VTC Spring).

[25]  Bin Cheng,et al.  Collaborative Perception for Automated Vehicles Leveraging Vehicle-to-Vehicle Communications , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[26]  Qing Yang,et al.  Cooper: Cooperative Perception for Connected Autonomous Vehicles Based on 3D Point Clouds , 2019, 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS).

[27]  Bernt Schiele,et al.  Learning Non-maximum Suppression , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Rui Guo,et al.  Cooperative LIDAR Object Detection via Feature Sharing in Deep Networks , 2020, 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall).

[29]  Raquel Urtasun,et al.  V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction , 2020, ECCV.

[30]  Sergio L. Netto,et al.  A Survey on Performance Metrics for Object-Detection Algorithms , 2020, 2020 International Conference on Systems, Signals and Image Processing (IWSSIP).

[31]  Peng Gao,et al.  Collaborative Localization for Occluded Objects in Connected Vehicular Platform , 2019, 2019 IEEE 90th Vehicular Technology Conference (VTC2019-Fall).

[32]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.