Plausibility Verification For 3D Object Detectors Using Energy-Based Optimization

Environmental perception obtained via object detectors have no predictable safety layer encoded into their model schema, which creates the question of trustworthiness about the system's prediction. As can be seen from recent adversarial attacks, most of the current object detection networks are vulnerable to input tampering, which in the real world could compromise the safety of autonomous vehicles. The problem would be amplified even more when uncertainty errors could not propagate into the submodules, if these are not a part of the end-to-end system design. To address these concerns, a parallel module which verifies the predictions of the object proposals coming out of Deep Neural Networks are required. This work aims to verify 3D object proposals from MonoRUn model by proposing a plausibility framework that leverages cross sensor streams to reduce false positives. The verification metric being proposed uses prior knowledge in the form of four different energy functions, each utilizing a certain prior to output an energy value leading to a plausibility justification for the hypothesis under consideration. We also employ a novel two-step schema to improve the optimization of the composite energy function representing the energy model.

[1]  Syed Tahseen Raza Rizvi,et al.  Knowledge Augmented Machine Learning with Applications in Autonomous Driving: A Survey , 2022, ArXiv.

[2]  Kira Maag,et al.  False Negative Reduction in Video Instance Segmentation using Uncertainty Estimates , 2021, 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI).

[3]  Lu Xiong,et al.  MonoRUn: Monocular 3D Object Detection by Reconstruction and Uncertainty Propagation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Mohammed S. Khesbak Depth Camera and Laser Sensors Plausibility Evaluation for Small Size Obstacle Detection , 2021, 2021 18th International Multi-Conference on Systems, Signals & Devices (SSD).

[5]  Martin Danelljan,et al.  Accurate 3D Object Detection using Energy-Based Models , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Florian Geissler,et al.  A Plausibility-Based Fault Detection Method for High-Level Fusion Perception Systems , 2020, IEEE Open Journal of Intelligent Transportation Systems.

[7]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[8]  Sanjai Rayadurgam,et al.  Run-Time Assurance for Learning-Enabled Systems , 2020, NFM.

[9]  Hanno Gottschalk,et al.  Detection of False Positive and False Negative Samples in Semantic Segmentation , 2019, 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[10]  D. Cremers,et al.  DirectShape: Direct Photometric Alignment of Shape Priors for Visual Vehicle Pose and Shape Estimation , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Laura von Rueden,et al.  Informed Machine Learning – A Taxonomy and Survey of Integrating Prior Knowledge into Learning Systems , 2019, IEEE Transactions on Knowledge and Data Engineering.

[12]  Qiang Xu,et al.  nuScenes: A Multimodal Dataset for Autonomous Driving , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[14]  Klaus C. J. Dietmayer,et al.  3D Shape Reconstruction in Traffic Scenarios Using Monocular Camera and Lidar , 2016, ACCV Workshops.

[15]  Jörg Stückler,et al.  Joint Object Pose Estimation and Shape Reconstruction in Urban Street Scenes Using 3D Shape Priors , 2016, GCPR.

[16]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[17]  Ian D. Reid,et al.  Simultaneous Monocular 2D Segmentation, 3D Pose Recovery and 3D Reconstruction , 2012, ACCV.

[18]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Torsten Bertram,et al.  Object existence probability fusion using dempster-shafer theory in a high-level sensor data fusion architecture , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[20]  Yann LeCun,et al.  Synergistic Face Detection and Pose Estimation with Energy-Based Models , 2004, J. Mach. Learn. Res..

[21]  Martin Danelljan,et al.  Energy-Based Models for Deep Probabilistic Regression , 2020, ECCV.

[22]  Jose Luis Blanco,et al.  A tutorial on SE(3) transformation parameterizations and on-manifold optimization , 2012 .

[23]  Max Welling Donald,et al.  Products of Experts , 2007 .