Performance Evaluation of Visual Object Detection for Moving Vehicle

With the rapid development of intelligent vehicle technology, the evaluation of the perception algorithms is crucial for safe driving of intelligent vehicles. This paper proposes a novel evaluation method for visual object detection of intelligent vehicles. The traditional evaluation methods are based on each frame of the video and treat all the objects equally. Distinguished from that, the proposed method applies the length of driving trajectory to evaluate the detection algorithm. Besides, the proposed method brings about the road region constraint, weight of safety and weight of early detection. The road region constraint ensures that only the objects relevant to intelligent vehicles are taken into account. The weight of safety for each object is determined based on its degree of relevance of the safe driving. The weight of early detection is set to evaluate whether the objects are detected in time or not. In experiments, the proposed evaluation method can reflect the relatively true detection ability of intelligent vehicles compared to the traditional methods.

[1]  Nanning Zheng,et al.  Artificial intelligence test: a case study of intelligent vehicles , 2018, Artificial Intelligence Review.

[2]  Nanning Zheng,et al.  Contour Guided Hierarchical Model for Shape Matching , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jiahuan Zhou,et al.  Fused Discriminative Metric Learning for Low Resolution Pedestrian Detection , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[5]  Paul Newman,et al.  1 year, 1000 km: The Oxford RobotCar dataset , 2017, Int. J. Robotics Res..

[6]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[7]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Chi Zhang,et al.  A Graded Offline Evaluation Framework for Intelligent Vehicle’s Cognitive Ability , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[9]  Pietro Perona,et al.  Pedestrian detection: A benchmark , 2009, CVPR.

[10]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[11]  Li Li,et al.  Intelligence Testing for Autonomous Vehicles: A New Approach , 2016, IEEE Transactions on Intelligent Vehicles.

[12]  Gang Hua,et al.  Can Visual Recognition Benefit from Auxiliary Information in Training? , 2014, ACCV.

[13]  Gang Hua,et al.  Multi-View Visual Recognition of Imperfect Testing Data , 2015, ACM Multimedia.

[14]  Ding Zhao,et al.  Accelerated Evaluation of Automated Vehicles in Car-Following Maneuvers , 2016, IEEE Transactions on Intelligent Transportation Systems.

[15]  Gang Hua,et al.  Auxiliary Training Information Assisted Visual Recognition , 2015, IPSJ Trans. Comput. Vis. Appl..