Multi-3D-Object Tracking by Fusing RGB and 3D-LiDAR Data

Multiple object tracking (MOT) is a fundamental problem in the autonomous driving research community. Through accurate and efficient tracking, ego-vehicle can get the location velocity of surrounding objects and make a reasonable future motion planning. Different from most of the methods adopting the RGB or 3D-LiDAR data independently, this paper aims to track the perceived objects by fusing RGB and 3D-LiDAR data, the standard sensor configuration in current autonomous vehicles. Specifically, we firstly use Hungarian algorithm as a backbone model to associate the 3D point cloud of each object in adjacent frames. Then, we fully explore the appearance feature in RGB frame and geometrical feature in 3D point cloud to restrict the wrongly associate target IDs because of the interaction of near objects. We evaluate our method on the newly proposed BLVD dataset, and show the favorable performance.

[1]  W. Marsden.  I and J , 2012 .

[2]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[3]  Dongbing Gu,et al.  Using Unsupervised Deep Learning Technique for Monocular Visual Odometry , 2019, IEEE Access.

[4]  Mohan M. Trivedi,et al.  Dynamic Probabilistic Drivability Maps for Lane Change and Merge Driver Assistance , 2014, IEEE Transactions on Intelligent Transportation Systems.

[5]  Mubarak Shah,et al.  Deep Affinity Network for Multiple Object Tracking , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Wenhan Luo,et al.  Multiple object tracking: A literature review , 2014, Artif. Intell..

[7]  Markus Spies,et al.  Environment-Aware Multi-Target Tracking of Pedestrians , 2019, IEEE Robotics and Automation Letters.

[8]  Carlo Tomasi,et al.  Features for Multi-target Multi-camera Tracking and Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Daniel Cremers,et al.  Tracking the Trackers: An Analysis of the State of the Art in Multiple Object Tracking , 2017, ArXiv.

[10]  Iyiola E. Olatunji.  Human Activity Recognition for Mobile Robot , 2018, ArXiv.

[11]  Jianru Xue,et al.  BLVD: Building A Large-scale 5D Semantics Benchmark for Autonomous Driving , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[12]  Silvio Savarese,et al.  Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Silvio Savarese,et al.  Recurrent Autoregressive Networks for Online Multi-object Tracking , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[14]  Markus Maurer,et al.  Multi-Target Tracking using a 3D-Lidar sensor for autonomous vehicles , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).

[15]  Aubrey B. Poore,et al.  Some assignment problems arising from multiple target tracking , 2006, Math. Comput. Model..

[16]  Robert T. Collins,et al.  Multi-target Tracking by Lagrangian Relaxation to Min-cost Network Flow , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Raquel Urtasun,et al.  End-to-end Learning of Multi-sensor 3D Tracking by Detection , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Konrad Schindler,et al.  Continuous Energy Minimization for Multitarget Tracking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Mohan M. Trivedi,et al.  No Blind Spots: Full-Surround Multi-Object Tracking for Autonomous Vehicles Using Cameras and LiDARs , 2018, IEEE Transactions on Intelligent Vehicles.

[20]  Hang-Bong Kang,et al.  Multiple Objects Fusion Tracker Using a Matching Network for Adaptively Represented Instance Pairs , 2017, Sensors.

[21]  Changsheng Xu,et al.  Learning Multi-Task Correlation Particle Filters for Visual Tracking , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Bernard Ghanem,et al.  Efficient Tracking Proposals using 2D-3D Siamese Networks on LIDAR , 2019, ArXiv.

[23]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[24]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.