Grid-Based Object Tracking With Nonlinear Dynamic State and Shape Estimation

Object tracking is crucial for planning safe maneuvers of mobile robots in dynamic environments, in particular for autonomous driving with surrounding traffic participants. Multi-stage processing of sensor measurement data is thereby required to obtain abstracted high-level objects, such as vehicles. This also includes sensor fusion, data association, and temporal filtering. Often, an early-stage object abstraction is performed, which, however, is critical, as it results in information loss regarding the subsequent processing steps. We present a new grid-based object tracking approach that, in contrast, is based on already fused measurement data. The input is thereby pre-processed, without abstracting objects, by the spatial grid cell discretization of a dynamic occupancy grid, which enables a generic multi-sensor detection of moving objects. On the basis of already associated occupied cells, presented in our previous work, this paper investigates the subsequent object state estimation. The object pose and shape estimation thereby benefit from the freespace information contained in the input grid, which is evaluated to determine the current visibility of extracted object parts. An integrated object classification concept further enhances the assumed object size. For a precise dynamic motion state estimation, radar Doppler velocity measurements are integrated into the input data and processed directly on the object-level. Our approach is evaluated with real sensor data in the context of autonomous driving in challenging urban scenarios.

[1]  Stephan Matzka,et al.  A comparison of track-to-track fusion algorithms for automotive sensor fusion , 2008, 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[2]  Marcus Baum,et al.  Extended Target Tracking Using Gaussian Processes with High-Resolution Automotive Radar , 2018, 2018 21st International Conference on Information Fusion (FUSION).

[3]  Klaus C. J. Dietmayer,et al.  Tracking of Extended Objects with High-Resolution Doppler Radar , 2016, IEEE Transactions on Intelligent Transportation Systems.

[4]  Dirk Wollherr,et al.  Data Association for Grid-Based Object Tracking Using Particle Labeling , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[5]  Charles E. Thorpe,et al.  Simultaneous localization and mapping with detection and tracking of moving objects , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[6]  Klaus Dietmayer,et al.  Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Klaus C. J. Dietmayer,et al.  Environment Estimation with Dynamic Grid Maps and Self-Localizing Tracklets , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[8]  Luca Bertinetto,et al.  Fully-Convolutional Siamese Networks for Object Tracking , 2016, ECCV Workshops.

[9]  Ulrich Hofmann,et al.  Fusion of occupancy grid mapping and model based object tracking for driver assistance systems using laser and radar sensors , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[10]  Y. Bar-Shalom,et al.  On optimal track-to-track fusion , 1997, IEEE Transactions on Aerospace and Electronic Systems.

[11]  Xiaogang Wang,et al.  Visual Tracking with Fully Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[12]  Sergiu Nedevschi,et al.  Modeling and Tracking the Driving Environment With a Particle-Based Occupancy Grid , 2011, IEEE Transactions on Intelligent Transportation Systems.

[13]  Sebastian Thrun,et al.  Model based vehicle detection and tracking for autonomous urban driving , 2009, Auton. Robots.

[14]  Lukas Rummelhard,et al.  Hybrid sampling Bayesian Occupancy Filter , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[15]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[16]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Klaus C. J. Dietmayer,et al.  Deep Object Tracking on Dynamic Occupancy Grid Maps Using RNNs , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[18]  Ulrich Hofmann,et al.  360 Degree multi sensor fusion for static and dynamic obstacles , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[19]  Karl Granström,et al.  Extended Object Tracking: Introduction, Overview and Applications , 2016, ArXiv.

[20]  Christian Laugier,et al.  Bayesian Occupancy Filtering for Multitarget Tracking: An Automotive Application , 2006, Int. J. Robotics Res..

[21]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[22]  Christian Waldschmidt,et al.  Instantaneous Actual Motion Estimation with a Single High-Resolution Radar Sensor , 2018, 2018 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM).

[23]  Klaus C. J. Dietmayer,et al.  Occupancy grid map-based extended object tracking , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[24]  Arthur P. Dempster,et al.  Upper and Lower Probabilities Induced by a Multivalued Mapping , 1967, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[25]  Torsten Bertram,et al.  Track-to-Track Fusion With Asynchronous Sensors Using Information Matrix Fusion for Surround Environment Perception , 2012, IEEE Transactions on Intelligent Transportation Systems.

[26]  Gerd Wanielik,et al.  Comparison and evaluation of advanced motion models for vehicle tracking , 2008, 2008 11th International Conference on Information Fusion.

[27]  Dirk Wollherr,et al.  Object tracking based on evidential dynamic occupancy grids in urban environments , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[28]  Sascha Wirges,et al.  Object Detection and Classification in Occupancy Grid Maps Using Deep Convolutional Networks , 2018, 2018 21st International Conference on Intelligent Transportation Systems (ITSC).

[29]  Dirk Wollherr,et al.  Grid-Based Environment Estimation Using Evidential Mapping and Particle Tracking , 2018, IEEE Transactions on Intelligent Vehicles.

[30]  Franz Korf,et al.  Object tracking and dynamic estimation on evidential grids , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[31]  Georg Tanzmeister,et al.  Spatiotemporal alignment for low-level asynchronous data fusion with radar sensors in grid-based tracking and mapping , 2016, 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[32]  Michael Aeberhard,et al.  Object-level fusion for surround environment perception in automated driving applications , 2017 .

[33]  Johann Marius Zöllner,et al.  Fully convolutional neural networks for dynamic object detection in grid maps , 2017, 2017 IEEE Intelligent Vehicles Symposium (IV).

[34]  Klaus C. J. Dietmayer,et al.  Offline Object Extraction from Dynamic Occupancy Grid Map Sequences , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[35]  Klaus C. J. Dietmayer,et al.  A random finite set approach for dynamic occupancy grid maps with real-time application , 2016, Int. J. Robotics Res..

[36]  Yuan Ting,et al.  Object tracking with de-autocorrelation scheme for a dynamic occupancy gridmap system , 2016 .

[37]  Véronique Berge-Cherfaoui,et al.  Credibilist occupancy grids for vehicle perception in dynamic environments , 2011, 2011 IEEE International Conference on Robotics and Automation.

[38]  Yaakov Bar-Shalom,et al.  A note on "book review tracking and data fusion: A handbook of algorithms" [Authors' reply] , 2013 .

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[41]  Dominik Nuss,et al.  Representation of Fused Environment Data , 2015 .

[42]  Klaus C. J. Dietmayer,et al.  Fusion of laser and radar sensor data with a sequential Monte Carlo Bayesian occupancy filter , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[43]  Trung-Dung Vu,et al.  Online Localization and Mapping with Moving Object Tracking in Dynamic Outdoor Environments , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[44]  Klaus C. J. Dietmayer,et al.  Instantaneous full-motion estimation of arbitrary objects using dual Doppler radar , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[45]  Bohyung Han,et al.  Learning Multi-domain Convolutional Neural Networks for Visual Tracking , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Christian Laugier,et al.  The Bayesian Occupation Filter , 2008, Probabilistic Reasoning and Decision Making in Sensory-Motor Systems.

[47]  Dirk Wollherr,et al.  Evidential Grid-Based Tracking and Mapping , 2017, IEEE Transactions on Intelligent Transportation Systems.

[48]  LI X.RONG,et al.  Survey of maneuvering target tracking. Part I. Dynamic models , 2003 .

[49]  Trung-Dung Vu,et al.  Grid-based localization and online mapping with moving objects detection and tracking: new results , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[50]  Thorsten Graf,et al.  Improved object tracking from detailed shape estimation using object local grid maps with stereo , 2013, 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013).