Low Latency And Low-Level Sensor Fusion For Automotive Use-Cases

This work proposes a probabilistic low level automotive sensor fusion approach using LiDAR, RADAR and camera data. The method is stateless and directly operates on associated data from all sensor modalities. Tracking is not used, in order to reduce the object detection latency and create existence hypotheses per frame. The probabilistic fusion uses input from 3D and 2D space. An association method using a combination of overlap and distance metrics, avoiding the need for sensor synchronization is proposed. A Bayesian network executes the sensor fusion. The proposed approach is compared with a state of the art fusion system, which is using multiple sensors of the same modality and relies on tracking for object detection. Evaluation was done using low level sensor data recorded in an urban environment. The test results show that the low level sensor fusion reduces the object detection latency.

[1]  Qing Huo Liu,et al.  Radar High-Speed Target Detection Based on the Scaled Inverse Fourier Transform , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Zoran Nikolic,et al.  TDA2X, a SoC optimized for advanced driver assistance systems , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Miao Wang,et al.  Radar/Lidar sensor fusion for car-following on highways , 2011, The 5th International Conference on Automation, Robotics and Applications.

[4]  Nico Kämpchen,et al.  Feature-level fusion of laser scanner and video data for advanced driver assistance systems , 2007 .

[5]  Marcus Obst,et al.  Empirical evaluation of vehicular models for ego motion estimation , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[6]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  郭秉璋.  基於Histogram of Oriented Gradients之課堂舉手辨識研究 , 2012 .

[8]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Karsten Berns,et al.  A fuzzy approach to low level sensor fusion with limited system knowledge , 2010, 2010 13th International Conference on Information Fusion.

[10]  Klaus C. J. Dietmayer,et al.  Instantaneous lateral velocity estimation of a vehicle using Doppler radar , 2013, Proceedings of the 16th International Conference on Information Fusion.

[11]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[12]  K. Dietmayer,et al.  DATA SYNCHRONIZATION STRATEGIES FOR MULTI-SENSOR FUSION , 2003 .

[13]  B. Tibken,et al.  Low level sensor fusion for autonomous mobile robot navigation , 1999, IMTC/99. Proceedings of the 16th IEEE Instrumentation and Measurement Technology Conference (Cat. No.99CH36309).

[14]  Daniel Göhring,et al.  Online vehicle detection using deep neural networks and lidar based preselected image patches , 2016, 2016 IEEE Intelligent Vehicles Symposium (IV).

[15]  M. Mahlisch,et al.  Sensorfusion Using Spatio-Temporal Aligned Video and Lidar for Improved Vehicle Detection , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[17]  Juan I. Nieto,et al.  Motion-Based Calibration of Multimodal Sensor Extrinsics and Timing Offset Estimation , 2016, IEEE Transactions on Robotics.

[18]  M. Buehler,et al.  Feature-level fusion for free-form object tracking using laserscanner and video , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[19]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[20]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[21]  Yassine Ruichek,et al.  Representing and Tracking of Dynamics Objects using Oriented Bounding Box and Extended Kalman Filter , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[22]  Ljubo Mercep,et al.  Context-Centric Design of Automotive Human-Machine Interfaces , 2014 .

[23]  Lior Rokach,et al.  A survey of Clustering Algorithms , 2010, Data Mining and Knowledge Discovery Handbook.

[24]  Christos-Savvas Bouganis,et al.  FPGA-Based Parallel DBSCAN Architecture , 2014, ARC.

[25]  B. V. K. Vijaya Kumar,et al.  A multi-sensor fusion system for moving object detection and tracking in urban driving environments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Ting Yuan,et al.  Track level fusion of extended objects from heterogeneous sensors , 2016, 2016 19th International Conference on Information Fusion (FUSION).

[27]  Qi Yue,et al.  FPGA based accelerator for parallel DBSCAN algorithm , 2014 .

[28]  Klaus Dietmayer,et al.  Pedestrian recognition in urban traffic using a vehicle based multilayer laserscanner , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[29]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[31]  Li Ping,et al.  The Factor Graph Approach to Model-Based Signal Processing , 2007, Proceedings of the IEEE.

[32]  Sonya A. H. McMullen,et al.  Mathematical Techniques in Multisensor Data Fusion (Artech House Information Warfare Library) , 2004 .

[33]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.