Multisource Fusion for Robust Road Detection Using Online Estimated Reliabilities

For highly available automated driving, a robust road estimation is indispensable. In order to tackle the challenges of this task, many works employ a fusion of multiple sources, e.g., visually detected lane markings, leading vehicle, digital maps, etc. However, each source has certain advantages and drawbacks depending on the operational scenarios. Hence, the assumption made by many existing approaches that the sources always are equally reliable for the fusion process is inappropriate. Therefore, this work proposes a novel concept by incorporating reliabilities into the multisource fusion so that the road estimation task can alternately select only the most reliable sources. Thereby, the reliability for each source is estimated online using classifiers trained with the sensor measurements, the past performance, and the context. Using real data recordings, experimental results show that the presented reliability-aware fusion increases the availability of automated driving up to 7 percentage points compared to the average fusion.

[1]  Audun Jøsang,et al.  Subjective Logic: A Formalism for Reasoning Under Uncertainty , 2016 .

[2]  Pei Wang Confidence as Higher-Order Uncertainty , 2001, ISIPTA.

[3]  Isabelle Bloch Information combination operators for data fusion: a comparative review with classification , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[4]  Klaus C. J. Dietmayer,et al.  Road course estimation in occupancy grids , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[5]  Christoph Wiedemann,et al.  A new method for robust far-distance road course estimation in advanced driver assistance systems , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[6]  Stefan Gehrig,et al.  Dead reckoning and cartography using stereo vision for an autonomous car , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[7]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[8]  Rudolf Kruse,et al.  Fusion: General concepts and characteristics , 2001, Int. J. Intell. Syst..

[9]  R. Bosch Lane data fusion for driver assistance systems , 2004 .

[10]  Jian Feng,et al.  Reliability Information Fusion Based on Bayesian Generalized Mean Operator , 2012 .

[11]  Ronen Lerner,et al.  Recent progress in road and lane detection: a survey , 2012, Machine Vision and Applications.

[12]  Wiley E. Thompson,et al.  Adaptive selection of sensors based on individual performances in a multisensor environment , 1991, Defense, Security, and Sensing.

[13]  Frank Klawonn,et al.  Computational Intelligence: A Methodological Introduction , 2015, Texts in Computer Science.

[14]  Rudolf Kruse,et al.  A survey of performance measures to evaluate ego-lane estimation and a novel sensor-independent measure along with its applications , 2017, 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[15]  Lauro Snidaro,et al.  Integration of contextual information for tracking refinement , 2011, 14th International Conference on Information Fusion.

[16]  Galina L. Rogova,et al.  Reliability In Information Fusion : Literature Survey , 2004 .

[17]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[18]  Ángel F. García-Fernández,et al.  Bayesian Road Estimation Using Onboard Sensors , 2014, IEEE Transactions on Intelligent Transportation Systems.

[19]  Rudolf Kruse,et al.  Application of Graphical Models in the Automotive Industry , 2008, Computational Intelligence in Automotive Applications.

[20]  Pierre Borne,et al.  Context-dependent trust in data fusion within the possibility theory , 1996, 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).

[21]  Jörg Kaiser,et al.  Validity-Based Failure Algebra for Distributed Sensor Systems , 2013, 2013 IEEE 32nd International Symposium on Reliable Distributed Systems.

[22]  T. Denœux Conjunctive and disjunctive combination of belief functions induced by nondistinct bodies of evidence , 2008 .

[23]  Rudolf Kruse,et al.  The Transferable Belief Model for Belief Representation , 1996, Uncertainty Management in Information Systems.

[24]  Fakhri Karray,et al.  Multisensor data fusion: A review of the state-of-the-art , 2013, Inf. Fusion.

[25]  James Llinas,et al.  Handbook of Multisensor Data Fusion : Theory and Practice, Second Edition , 2008 .

[26]  Ljubo Vlacic,et al.  A Fault Tolerant Perception system for autonomous vehicles , 2016, 2016 35th Chinese Control Conference (CCC).

[27]  Yoshiko Kojima,et al.  CADAS: A multimodal advanced driver assistance system for normal urban streets based on road context understanding , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[28]  Liang Xiao,et al.  CRF based road detection with multi-sensor fusion , 2015, 2015 IEEE Intelligent Vehicles Symposium (IV).

[29]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[30]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[31]  Hugh F. Durrant-Whyte,et al.  Sensor Models and Multisensor Integration , 1988, Int. J. Robotics Res..

[32]  Rudolf Kruse,et al.  Graphical Models for Industrial Planning on Complex Domains , 2006, Decision Theory and Multi-Agent Planning.

[33]  Christian Borgelt,et al.  Graphical Models: Representations for Learning, Reasoning and Data Mining , 2009 .

[34]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[35]  Alberto Elfes,et al.  Environment-aware sensor fusion for obstacle detection , 2016, 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[36]  Chandrajit L. Bajaj,et al.  The algebraic degree of geometric optimization problems , 1988, Discret. Comput. Geom..

[37]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[38]  Torsten Bertram,et al.  Object existence probability fusion using dempster-shafer theory in a high-level sensor data fusion architecture , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[39]  Klaus C. J. Dietmayer,et al.  Towards autonomous self-assessment of digital maps , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[40]  Christoph Stiller,et al.  Efficient Road Scene Understanding for Intelligent Vehicles Using Compositional Hierarchical Models , 2015, IEEE Transactions on Intelligent Transportation Systems.

[41]  Mohan M. Trivedi,et al.  Learning multi-lane trajectories using vehicle-based vision , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[42]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[43]  Fuyuan Xiao,et al.  Modeling Sensor Reliability in Fault Diagnosis Based on Evidence Theory , 2016, Sensors.

[44]  Kruse Rudolf,et al.  Learning of lane information reliability for intelligent vehicles , 2016 .

[45]  Alain Appriou Situation assessment based on spatially ambiguous multisensor measurements , 2001, Int. J. Intell. Syst..

[46]  P. Smets,et al.  Assessing sensor reliability for multisensor data fusion within the transferable belief model , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[47]  Uwe Mönks Information fusion under consideration of conflicting input signals , 2016 .

[48]  Paul D. Gader,et al.  Context-Dependent Multisensor Fusion and Its Application to Land Mine Detection , 2010, IEEE Transactions on Geoscience and Remote Sensing.